Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

/per-rule test doesn't catch Automatus error #252

Closed
mildas opened this issue Aug 6, 2024 · 1 comment
Closed

/per-rule test doesn't catch Automatus error #252

mildas opened this issue Aug 6, 2024 · 1 comment

Comments

@mildas
Copy link
Contributor

mildas commented Aug 6, 2024

/per-rule test catches only errors/failures from test scenarios. However, it Automatus error occurs, it's not caught and test finishes as pass:

2024-08-02 11:04:53 test.py:92: running: ./automatus.py rule --libvirt qemu:///system contest --product rhel9 --remediate-using oscap ...
Setting console output to log level INFO
INFO - The base image option has not been specified, choosing libvirt-based test environment.
INFO - Logging into /root/content/tests/logs/rule-custom-2024-08-02-1104/test_suite.log
2024-08-02 11:04:56 test.py:118: using automatus log dir: /root/content/tests/logs/rule-custom-2024-08-02-1104
WARNING - Script wrong_value_including_relative_path.fail.sh is not applicable on given platform
WARNING - Script correct_value_multiple_includes.pass.sh is not applicable on given platform
WARNING - Script param_conflict_multiple_includes.fail.sh is not applicable on given platform
WARNING - Script wrong_value_different_includings.fail.sh is not applicable on given platform
WARNING - Script correct_value_including_relative_path.pass.sh is not applicable on given platform
WARNING - Script param_conflict_including_relative_path.fail.sh is not applicable on given platform
WARNING - Script correct_entry.pass.sh is not applicable on given platform
WARNING - Script missing_entry.fail.sh is not applicable on given platform
WARNING - Script wrong_control.fail.sh is not applicable on given platform
WARNING - Script correct_value_etcdefault_dir.pass.sh is not applicable on given platform
WARNING - Script wrong_value.fail.sh is not applicable on given platform
WARNING - Script correct_value_noupdate.fail.sh is not applicable on given platform
WARNING - Script correct_value_etcdefault_dir_noupdate.fail.sh is not applicable on given platform
WARNING - Script wrong_value_etcdefault_dir.fail.sh is not applicable on given platform
WARNING - Script arg_not_there_grubenv.fail.sh is not applicable on given platform
WARNING - Script wrong_value_etcdefault.fail.sh is not applicable on given platform
WARNING - Script missing_profiles.fail.sh is not applicable on given platform
WARNING - Script correct_value.pass.sh is not applicable on given platform
WARNING - Script wrong_value.fail.sh is not applicable on given platform
WARNING - Script fedora_key.fail.sh is not applicable on given platform
ERROR - Terminating due to error: Unable to upload test scripts: Cannot extract data tarball /root/ssgts/tmpi9t3x9wf.tar.gz.
WARNING - Nothing has been tested!
2024-08-02 11:06:32 test.py:167: lib.virt.Guest.booted:641: shutting down contest
2024-08-02 11:06:33 test.py:167: lib.virt.wait_for_domstate:743: waiting for contest to be shut off for 300sec
2024-08-02 11:06:34 test.py:169: lib.waive.collect_waivers:141: using /builds/security-compliance/productization/runcontest-XTvqK4/datadir/task12/plans/daily/discover/default-0/tests/conf/waivers for waiving
2024-08-02 11:06:34 test.py:169: lib.results.report_plain:182: PASS /
@mildas
Copy link
Contributor Author

mildas commented Aug 6, 2024

Closing as it's duplicate of #179

@mildas mildas closed this as completed Aug 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant