You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Some tests, like test/facility.sh, use hard-coded sleep statements to ensure that prior operations finish.
While running the test suite of sysklogd inside a heavily CPU-limited podman container, a few tests (test/memleak.shtest/facility.shtest/secure.shtest/remote.sh) showed flaky behaivior, i.e. they failed occasionally. The same behaivior showed on an old notebook with a > 10 year old i3 CPU running Debian 11. This may be a minor problem for CI environments, which often heavily restrict the available CPU time.
A simple solution would be to just sleep longer - however, this would increase the test suite runtime significantly, and I would assume that only a very small amount of users are affected by the flakyness in the first place. It may be a better idea to just add it to the documentation as a caveat.
Alternatively, it may be a good idea to, in case of an unexpected failure like an empty grep result, recheck and sleep a few times inside the tests and only fail after n retries.
The text was updated successfully, but these errors were encountered:
Yes, they are flaky. I have not had the time to fix that, use them mainly for regression testing when modifying the code base. What are you using them for?
Some tests, like
test/facility.sh
, use hard-coded sleep statements to ensure that prior operations finish.While running the test suite of sysklogd inside a heavily CPU-limited podman container, a few tests (
test/memleak.sh
test/facility.sh
test/secure.sh
test/remote.sh
) showed flaky behaivior, i.e. they failed occasionally. The same behaivior showed on an old notebook with a > 10 year old i3 CPU running Debian 11. This may be a minor problem for CI environments, which often heavily restrict the available CPU time.A simple solution would be to just sleep longer - however, this would increase the test suite runtime significantly, and I would assume that only a very small amount of users are affected by the flakyness in the first place. It may be a better idea to just add it to the documentation as a caveat.
Alternatively, it may be a good idea to, in case of an unexpected failure like an empty grep result, recheck and sleep a few times inside the tests and only fail after n retries.
The text was updated successfully, but these errors were encountered: