You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When running tests with cucumber/serenity runner exceptions can be ignored, resulting in runs which pass but that should have failed. To reproduce there is a repo fork with branch with a test to reproduce the issue.
Exceptions in tests do get ignored when:
@Step annotation is used in single scenario
or atleast one scenario uses @Step annotations in feature file run
Attaching a picture from 2 runs:
both scenarios DOES throw exception
when ran as a single test without @Step annotation -- test fails (picture on the left)
when ran as a whole feature -- tests pass even though both tests should fail (picture on the right)
The text was updated successfully, but these errors were encountered:
UncleInf
changed the title
flaky tests because exceptions in tests can be ignored
Flaky tests, because exceptions in tests can be ignored
Oct 21, 2024
When running tests with cucumber/serenity runner exceptions can be ignored, resulting in runs which pass but that should have failed. To reproduce there is a repo fork with branch with a test to reproduce the issue.
Exceptions in tests do get ignored when:
@Step
annotation is used in single scenario@Step
annotations in feature file runAttaching a picture from 2 runs:
@Step
annotation -- test fails (picture on the left)The text was updated successfully, but these errors were encountered: