Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Grouping tests by files instead of items #20

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

ishefi
Copy link

@ishefi ishefi commented May 20, 2020

Thanks for this plugin! We are using it and it helped lowering our pipeline running times from ~60m to ~7m.

However, I found out that failures of unittets tend to be across a small number of files. Since unitests of the same file are devided between multiple (usually all) groups, multiple groups are failing due to a bug that stems in the same area.

This PR adds an option to split by files rather than by items, making the investigation of failures much easier.

Couple of notes, as I'm a bit afraid this PR might not be in the spirit of this project:

  1. If the number of files is smaller than total groups some groups will run no tests.
  2. Some jobs might take longer than others, as there is no way to guarantee tests will be equally divided between jobs. The division is done using a greedy algorithm which tries to split the tests so each job runs about the same number of tests, but in extreme cases some jobs will work much harder than others (e.g.: only two test files need to run: file1 with 1000 tests, file2 with 2 tests).

Thank you again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant