testing: implement full pipeline test with an example datasets

[sophiamaedler]

Whenever a commit is merged, a full test of the entire pipeline should be run to ensure that none of the implemented changes break anything in the current workflow.

Dataset types to test:

• single timepoint project
• time course project

Segmentation Workflows to test:

• WGA segmentation
• cellpose segmenation
• nucleus segmentations
• cytosol segmentations

Segmentation classes:

• sharded
• multi-threaded
• with and without downsampling

The chosen datasets to perform the tests on need to be small enough to be supplied with the repo. The expected results from each run should also be saved and compared with the obtained results. This way any code changes that change obtained results can immediately be flagged and identified.

[mschwoer]

the scope of this test sounds similar to what we did in alphaDIA, so maybe this code here helps: https://github.com/MannLabs/alphadia/tree/main/tests/e2e_tests

[sophiamaedler]

the scope of this test sounds similar to what we did in alphaDIA, so maybe this code here helps: https://github.com/MannLabs/alphadia/tree/main/tests/e2e_tests

indeed this does look helpful. Thanks for sharing! We would try and setup something similar I think.