Parallel tests based on tags or suites

[olliegator]

Hi,

I was looking for parallel test execution based on tags or suites, similar to Junit 4 suites, suitesAndClasses and suitesAndMethods or TestNG tests, but I couldn't find any information about it.

If there any way of having this kind of parallelization in Junit 5?

[marcphilipp]

Have you seen the Parallel Execution section in the User Guide? You can, e.g., run classes in parallel but methods inside each class sequentially using this configuration:

junit.jupiter.execution.parallel.enabled = true
junit.jupiter.execution.parallel.mode.default = same_thread
junit.jupiter.execution.parallel.mode.classes.default = concurrent

[olliegator]

Hi @marcphilipp,

Yes I saw that, but perhaps I wasn't clear enough.
If I have classes A1Test, A2Test with tag A, and B1Test, B2Test with tag B I want to run all the tests of classes ATest sequentially in one thread and in parallel all the tests of classes BTest sequentially in another thread.

This could also work by having ATestSuite which runs A1Tests and A2Test and BTestSuite which runs B1Test and B2Test, and then run the two suites in parallel.

[marcphilipp]

I'm afraid that's currently not possible since the Suite engine does not support executing suite classes in parallel.

[kaipe]

It's possible to follow the Parallel Execution guide-lines and use @ResourceLock("tag ...") to tag the test-classes. It's a compromise that would allow A1Test and A2Test in a sequence that executes in parallel with another sequence containing B1Test and B2Test.
But afaik it would only have the tests execute sequentially on their desired sequences, i.e. if it's also necessary to preserve thread-local context during the execution of either sequence then it might not work as desired. It seems to me each test gets its own thread, which is only started after locking the resource of its sequence:

@ResourceLock("tag A")
class A1Test extends DelayBeforeAndAfter {
    @Test void a1foo() {}
    @Test void a1bar() {}
}
@ResourceLock("tag A")
class A2Test extends DelayBeforeAndAfter {
    @Test void a2foo() {}
    @Test void a2bar() {}
}
@ResourceLock("tag B")
class B1Test extends DelayBeforeAndAfter {
    @Test void b1foo() {}
    @Test void b1bar() {}
}
@ResourceLock("tag B")
class B2Test extends DelayBeforeAndAfter {
    @Test void b2foo() {}
    @Test void b2bar() {}
}
@Execution(ExecutionMode.CONCURRENT)
class DelayBeforeAndAfter {
    static final long startTime = System.currentTimeMillis();
    String time() {
        return new java.text.DecimalFormat(" 0.000s").format(
                (System.currentTimeMillis() - startTime) / 1000.0);
    }
    @BeforeEach void recordStartTime(TestInfo info) throws Exception {
        System.out.println(info.getDisplayName() + " started at " + time());
        Thread.sleep(500);// Sleep 0.5 s
    }
    @AfterEach void recordEndTime(TestInfo info) throws Exception {
        Thread.sleep(500);// Sleep 0.5 s
        System.out.println(info.getDisplayName() + " ended at " + time());
    }
}

I had the above executed with this suite-definition ...

@Suite
@SelectClasses({A1Test.class,A2Test.class,B1Test.class,B2Test.class})
public class ParallelSuite {}

... to produce this output:

a1bar() started at  0.017s
b1bar() started at  0.017s
b1bar() ended at  1.025s
a1bar() ended at  1.025s
a1foo() started at  1.044s
b1foo() started at  1.044s
a1foo() ended at  2.046s
b1foo() ended at  2.046s
b2bar() started at  2.059s
a2bar() started at  2.059s
b2bar() ended at  3.060s
a2bar() ended at  3.060s
a2foo() started at  3.065s
b2foo() started at  3.066s
a2foo() ended at  4.067s
b2foo() ended at  4.067s

The timestamps reveal that there are at most one a-test and one b-test running in parallel at any moment. I.e. "tag A"-tests run in one sequence that executes in parallel with a sequence that carries all the "tag B"-tests! (But neither sequence uses any dedicated thread for executing its tests.)