Document that Assumptions cannot abort a failed test

[er2]

Steps to reproduce

import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;

import static org.junit.jupiter.api.Assertions.fail;
import static org.junit.jupiter.api.Assumptions.assumeTrue;

public class PostFactoAssumptionsTest {

    @AfterEach
    public void tearDown() {
        assumeTrue(false);
    }

    @Test
    public void testAssumptions() {
        fail();
    }
}

Expected Behavior: Test Ignored
Actual Behavior: Test Failed

Context

• Used versions (Jupiter/Vintage/Platform): 5.10.3
• Build Tool/IDE: IntelliJ, Maven

What I'm trying to do

import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;

import java.time.Duration;
import java.time.LocalDate;
import java.time.LocalDateTime;
import java.util.List;

import static org.junit.jupiter.api.Assumptions.assumeTrue;

@UseH2
public class GetByAgeTest {

    LocalDateTime NOW = LocalDateTime.now();

    MyService service = new MyService();

    @BeforeEach
    public void setUp() {
        insertRowIntoDb(1, NOW);
        insertRowIntoDb(2, NOW.minusDays(1));
    }

    @AfterEach
    public void tearDown() {
        boolean dateDidntIncrementDuringTestRun = NOW.toLocalDate().equals(LocalDate.now());
        assumeTrue(dateDidntIncrementDuringTestRun);
    }

    @Test
    public void testGetByAge() {
        var results = service.getNewerThan(Duration.ofDays(1));
        assertThat(results).hasSize(1);
    }
}

public class MyService {
    public List<MyEntity> getNewerThan(Duration duration) {
        return queryDb("""
                SELECT *
                FROM my_table
                WHERE dateCol > trunc(now()) - ?
                """, duration.toDays());
    }
}

[sbrannen]

That's by design.

It is impossible to abort a test that has already failed.

JUnit uses a ThrowableCollector behind the scenes for this support, and the Javadoc for that class states the following.

This class distinguishes between Throwables that abort and those that fail test execution. The latter take precedence over the former, i.e. if both types of Throwables were collected, the ones that abort execution are reported as suppressed Throwables of the first Throwable that failed execution.

So, this is in fact documented at the "platform level", but this is not clearly documented in the Javadoc or User Guide regarding Assumptions in JUnit Jupiter.

In light of that, I have changed this to a documentation issue.

[marcphilipp]

@er2 Here's a way to accomplish what you're looking for:

import static org.junit.jupiter.api.Assertions.fail;
import static org.junit.jupiter.api.Assumptions.abort;

import java.time.LocalDateTime;
import java.time.temporal.ChronoUnit;

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
import org.junit.jupiter.api.extension.TestExecutionExceptionHandler;

public class GetByAgeTest {

	LocalDateTime NOW = LocalDateTime.now().truncatedTo(ChronoUnit.SECONDS);

	@RegisterExtension
	TestExecutionExceptionHandler exceptionHandler = (context, throwable) -> {
		boolean timeDidntIncrementDuringTestRun = NOW.equals(LocalDateTime.now().truncatedTo(ChronoUnit.SECONDS));
		if (timeDidntIncrementDuringTestRun) {
			throw throwable;
		} else {
			abort();
		}
	};

	@Test
	public void fast() {
		fail();
	}

	@Test
	void slow() throws InterruptedException {
		Thread.sleep(1_500);
		fail();
	}
}