Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue: #3708 add ParameterizedTest#argumentCountValidation #4045

Open
wants to merge 19 commits into
base: main
Choose a base branch
from

Conversation

JonasJebing
Copy link

@JonasJebing JonasJebing commented Oct 4, 2024

Overview

This allows parameterized tests to fail
when there are more arguments provided than declared by the test method. This is done in a backwards compatible way
by only enabling that validation when the new
junit.jupiter.params.argumentCountValidation is set to strict or ParameterizedTest#argumentCountValidation is set to ArgumentCountValidationMode.STRICT.

Open Questions

  • Should these additions be declared as experimental or stable?
  • Should this feature be documented in the User Guide and Release Notes, given that it is declared as experimental?
  • Should the new precondition be documented on org.junit.jupiter.params.ParameterizedTestExtension#provideTestTemplateInvocationContexts, even though it is just an interface override and none of the existing preconditions are documented there?

I hereby agree to the terms of the JUnit Contributor License Agreement.


Definition of Done

This allows parameterized tests to fail
when there are more arguments provided than declared by the test method.
This is done in a backwards compatible way
by only enabling that validation when the new
`junit.jupiter.params.argumentCountValidation` is set to `strict`
or `ParameterizedTest#argumentCountValidation` is set to
`ArgumentCountValidationMode.STRICT`.
@JonasJebing JonasJebing force-pushed the argument-count-validation-mode branch from 2b42de8 to 2e56d56 Compare October 5, 2024 20:30
Comment on lines 151 to 152
logger.warn(() -> String.format(
"Ignored invalid configuration '%s' set via the '%s' configuration parameter.", value, key));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error message should be the same as in EnumConfigurationParameterConverter. Unfortunately, we can't (easily) reuse that here.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there some way that we could make it more easily reusable across the project? Maybe I could add another PR to address that after merging this one.
For now, I basically copied the messages from EnumConfigurationParameterConverter.

@marcphilipp
Copy link
Member

  • Should these additions be declared as experimental or stable?

Our guideline is to introduce new features as "experimental".

  • Should this feature be documented in the User Guide and Release Notes, given that it is declared as experimental?

I think we should add a small sample to the User Guide otherwise the chance to get feedback is slim.

  • Should the new precondition be documented on org.junit.jupiter.params.ParameterizedTestExtension#provideTestTemplateInvocationContexts, even though it is just an interface override and none of the existing preconditions are documented there?

Documenting preconditions is mainly interesting for public API so we can omit it here since ParameterizedTestExtension is package-private.

# Conflicts:
#	junit-jupiter-params/src/main/java/org/junit/jupiter/params/ParameterizedTest.java
#	junit-jupiter-params/src/main/java/org/junit/jupiter/params/ParameterizedTestExtension.java
- add `writing-tests-parameterized-tests-argument-count-validation`
  section to the user guide.
- cache configuration parameter in `ExtensionContext.Store`
- adjust log and error messages

Issue junit-team#3708
@JonasJebing
Copy link
Author

Thank you for all the pointers and comments on this PR @marcphilipp. I've addressed all your comments. Could you have another look?

@JonasJebing JonasJebing marked this pull request as ready for review October 23, 2024 15:59
Copy link
Member

@marcphilipp marcphilipp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making those changes! I think we're very close now. Please also add an entry to the 5.12.0-M1 release notes.

JonasJebing and others added 7 commits October 24, 2024 09:58
release-notes-5.12.0-M1-junit-jupiter-new-features-and-improvements
to be exact

Issue junit-team#3708
Co-authored-by: Marc Philipp <mail@marcphilipp.de>
Co-authored-by: Marc Philipp <mail@marcphilipp.de>
Co-authored-by: Marc Philipp <mail@marcphilipp.de>
Co-authored-by: Marc Philipp <mail@marcphilipp.de>
@JonasJebing
Copy link
Author

@marcphilipp could you review again? I applied all suggestions and added a release note.

Comment on lines 2044 to 2048
@ParameterizedTest(argumentCountValidation = ArgumentCountValidationMode.STRICT)
@CsvSource({ "42, -666" })
void testWithArgumentCountValidation(int number) {
assertTrue(number > 0);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move this snippet to ParameterizedTestDemo (and annotate it with @ExpectToFail; see other usages on how to avoid that annotation from being included in the user guide). We include them from there to ensure they compile and pass or fail as expected.

Co-authored-by: Marc Philipp <mail@marcphilipp.de>
JonasJebing and others added 2 commits October 25, 2024 15:23
Co-authored-by: Marc Philipp <mail@marcphilipp.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

@ParameterizedTest ignores additional arguments beyond formal parameter list
3 participants