Handle skipped tests in "Not run" column - #3743
Open
jjomier wants to merge 9 commits into
Open
Conversation
williamjallen
left a comment
williamjallen
left a comment
Collaborator
There was a problem hiding this comment.
I'm conflicted on whether this belongs in CDash, or whether this should be added as a CTest property. I'm leaning towards the latter because it's possible that some projects will want to dynamically set the list of not run test warnings. The build error filter feature removed in #3650 had roughly the same considerations. Any thoughts on this?
Contributor
Author
|
It could be in addition to having a "Skipped" test property in CTest. However in CDash we would need a way to handle the display of skipped tests. Where you thinking something else? |
Collaborator
|
@jjomier Is your goal just to differentiate between disabled tests and tests which were "not run" for a different reason? We want to minimize the number of configuration options if possible, so I'm hesitant to approve this as-is. After discussing this internally, we think the best approach is to just match the text "Disabled" in the details field without it being user-configurable. Thoughts? |
Contributor
Author
|
@williamjallen yes the idea is to not pollute the tests that are marked as not run when we look at the dashboard. I like the idea of matching "Disabled". |
jjomier
force-pushed
the
jjomier/handle_skipped_tests
branch
from
August 7, 2026 12:16
ef9cd03 to
fc65763
Compare
williamjallen
requested changes
Aug 18, 2026
This commit introduces a new `notRunSkippedDetailsRegex` property to the Project model, allowing users to define regex patterns for not-run test details. A corresponding validation rule is added to ensure the regex is valid. The Build model now includes a method to count not-run tests that do not match the specified patterns, and the GraphQL schema is updated to expose this functionality. Additionally, UI components are modified to display the warning count and adjust test status colors based on the new regex patterns. Tests are also added to verify the new functionality.
This commit removes the `notRunSkippedDetailsRegex` property from the Project model and associated validation rule, simplifying the logic for handling not-run tests. The `notRunTestsWarningCount` method in the Build model is updated to exclude tests with details marked as "Disabled". The GraphQL schema and UI components are adjusted accordingly to reflect these changes. Additionally, tests are updated to ensure correct behavior without the regex pattern handling.
This commit improves the formatting of the `testStatusToColorClass` and `testStatusToTextColorClass` functions in the TestDisplay component. The switch cases have been reformatted for better readability, maintaining the same functionality. No changes to logic or behavior were made.
This commit simplifies the `notRunTestsWarningCount` method in the Build model by using a more efficient query to count not-run tests that do not have details marked as "Disabled". Additionally, the API response in the Index controller is updated to reflect the new method signature. Unused test methods in the TestDisplayTest class have been removed to clean up the codebase.
Replace the App\Utils\TestDisplay helper with a notRunWarning() query scope on the Test model, so the "Disabled" completion status is interpreted in one place next to the data it describes. The status color class is derived directly from the existing Test::DISABLED constant where it is needed, and the unused text/GraphQL color class helpers are dropped.
Counting every test of every build was only viable for small instances, since the index page renders many builds at once. Record the number of not-run tests worth warning about in a new build.testnotrunwarning column as test results are submitted, and read that column instead, so the index page cost no longer scales with the number of tests.
Nothing links to a specific settings section now that the not-run details are matched without a per-project option, so the anchor prop on FormSection and the hash navigation it existed for are unused.
The text-color mapping was leftover from the old PHP TestDisplay utility and is not imported anywhere in the Vue UI.
Master moved the configure/build/test status colors onto BuildTimelineCard. Use notRunTestsWarningCount there so tests whose details are "Disabled" do not turn the Test step orange.
jjomier
force-pushed
the
jjomier/handle_skipped_tests
branch
from
August 25, 2026 20:03
3fda663 to
a887248
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit introduces a new
notRunSkippedDetailsRegexproperty to the Project model, allowing users to define regex patterns for not-run test details. A corresponding validation rule is added to ensure the regex is valid. The Build model now includes a method to count not-run tests that do not match the specified patterns, and the GraphQL schema is updated to expose this functionality. Additionally, UI components are modified to display the warning count and adjust test status colors based on the new regex patterns. Tests are also added to verify the new functionality.