[WIP] Nightly full pa11y scan #3834
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The pa11y_targets file is always going to be created by the jekyll build and it doesn't seem harmful to go ahead and store the list of changed files into the env variable regardless of whether this is a full site scan being done nightly or if it's in response to a commit from GitHub on a pull request. This simplifies the configuration logic.
caleywoods
commented
May 16, 2024
- Introduced a new env var `$DO_PA11Y_SCAN` which is used to conditionally run the changed files pa11y scan job. - The pa11y scan step is skipped all together in cases where the workflow trigger is a webhook but there were no changed files detected. This happens in cases where things like dependabot updates dependencies etc
|
Closing this - it will be re-opened soon based on the replatforming work. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull request summary
#3829 Changed how pa11y scans are handled in CI so that instead of full site scans using the sitemap it only scans the files that were changed. Part of that scope of work was to create a scheduled full site scan with pa11y since that now never occurs otherwise.
There are three parts to completing this work, only the first step is complete currently:
repository_dispatchevent and creates a GitHub issue from the pa11y failures, those pa11y failures are provided in the event that will be sent to GitHub from our CircleCI step listed in step 3 [ ✅ ]18f/18f.gsa.govrepo that triggers therepository_dispatchGitHub action [ ✅ ]You can see a successful run of the scheduled "nightly" execution here on CircleCI.
Attempts to close #3832
Current status: tweaking the logic in our GitHub action responsible for creating the GH issue when a scan fails so that it tries to handle issue creation intelligently. The goal is to have it not open a new issue if there's an issue already open and the scan result between the two runs was identical.