feat(ci): cache-bust image refs when only an asset changes - #8
Merged
Conversation
Re-rendering a diagram in place reached nobody. devto-cli skips an article whose markdown is byte-identical to what is live, and dev.to's Bunny CDN serves every image `immutable` for a year, so even a push that did go out left readers on the old PNG. scripts/bump_asset_versions.py rewrites `?v=<sha256(bytes)[:8]>` on every reference to a changed asset: the markdown now differs (devto-cli pushes) and the URL now differs (the CDN refetches). publish.yml also triggers on articles/assets/** and folds the affected articles into the same dev push batch, so an asset-only commit republishes its articles on its own. Ownership is resolved by reference, not by directory name. 40 asset directories have no same-named article (assets/spire/ backs spiffe-spire-deep-dive.md, assets/IssueHub/ backs issuehub.md), so every top-level article is scanned for a reference to each changed asset. Markdown images, <img src>, and the absolute cover_image URL are all covered; fenced blocks and inline code are not, and cover_image is only matched inside frontmatter so a YAML sample in the body cannot be corrupted. The token is a content hash rather than a counter, which makes reruns idempotent and self-healing: a run that bumps but then fails before the commit recomputes the same value next time. This replaces the manual `?v=2` bump the docs used to prescribe.
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Re-rendering a diagram in place never reached a reader. Two independent things had to be worked around by hand:
devto-clionly re-pushes an article whose markdown differs from what is live. Re-rendering01-foo.pngchanges no markdown, so the push was a no-op.cache-control: public, max-age=31536000, immutable. Even when a push did go out, the old PNG kept being served for a year.publish.ymlalso only triggered onarticles/**/*.md, so an asset-only commit never started the workflow at all.The documented workaround was to hand-bump
?v=2on every image ref in the article and increment it on each re-render. Two articles (aws-iam-deep-dive,aws-sts-deep-dive) carry that manual token today.Change
scripts/bump_asset_versions.pyrewrites?v=<sha256(bytes)[:8]>on every reference to a changed asset. The markdown then differs (sodevto-clipushes) and the URL differs (so the CDN refetches).publish.ymlnow:articles/assets/**as well as markdown,dev pushbatch,id/datewriteback.make bump-assetsdoes the same locally against$(BASE);--dry-runpreviews without writing.Notes on the implementation
Ownership is resolved by reference, not by directory name. Deriving
articles/<slug>.mdfromarticles/assets/<slug>/looks obvious but is wrong here: 40 asset directories have no same-named article.assets/spire/backsspiffe-spire-deep-dive.md,assets/IssueHub/backsissuehub.md. The script scans every top-level article for a reference to each changed asset instead, which also handles an image shared by two articles.Coverage. Markdown images,
<img src>, and the absolutecover_imageURL. Fenced blocks (backtick and tilde) and inline code are skipped, using the same line-by-line fence tracking asvalidate_articles.py::strip_coderather than a.*?regex, so~~~and four-backtick fences behave identically in both scripts.cover_imageis only matched inside the frontmatter span, so a YAML sample in the body (an article documenting this very pipeline, say) cannot be corrupted. URLs pointing at other repos are left alone.The token is a content hash, not a counter. Reruns are idempotent, and a run that bumps but then fails before committing recomputes the same value on the next attempt.
Relative paths with a query survive
devto-cli's rewriting. Confirmed by readingdevto-cli@1.4.0'slib/util.js, and corroborated byaws-iam-deep-diveandaws-sts-deep-dive, which have shipped?v=2through this pipeline already. Details under Verification.Verification
Full-corpus run, not just a smoke test
Cloned the branch, ran the script for real against every tracked image asset in
articles/assets/at once, and checked the result four ways:?v=from old and new: all 104 files byte-identicalvalidate_articles.py --allmarkdownlint-cli2over all 104The "inside code" oracle deliberately does not reuse
code_mask. It usesvalidate_articles.py::strip_code, a separate pre-existing implementation, as a second opinion. Worth recording that the first version of that oracle was itself wrong: it compared character offsets, butstrip_codepreserves line count, not offsets (a fenced line becomes"", not spaces). It reported 326 false positives until it was rewritten to compare per line.The 7 "no article references this" warnings are genuine orphans, confirmed by grep.
Collect step, simulated in a throwaway repo
devto-cli behaviour, read from the source
The one assumption left was whether a relative path carrying a query survives
devto-cli's rewriting. Confirmed against@sinedied/devto-cli@1.4.0rather than inferred:lib/util.js:11getFullImagePathispath.normalize(path.join(basePath, imagePath)), which treats?v=abcas part of the filename and passes it through../assets/demo/foo.png?v=2d711642becomeshttps://raw.githubusercontent.com/0-draft/dev.to/main/articles/assets/demo/foo.png?v=2d711642, verified by running the same expression under node.lib/util.js:25guardscover_imagerewriting withisUrl(...), so an absolute cover URL is passed through untouched, tokens included.updateRelativeImageUrlsusesString.replace(link, newLink), which only rewrites the first occurrence of a byte-identical link. Checked that the bump introduces no duplicate-identical link pair that did not already exist: 0 across the corpus.Fixtures
Markdown image,
<img>following a bumped markdown image, backtick fence, tilde fence, inline code, inline code sharing a line with a real image (both orders), image with a title, image in a table cell, absolute this-repo URL in the body, foreign-repo raw URL,cover_imagein single and double quotes, and an article underarticles/TIL/. Only the real references change;TIL/is left alone because it never publishes.Not verified
The workflow has not been executed on GitHub.
publish.ymlonly runs onmain, anddev push --dry-runneedsDEVTO_API_KEY, which is a repo secret and is not available locally. The collect step is covered by the simulation above and byactionlint(which runs shellcheck over therun:blocks); the first real exercise will be the merge commit.Known limitation
code_masksharesstrip_code's blind spot: a four-space indented code block is not recognised as code. Both image refs in this corpus that look indented turned out to be list-item continuations, which render as images and do want bumping, so the gap is not currently exercised. Noted in the docstring.Reviewer note
The first draft of
_rewritecalledpattern.subonce per pattern against a code mask indexed on the original text. The first pattern's insertions shifted the text out from under the mask, so the second pattern read the wrong mask byte and, once the text grew past the mask, raisedIndexError.articles/shogi-on-github-profile.mdhas exactly that shape (markdown image at line 20,<img>at line 91) and would have failed the publish step. It now collects all edits in one pass and splices once. The regression fixture is the<img>-after-markdown-images case above.