feat(docs): add fork-preview banner, dual Atom feeds, gh-pages reset - #451
Draft
netravnen wants to merge 8 commits into
Draft
feat(docs): add fork-preview banner, dual Atom feeds, gh-pages reset#451netravnen wants to merge 8 commits into
netravnen wants to merge 8 commits into
Conversation
docs.peeringdb.com is served via GitHub Pages today, but entirely manually -- someone runs `mkdocs gh-deploy --clean` from their own machine, which pushes the built site to upstream's `gh-pages` branch. There is no CI/CD in this repo's history at all, so nothing catches a broken link or nav omission before it ships, and the fork has no equivalent preview of its own. This adds two workflows scoped to this fork only -- upstream's manual production deploy is untouched. Changes: - .github/workflows/docs-build-check.yml: runs `mkdocs build --strict` on every pull request and on push to master/ admincom/docs-improvements, using uv to install the pinned dependency set from uv.lock. Fails the check on any warning (broken links, pages missing from nav, etc.) -- the automated version of the manual --strict checks run throughout this session's doc work. - .github/workflows/docs-deploy.yml: automates the exact `mkdocs gh-deploy --clean` command README.md already documents as the manual process, targeting the fork's own (currently nonexistent) gh-pages branch. Strips docs/CNAME before building first, since that file lives inside docs_dir and would otherwise get copied into the built site and wrongly claim docs.peeringdb.com as the fork's own custom domain. Runs on push to master/admincom/docs-improvements plus workflow_dispatch for manual redeploys; --force is required since a stateless CI runner never has prior gh-pages history to compare against. Security: - Deploy workflow requests `contents: write` on GITHUB_TOKEN, scoped to this repo only; PR-check workflow stays `contents: read` since it never publishes anything. Neither workflow runs on pull_request_target or handles third-party PR code with elevated permissions, so forked-PR privilege escalation isn't a concern. Testing: - Both YAML files parse cleanly with PyYAML; `mkdocs build --strict` still passes with 0 warnings locally. End-to-end verification (Actions tab going green, gh-pages branch creation, Pages URL serving correctly) requires pushing to GitHub and is documented as follow-up manual verification, not achievable from a local clone. Backwards Compatibility: - N/A -- new files only; no existing workflow, branch, or build behavior is changed. Upstream's manual deploy process is completely unaffected. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The blog page listed posts in docs/blogs.md with no way to subscribe to new ones. Add a mkdocs hook that regenerates docs/atom.xml from that list on every build, so a feed reader can follow new posts without manually checking the page. Post dates come from git history rather than blogs.md's hand-typed date text: a bulk git-log pass over docs/blog/ resolves each post's first-added commit date in one shot, falling back to a slower per-file `git log --follow` lookup only for the handful of posts that pass has missed (files introduced via a rename, which git classifies as an R event rather than A, so a --diff-filter=A bulk pass silently skips them). This gives real commit timestamps (time of day, correct timezone) instead of a hand-typed day-only date, and self-corrects if blogs.md's date text is ever wrong or stale. The feed's self-link is driven by an ATOM_SITE_URL environment variable (falling back to mkdocs.yml's site_url) rather than always hardcoding the production URL, so this fork's own preview deploy can correctly self-reference wherever it actually serves from instead of falsely claiming to be docs.peeringdb.com. docs-deploy.yml sets it from GitHub Actions context; docs-build-check.yml gets fetch-depth: 0 added to its checkout, which the git-log date lookups require -- without full history, every post would appear to have no git history at all, and mkdocs build --strict promotes that warning to a failure. Duplicate timestamps (two posts added in the same commit) are nudged apart by 1-second increments after date resolution, so every entry's atom:updated value stays unique per the W3C Feed Validator's interoperability recommendation, without affecting sort order. Changes - scripts/generate_atom_feed.py (new): on_pre_build hook. Bulk + per-file git-log date lookup, markdown-to-plaintext excerpt extraction for each entry's summary, ISO-8601 formatting (normalized to UTC for git's real commit timezones), duplicate-timestamp dedupe, Atom 1.0 XML construction via stdlib ElementTree/minidom -- no third-party feed library. - mkdocs.yml: register the hook under hooks:. - docs/blogs.md: "Subscribe via Atom feed" line; also the underlying source list this hook parses. - peeringdb_theme/main.html: <link rel="alternate"> feed-discovery tag. - .gitignore: docs/atom.xml is a generated build artifact, not tracked. - .github/workflows/docs-build-check.yml, .github/workflows/docs-deploy.yml: fetch-depth: 0 (required for the git-log date lookups); docs-deploy.yml sets ATOM_SITE_URL from GitHub Actions context and strips any production docs/CNAME before building, so this fork's preview never falsely claims to be docs.peeringdb.com. Security - N/A Testing - uv run mkdocs build --strict: 0 warnings. - Spot-checked generated dates against git log for several posts, including a renamed file resolved via the --follow fallback. - Confirmed duplicate-timestamp entries get distinct atom:updated values while preserving their original relative order. Backwards Compatibility - N/A Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The fork's preview site needs its own domain, distinct from upstream production's docs.peeringdb.com -- docs-dev.peeringdb.dk. The straightforward way to do this (edit docs/CNAME's tracked content directly) has the same problem as removing it outright: docs/CNAME is the same file both this fork's automation and upstream's manual `mkdocs gh-deploy --clean` rely on, so whatever value sits in it on this branch would silently become production's value the moment this branch merges upstream -- either breaking the custom domain entirely (if removed) or replacing it with the wrong one (if hardcoded to the dev domain). Instead of choosing a value to store, generate the file at build time from configuration, mirroring the pattern already used for the atom feed's self-link: an optional SITE_URL_OVERRIDE env var takes priority when set, falling back to mkdocs.yml's site_url (https://docs.peeringdb.com/) when unset. This makes the merge-safety concern moot rather than just documented: upstream's deploy process sets no override, so it always regenerates the correct production CNAME automatically, with no manual edit required at merge time. Changes: - scripts/generate_cname.py: new mkdocs on_pre_build hook. Derives the GitHub Pages custom domain from SITE_URL_OVERRIDE if set, else config.site_url, via urllib.parse (stdlib only, no new dependency), and writes docs/CNAME before mkdocs collects static files -- same timing/mechanism already used for the atom feed. - scripts/generate_atom_feed.py: rename the ATOM_SITE_URL env var to SITE_URL_OVERRIDE, since it now drives two hooks, not just the feed. - mkdocs.yml: register the new hook alongside the existing one. - docs/CNAME: untracked (git rm --cached) and added to .gitignore, alongside docs/atom.xml -- both are now build artifacts regenerated every run, not source content. - .github/workflows/docs-deploy.yml: drop the now-unnecessary "strip CNAME" step, and set SITE_URL_OVERRIDE from a repo variable (Settings -> Actions -> Variables -> SITE_URL_OVERRIDE, already configured to https://docs-dev.peeringdb.dk) rather than hardcoding the domain in the workflow file. Security: - N/A -- documentation only. urllib.parse.urlparse on a value from a repo-scoped Actions variable is not an injection vector. Testing: - mkdocs build --strict passes with 0 warnings in both modes: with no SITE_URL_OVERRIDE set, site/CNAME correctly resolves to docs.peeringdb.com (config.site_url fallback, matching current production behavior exactly); with SITE_URL_OVERRIDE= https://docs-dev.peeringdb.dk/, site/CNAME resolves to docs-dev.peeringdb.dk and the atom feed's self-link matches it too, confirming both hooks stay in sync off the same override. Backwards Compatibility: - N/A for upstream -- unchanged behavior when no override is set. For this fork specifically, docs/CNAME's tracked value changes from a static docs.peeringdb.com to being generated at build time from the SITE_URL_OVERRIDE repo variable; the fork's actual GitHub Pages custom-domain setting and docs-dev.peeringdb.dk DNS record still need to be configured to match (outside this branch's scope). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
generate_cname.py and generate_atom_feed.py each independently resolved SITE_URL_OVERRIDE-or-site_url with slightly different fallback logic. Extract one shared policy so both (and later hooks) always agree on what "the site URL" is for a given build. scripts/site_url.py's resolve_site_url() resolves in this order: 1. PROD=TRUE -- real production, full stop. 2. SITE_URL_OVERRIDE, if set. 3. The local dev-server address mkdocs itself is serving on (e.g. http://127.0.0.1:8000/), if `mkdocs serve` is what's actually running. Requires callers to be wired to on_pre_build, not on_config: mkdocs's own serve.py only overwrites config.site_url with the dev address *after* on_config has already run (see commands/serve.py's `config.site_url = f'http://{config.dev_addr}...'`, right before it calls build()), so on_config never sees it. 4. LOCAL_SITE_URL ("http://localhost:8000/", mkdocs's own default --dev-addr) -- the fallback when none of the above apply: a plain `mkdocs build` with no env vars and no live dev server, e.g. docs-build-check.yml's CI check or a one-off local build served separately afterward (a static server started after the build already finished can't be predicted at build time -- there's nothing to detect there). Deliberately assumes "local" rather than guessing a real GitHub Pages URL from `git remote get-url origin`: a real checkout always has an `origin` remote, so a git-remote-based guess would almost always "succeed" with a URL that doesn't correspond to anything actually being served, which is more misleading than an honest localhost default. Changes - scripts/site_url.py (new): resolve_site_url() and origin_repo_url(), the shared policy described above. - scripts/generate_cname.py: use the shared resolver instead of its own SITE_URL_OVERRIDE-or-site_url fallback. Also now skips writing docs/CNAME when the resolved host is a loopback address, since a CNAME file only means anything for a real GitHub Pages deploy. - scripts/generate_atom_feed.py: use the shared resolver instead of its own fallback. Security - N/A Testing - uv run mkdocs build --strict in four scenarios (no env vars / SITE_URL_OVERRIDE / PROD=TRUE / both), all 0 warnings. - Real `mkdocs serve` run: both docs/CNAME and the feed's self-link correctly resolve to the actual dev-server address, confirmed via curl that the server responded at that address. - Confirmed a plain `mkdocs build` with no env vars and no dev server running resolves to http://localhost:8000/ rather than a GitHub Pages guess, and that docs/CNAME is correctly left untouched (a loopback host skips the write). Backwards Compatibility - N/A Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This fork's preview deploy currently looks identical to real
production, making it easy to mistake one for the other. Add a
banner above the nav bar showing the fork's repo, the URL currently
serving the page, and where production actually lives.
The banner is on by default -- any build without further signal is
assumed to be a dev/fork build -- and is suppressed only when PROD=TRUE
is set at build time, deliberately the opposite default from
site_url.resolve_site_url()'s own production-safe fallback, since a
real production deploy lives outside this repo and can set PROD=TRUE
explicitly.
Changes
- scripts/generate_banner.py (new): on_pre_build hook that exposes
fork repo/live/production URLs to templates via
config.extra.fork_banner, gated on PROD=TRUE. Uses on_pre_build
(like the cname/atom-feed hooks) rather than on_config, required for
resolve_site_url()'s local-serve detection tier to see mkdocs's
overwritten config.site_url.
- peeringdb_theme/base.html (new): full copy of mkdocs 1.4.3's packaged
base.html (pinned version) with the banner markup inserted before the
nav bar div, since no block wraps that div for main.html to override.
- docs/extra.css: .fork-banner styling. Height is fixed to 3.5rem to
match the nav bar's own documented height (mkdocs's base.css:
scroll-padding-top comment), which .nav-link's 1rem/1rem padding plus
Bootstrap's 1.5rem line-height already add up to -- no JS measurement
needed. The banner is sticky at top:0 with the nav bar offset below it
(body.has-fork-banner .navbar.fixed-top { top: 3.5rem }) so both stay
pinned together while scrolling.
- mkdocs.yml: register generate_banner.py in hooks:.
Security
- N/A
Testing
- uv run mkdocs build --strict in the same four scenarios as the
previous commit, all 0 warnings, banner present/absent as expected
in each.
- Visually verified in-browser: banner height matches the nav bar,
stays sticky above it while scrolling, single-line content doesn't
wrap.
Backwards Compatibility
N/A -- peeringdb_theme/base.html duplicates mkdocs 1.4.3's packaged
base.html. If mkdocs is ever upgraded, this file needs to be re-diffed
against the new packaged version and the banner insertion reapplied.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The blog feed generator (generate_atom_feed.py) inlined its Atom feed/entry XML construction, ISO-8601 formatting, markdown-link stripping, and timestamp-collision dedupe logic (commit a02694c6). A second, independent feed for release notes is coming in a follow-up commit -- pull the generic, non-blog-specific pieces out into a shared scripts/atom_common.py first so that feed doesn't have to duplicate ~60 lines of ElementTree/minidom boilerplate. Rename generate_atom_feed.py to generate_blog_atom_feed.py for symmetry with the upcoming generate_release_notes_atom_feed.py, and relocate its output from docs/atom.xml to docs/blog/atom.xml so the feed is co-located with its source content the same way the release notes feed will be (docs/blog/ + docs/release_notes/). Changes - scripts/atom_common.py (new): build_feed(), dedupe_published(), isoformat(), plain_text(), truncate() -- extracted unchanged from generate_atom_feed.py. - scripts/generate_atom_feed.py -> scripts/generate_blog_atom_feed.py (renamed + refactored): keeps its blog-specific git-log date derivation and post-excerpt logic, now calls atom_common.build_feed() instead of inlining feed construction. Output moved to docs/blog/atom.xml. - mkdocs.yml: hooks: entry updated for the rename. - peeringdb_theme/main.html: blog feed's <link rel="alternate"> href updated to blog/atom.xml. - docs/blogs.md: "Subscribe via Atom feed" link updated to the new blog/atom.xml path. - .gitignore: docs/atom.xml entry updated to docs/blog/atom.xml. Security - N/A Testing - uv run mkdocs build --strict in four scenarios (no env vars / SITE_URL_OVERRIDE / PROD=TRUE / both), all 0 warnings. - Diffed the feed's content against a pre-refactor build: byte- identical apart from the self-link, confirming the atom_common extraction and the file move didn't change any feed data. - Confirmed docs/blogs.md's and site/blogs/index.html's rendered Subscribe links resolve to the new blog/atom.xml path. Backwards Compatibility Subscribers following the old docs/atom.xml URL on this fork's preview will need to resubscribe at docs/blog/atom.xml -- flagging since it's a real, if minor, break. Production (docs.peeringdb.com) has never had this feed live, so nothing changes there. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The blog feed was the only Atom feed this repo generated. Add a
second, independent feed for release notes, sourced from
docs/release_notes/index.md's `## Release X.Y.Z` sections, so
subscribers can follow releases without also getting blog posts (and
vice versa). Builds on the shared scripts/atom_common.py extracted in
the previous commit.
Release notes entry dates come straight from each section's
`Release Date:` line (falling back to `Beta Announcement Date:` if
absent) rather than git history -- unlike blog posts, this content is
already hand-dated inline, so there's no gap to fill. Entry URLs are
built with markdown.extensions.toc.slugify() (mkdocs's own anchor
slugifier, confirmed against the real built HTML: "Release 2.81.0"
becomes id="release-2810") rather than a hand-rolled one, guaranteeing
they always match the real heading IDs mkdocs generates. Entry
summaries are a plain issue count ("N GitHub issues addressed in this
release"), counted by matching table rows starting with `| [` rather
than parsing cell contents -- the summary column has occasional
unescaped `{`/`[`/`'` characters that would make a content-parsing
regex fragile for no real benefit, since a feed entry's job is to
notify, not replace the page a click away. Only index.md is scoped in,
not the per-year release_notes_YYYY.md archives -- a feed is about
what's new, and index.md alone already covers back to release 2.75.0.
Changes
- scripts/generate_release_notes_atom_feed.py (new): on_pre_build hook
parsing docs/release_notes/index.md into feed entries as described
above, writing docs/release_notes/atom.xml via atom_common.build_feed().
- mkdocs.yml: hooks: entry for the new hook.
- peeringdb_theme/main.html: second <link rel="alternate"> for the
release notes feed.
- docs/release_notes/index.md: "Subscribe via Atom feed" line, matching
docs/blogs.md's existing convention.
- .gitignore: docs/release_notes/atom.xml entry -- a build artifact,
not tracked.
Security
- N/A
Testing
- uv run mkdocs build --strict in four scenarios (no env vars /
SITE_URL_OVERRIDE / PROD=TRUE / both), all 0 warnings, both feeds'
self-links agreeing with docs/CNAME in every scenario.
- Spot-checked the feed: Release 2.81.0's entry URL
(.../release_notes/#release-2810) matches the real anchor ID in the
built HTML, published date matches its Release Date line, and the
issue count (8) matches a manual count of that section's table rows.
Backwards Compatibility
N/A -- new feed, nothing previously depended on this URL.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
mkdocs gh-deploy already regenerates the whole site tree from the current branch on every run, so removed pages never linger. What wasn't guaranteed was the gh-pages branch's commit history: without --no-history, ghp-import fetches the existing gh-pages HEAD and commits on top of it, so the branch's state technically depends on whatever was already there rather than purely on this run's build. --no-history makes every deploy a single force-pushed root commit, so gh-pages is unconditionally reset on top of whatever this branch built, with no path for prior branch state to carry forward. Changes: - Added --no-history to the mkdocs gh-deploy invocation in docs-deploy.yml, alongside the existing --clean --force flags. Security: - N/A Testing: - Not run through an actual gh-deploy (would force-push the shared gh-pages branch); verified by reading mkdocs/ghp-import's documented --no-history behavior against the existing --force (permissions: contents: write) setup already in this workflow. Backwards Compatibility: - gh-pages loses its multi-commit history going forward (each deploy now replaces it with one commit instead of appending). The served site content is unaffected; only the branch's own commit log style changes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Split out of #449 to make review easier. The content fixes moved to a
separate PR; #448 is the tracking issue for the whole line of work.
This branch started as infra for running a dev/preview fork of this
site, and grew to fix a few problems that showed up while doing that:
preview deploys were indistinguishable from production in a browser,
the two content streams (blog posts and release notes) only had one
combined feed to subscribe to, and
gh-pagesaccumulated deployhistory that made no promise about matching the branch that built it.
Opening as draft because part of this (the fork-preview banner) is
fork-specific scaffolding that may not be wanted upstream as-is;
flagging for discussion rather than asking for a merge.
Changes
and where production lives, so a preview deploy is never mistaken
for the real site. Suppressed via
PROD=TRUE.docs/blog/atom.xmlanddocs/release_notes/atom.xml, so blog and release-notes readers cansubscribe separately. Share a common
scripts/atom_common.py.Release-notes entries are parsed from
release_notes/index.md'sexisting
## Release X.Y.Zsections.scripts/site_url.py): one sharedprecedence (
PROD=TRUE>SITE_URL_OVERRIDE> detected localdev-server address >
http://localhost:8000/) used by the banner,both feeds, and dynamic
docs/CNAMEgeneration, so they can'tdisagree about what URL they're building for.
--no-historyso every deploy is asingle fresh commit built from the current branch, instead of
accumulating commits on top of whatever was already there.
Testing
uv run mkdocs build --strict-- 0 warnings.mkdocs serveand aseparately-served static build.
PROD=TRUE.Notes for reviewers
scripts/generate_banner.pyand theSITE_URL_OVERRIDEdev-domainplumbing in
docs-deploy.ymlare the pieces most likely to needchanges, or removal, before this could land on
masteras-is. Happyto split those out of the eventual mergeable set if preferred.