Skip to content

feat(assistant): add the in-page AI assistant to docs.celo.org (#2250) - #2286

Open
GigaHierz wants to merge 6 commits into
mainfrom
GigaHierz/issue-2250-options
Open

feat(assistant): add the in-page AI assistant to docs.celo.org (#2250)#2286
GigaHierz wants to merge 6 commits into
mainfrom
GigaHierz/issue-2250-options

Conversation

@GigaHierz

@GigaHierz GigaHierz commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Refs #2250 — the research, comparison and decision are recorded on the issue. Not "Closes": the ops boxes on the issue stay open after this merges.

What this adds

  • assistant.js — injects the assistant widget, scoped to hosts the API accepts.
  • use-docs-with-ai.mdx — documents the Ask AI button alongside the existing MCP / llms.txt entry points.
  • AGENTS.md — adds the assistant to the AI-tooling entry points, and records that any .js under the content root runs on every published page.

The widget and its API live in celo-org/docs-ai-assistant, deliberately outside this repo — any .js committed here ships to every reader.

How it works

There is no second content pipeline. The backend answers through the Mintlify-hosted MCP endpoint at docs.celo.org/mcp, reindexed on every docs deploy, so the assistant cannot serve a stale index. Citations come free because the MCP search tool returns page URLs. Haiku 4.5 by default; Sonnet 5 on escalation.

The widget is served from docs-assistant.celo.org — the branded host, not the auto-generated Vercel one, so no hostname that could later be released is ever referenced from a docs page.

Verified against the live deployment

Check Result
Answer quality Grounded, ~8s, citations resolve 200
Escalation path Sonnet 5, 4 citations
Off-topic question Refuses rather than guessing
Origin allowlist Non-allowed origins get 403
Rate limiting Exactly 15 requests / 10 min per IP, then 429
submit_feedback MCP tool Denylisted — not exposed to the model
mint broken-links success no broken links found
Dependencies 0 vulnerabilities

Still open (tracked on #2250, not blocking this PR)

  • ALLOWED_ORIGINS needs http://localhost:3000 added so mint dev can exercise the widget.
  • Naming the owner for the API key and its spend-cap alerts.

🤖 Generated with Claude Code

Trivial marker script to confirm Mintlify's content-directory JS
injection works on the current plan before building the real widget
loader.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@GigaHierz
GigaHierz marked this pull request as ready for review August 25, 2026 14:33
@GigaHierz
GigaHierz requested a review from a team as a code owner August 25, 2026 14:33
@GigaHierz
GigaHierz requested a review from palango August 25, 2026 14:53
GigaHierz and others added 2 commits August 26, 2026 09:19
Replaces the injection-test marker with the real loader. The widget is
served from the assistant deployment and answers from the docs MCP
endpoint with links back to the pages it used.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds the Ask AI button to the entry-point table and a short section
covering what it answers from, that it cites its sources, and that
uncited gaps feed back into what gets written.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@GigaHierz GigaHierz changed the title test(assistant): verify custom JS injection on current Mintlify plan (#2250) feat(assistant): add the in-page AI assistant to docs.celo.org (#2250) Aug 26, 2026
GigaHierz and others added 2 commits August 27, 2026 20:23
…org host

Swap the temporary Vercel preview URL for the production CNAME now that it
resolves (200, application/javascript).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@palango palango left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid research behind this, and the #2250 comparison answers most of what I'd otherwise have asked: Mintlify's own assistant ruled out on price, custom JS confirmed on Starter, and the MCP endpoint as the backend so there's no second index to go stale. Three things block merge for me, then a list of smaller fixes.

The widget breaks outside production

assistant.js injects on every host, but the API only ever allows docs.celo.org:

$ curl -s -X OPTIONS -H "Origin: http://localhost:3000" \
       -H "Access-Control-Request-Method: POST" \
       -D- -o /dev/null https://docs-ai-assistant-ten.vercel.app/api/chat
access-control-allow-origin: https://docs.celo.org
vary: Origin

widget.js is a classic script, so it loads anywhere regardless. On mint dev, which AGENTS.md tells everyone to run before opening a PR, the Ask AI button appears, accepts a question, and then the preflight fails and widget.js:184 writes the raw browser error into the chat bubble as if it were the assistant's answer. Same on any preview build, which is why no reviewer can actually try the feature this PR adds.

if (location.hostname !== 'docs.celo.org') return;

The Note promises something the widget contradicts

If the docs don't cover your question, the assistant says so instead of guessing

widget.js:181 tests the finished answer for a docs.celo.org citation and, finding none, calls offerEscalation(), which renders "No docs pages cited. / Try a more thorough answer" and re-asks with the stronger model. An uncited answer is a case the UI is built to handle, so it isn't a refusal.

Where the sentence sits is what makes this matter. It lands directly after "Follow the citations before relying on anything load-bearing, especially contract addresses and code", so it withdraws the page's only safety caution. A reader told the assistant won't guess has no reason to check the citation on an answer that invented a checksummed address. I'd describe the escalation button and drop the guarantee.

Same paragraph: "that gap is a signal we use to decide what to write next" describes a loop I can't find. The widget's only network destination is /api/chat, and there's no telemetry in the diff. If the backend logs unanswered questions, say where they land.

Land the CNAME before this merges

The body already flags docs-ai-assistant-ten.vercel.app as temporary. The trouble is that performing the swap is what creates the exposure: renaming or deleting the Vercel project releases that auto-generated hostname, and whoever registers it next serves arbitrary JS on every docs page with same-origin DOM access, including the pages that print contract addresses and RPC endpoints. There's no SRI, no crossorigin, and no CSP anywhere in the repo to fall back on (no vercel.json, no _headers, no middleware, and docs.json has no headers key).

Shipping docs-assistant.celo.org from the start avoids the window. If this has to merge first, please file the two "Before merging" steps as issues, because a squash deletes the body that currently holds them.

Smaller, all fixable here

  • AGENTS.md is now wrong about its own subject. Line 124's AI tooling row lists five entry points, and line 139 closes with "the other two entry points", a fixed count. Both need the assistant.
  • The frontmatter description still reads "the docs MCP server, llms.txt, and per-page Markdown", so the new entry point is invisible in llms.txt, which AGENTS.md:47 calls the thing an agent sees before deciding to open a page.
  • The table's URL column now holds **Ask AI** button on any page in its first row, and the table ships verbatim in llms-full.txt, so an agent reading that column for endpoints gets a button label back. Renaming the header to "Where" fixes all five rows at once.
  • No onerror on the append, while the prose promises a button in a named corner. An adblocker on *.vercel.app, a Vercel outage, or a 429 from the 15-per-10-minutes limit (shared office IPs will hit that) all end with no button, no signal, and a reader who concludes their browser is broken.
  • script.async = true does nothing here; scripts built with createElement are already async.
  • Worth setting script.dataset.apiUrl, which the widget already supports, rather than leaning on document.currentScript.

Worth follow-up issues

This is the first executable code in the repo (git ls-tree -r origin/main finds zero .js), and nothing records that anywhere. AGENTS.md §1 doesn't mention that a .js under the content root runs on every published page, CODEOWNERS has no rule for executable assets so a root .js draws the same catch-all review as prose, and CI can't see .js at all. Worth remembering submodules/developer-tooling sits under that root too.

On privacy: the Note covers accuracy but not that the question leaves the site. There's no privacy policy anywhere on docs.celo.org, GA4 landed three commits ago, and the loader sets no referrerPolicy, so the host collects IP plus exact page for every reader with no interaction at all. A line saying where questions go, plus a warning against pasting secrets next to the input, would cover it.

Finally, #2285 adds seo-schema.js with the same injection bootstrap and its own restatement of the Mintlify mechanism. Whichever of the two merges second, it would be good to document that mechanism once in AGENTS.md and have both files point at it.

Nits: Refs #2250 rather than "Closes the implementation half", since the ops boxes are still open, and AGENTS.md §9 asks for mint broken-links output in the body.

…laims

Blockers:
- The loader injected on every host while the API only accepts
  docs.celo.org, so on `mint dev` and previews the button appeared,
  took a question, then failed CORS. It now runs only on hosts the API
  accepts, and localhost is being added to the allowlist so the feature
  can actually be reviewed.
- The page promised the assistant "says so instead of guessing" for
  uncovered questions, directly after the caution to follow citations.
  Uncited answers are a case the UI handles with an escalation button,
  not a refusal, so the guarantee withdrew the page's only safety
  warning. Describes the button instead and keeps the caution.
- Says where unanswered questions land rather than implying a feedback
  loop that is not visible here.

Also: an onerror on the injection, since an ad blocker or outage
otherwise leaves no button and no explanation; an explicit
data-api-url; strict-origin referrer policy; and the no-op async
dropped.

Docs and AGENTS.md: the assistant added to the frontmatter description
so it reaches llms.txt, the table's URL column renamed to Where since
its first row holds a button label and the table ships in
llms-full.txt, the AI tooling row and the fixed entry-point count
updated, and §1 now records that any .js under the content root runs
on every published page.

mint broken-links: success, no broken links found.

Refs #2250

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@GigaHierz

Copy link
Copy Markdown
Contributor Author

Worked through all three blockers and the smaller items. One correction and one thing I'd like your read on.

Blockers

Widget breaks outside production. Confirmed — I re-ran your preflight against the branded host and it still returns only https://docs.celo.org. Two changes, because the guard alone would leave reviewers with no widget at all: assistant.js now injects only on hosts the API accepts (docs.celo.org, localhost, 127.0.0.1), and http://localhost:3000 is being added to ALLOWED_ORIGINS so mint dev actually exercises it.

Also fixed the symptom you traced: the catch rendered err.message for any failure, so Failed to fetch appeared inside the answer bubble as though the assistant had said it. Only errors the API itself returns are shown verbatim now.

The Note promised something the widget contradicts. Your reading is right and the wording was mine. An uncited answer is a case the UI is built to handle, so it isn't a refusal, and putting the guarantee immediately after "follow the citations before relying on anything load-bearing" withdrew the page's only caution. The guarantee is gone, the escalation button is described, and the caution is now a <Warning> that says an answer with no links is not grounded in the docs.

On the gap loop: it does exist, but you were right that nothing in this diff showed it. Unanswered questions are logged with the pages cited, and a weekly cron now clusters them and files one issue in this repo. The page says so rather than gesturing at a process. One issue per week, not per question — the same gap arrives phrased several ways, and off-topic questions come back uncited too, so per-question issues would outpace triage.

CNAME before merge. Already done, in 1dbb436 — the loader points at docs-assistant.celo.org, so no auto-generated hostname is referenced from a docs page and the swap window you describe never opens. The PR body was stale; rewritten.

Smaller items

All fixed: AI-tooling row and the fixed entry-point count in AGENTS.md, the frontmatter description (so the assistant reaches llms.txt), the table column renamed to Where, an onerror on the injection, the no-op async dropped, and an explicit data-api-url. Added referrerPolicy = 'strict-origin' for the privacy point.

One correction

#2285 no longer adds seo-schema.jsgh pr view 2285 --json files returns ["docs.json"] only; it now uses Mintlify's native seo.organization. So there's no second injection bootstrap to reconcile. I documented the mechanism once in AGENTS.md §1 anyway, since your underlying point stands: this is the first executable code in the repo and nothing recorded what a .js under the content root does.

Your read on two things

CODEOWNERS. Adding a rule for *.js seems right, but I'd rather you pick the owning team than guess.

Privacy. The widget disclaimer now says questions go to Anthropic, that they're logged to find gaps, and not to paste secrets; the page repeats it. That's disclosure, not a policy — docs.celo.org still has none, and GA4 landed days ago. Worth its own issue rather than something I settle here.

mint broken-links: success no broken links found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants