Skip to content

fix(test): count examples/* source in the coverage denominator - #749

Merged
sroussey merged 1 commit into
claude/coverage-dist-bundle-fix-ew0vj8from
claude/coverage-denominator-examples
Aug 13, 2026
Merged

fix(test): count examples/* source in the coverage denominator#749
sroussey merged 1 commit into
claude/coverage-dist-bundle-fix-ew0vj8from
claude/coverage-denominator-examples

Conversation

@sroussey

@sroussey sroussey commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

Based on #741 (retarget to main once that merges). Rebased onto origin/main (67bed681). Independent of #748 — deliberately separate, since adding a CI job and changing the denominator in one PR makes the coverage delta unreadable.

⚠️ build is red, inherited from main

main has not built since 5b94e05a@workglow/anthropic#build-types fails with Anthropic_StructuredGeneration.ts(57,41): error TS2739 … missing max_tokens, messages, model. This branch previously passed build only because its base predated the break; after rebasing onto current main it inherits the failure. Not caused by this PR and not fixed here — another agent owns that fix. Every job downstream of build is blocked by it.

What

examples is a first-class workspace group in both scripts/lib/workspaceSource.ts (WORKSPACE_GROUPS) and scripts/lib/testDiscovery.ts. All three example packages are published and none is private@workglow/cli, @workglow/eval, @workglow/web, all at 0.3.38 — and they carry 24 test files between them (7 / 14 / 3).

coverage.include listed only packages/*/src and providers/*/src. So example source was rewritten to src by the resolver, executed by its own tests, and then omitted from the measurement. Demonstrated:

# before
$ cd examples/cli && vitest run --config ../../vitest.config.ts --project cli --coverage
File      | % Stmts | % Branch | % Funcs | % Lines
All files |       0 |        0 |       0 |       0

# after
All files          |   24.15 |    14.71 |   26.22 |   24.87
 src/ui            |   26.87 |    12.19 |   26.08 |   27.54
 src/ui/components |   50.76 |    53.48 |   54.54 |   54.23
 src/ui/rows       |   44.25 |    37.73 |   41.66 |   45.03

Not merely a wrong number: 28 passing tests were contributing coverage to nothing at all.

Why this fix

include gains examples/*/src/**/*.{ts,tsx}, and the adjacent comment now names all three groups so the next reader cannot infer the omission was intentional.

coverage.root: __dirname rather than making the globs absolute. coverage.root is the documented base for include/exclude, and this config is invoked from package directories too (vitest run --config ../../vitest.config.ts --project cli), where a repo-relative glob would otherwise match nothing. It also sidesteps the question of whether vitest 4 accepts absolute glob patterns, which could not be settled here. From the repo root it is a no-op, since root and __dirname coincide.

examples/*/src/test/** added to exclude. Found while verifying: the examples keep their suites in src/test, which also holds examples/cli/src/test/chromeAvailability.ts — a test helper with no .test. in its name, so the existing filename rules miss it. It showed up as src/test | 18.75%. Counting testing-only helpers is exactly what the neighbouring packages/test/** and **/testing/** entries exist to prevent.

Tests

scripts/workspaceSource.test.ts gains the invariant that actually broke: every entry in WORKSPACE_GROUPS must be the prefix of some coverage.include glob. It reads the real vitest.config.ts (the idiom testDiscovery.test.ts already uses for project roots) rather than re-deriving it, so the resolver and the denominator cannot drift apart again.

A missing group is otherwise invisible: a coverage report shows a shorter file list, never an error — which is how this survived in the first place.

Actually executed:

  • vitest --project scripts: 3 files, 15 passed.
  • With vitest.config.ts reverted: 1 failed / 14 passed, AssertionError: expected [ 'examples' ] to deeply equal [] — it names the missing group.
  • Coverage from examples/cli before and after (output above).
  • Coverage from the repo root across two projects: example source listed, packages/* rows still present, so coverage.root did not disturb the existing globs.
  • prettier --check clean.

Not executed: a full bun run test:vitest:unit --coverage over all 518 files. Too slow here; the narrow runs above exercise both invocation styles.

Risk / blast radius

The reported total will move, and probably down — three packages' source joins the denominator at whatever their real coverage is (~24% for cli). That is a corrected measurement, not a regression, but a reviewer watching a coverage gate must expect the step change. This is why it is not bundled with #748.

Config-only otherwise. No product code, no test behaviour, nothing shipped to consumers.

Unverified

  • vitest 4's exact base directory for coverage.include was not observed directly; coverage.root was chosen because it is the documented base and does not depend on that answer. Both invocation styles were checked empirically instead.
  • Whether any coverage threshold or badge downstream is keyed to the old total.
  • The build failure inherited from main (see the note at the top).
  • Run under Node v22.22.2, not the Node 24 the repo asks for.

Copy link
Copy Markdown
Collaborator Author

test-vitest-ai-provider-api is red for an environmental reason, not this PR: xAI returns 403 — "has either used all available credits or reached its monthly spending limit" — from Xai_StructuredGeneration.ts:38. It needs a human to top up or raise the limit. Written up in more detail on #748, which shares this base and fails the same way.

This PR changes two files, both config (vitest.config.ts, scripts/workspaceSource.test.ts), and cannot reach a live provider call.


Generated by Claude Code

`examples` is a first-class workspace group in both workspaceSource.ts and
testDiscovery.ts, and all three example packages — @workglow/cli, @workglow/eval
and @workglow/web — are published with none marked private. They carry 24 test
files between them. The denominator listed only packages/* and providers/*, so
their source was rewritten to src, executed by their own tests, and then left
out of the measurement entirely: a coverage run from examples/cli reported
`All files 0%` with no file rows at all.

Pins coverage.root to the config's own directory rather than making the globs
absolute. coverage.root is the documented base for include/exclude, and this
config is invoked from package directories too
(vitest run --config ../../vitest.config.ts), where a repo-relative glob would
otherwise match nothing. It also does not depend on whether vitest accepts
absolute glob patterns.

Excludes examples/*/src/test/**: those dirs hold the example suites plus the
occasional non-`.test.` helper the filename rules cannot catch, and counting a
test helper is what the adjacent excludes already exist to prevent.

Guards the invariant that broke, in workspaceSource.test.ts: every entry in
WORKSPACE_GROUPS must be a prefix of some coverage.include glob, read from the
actual config so the two cannot drift apart again. A missing group is invisible
in a coverage report — it shows a shorter file list, not an error.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H797qbH356jjznKgUax63o
@sroussey
sroussey force-pushed the claude/coverage-denominator-examples branch from 30bc818 to b1bc569 Compare August 13, 2026 03:55
@sroussey
sroussey merged commit 4bce057 into claude/coverage-dist-bundle-fix-ew0vj8 Aug 13, 2026
10 of 11 checks passed
@sroussey
sroussey deleted the claude/coverage-denominator-examples branch August 13, 2026 05:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants