Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .changeset/engine-reach-in-route.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
---
"@taskless/cli": patch
---

Tell the routing recipe what the local engines can actually read.

`route` chose between `sg`, `vale`, and the runtime tier on the shape of the
evidence alone, and had nothing to say about language reach — so a rule over a
GitHub Actions workflow was escalated to `create-runtime-rule`, which needs a
login, because nothing stated that ast-grep parses YAML. It does. Nothing in
the repository could have said so either: the vendored ast-grep schema types
`language` as a bare string with no enum, `verify` never checks the field, and
`detect --json` reports the repo's own languages in a different vocabulary.
Vale self-reports nothing at all.

`route` now states both engines' reach, and `create-vale-rule` repeats Vale's
where a matcher is written. Both read the lists from constants pinned to the
engine versions this CLI ships, rather than from prose typed into the recipe —
an engine bump that changes what a binary parses now fails a vendor-contract
test instead of leaving a confident, wrong sentence in front of an agent.
Vale's reach was measured by probing the shipped binary, tier by tier, since it
publishes no capability listing.

The Vale half carries a hazard worth naming on its own. Vale supports
reStructuredText, AsciiDoc, XML, DITA, and MDX by shelling out to an external
converter, and this CLI ships none of them — so one such file caught by a
rule's glob exits 2 with an `E100` and abandons the whole run, silencing every
other Vale rule over every other file. `create-vale-rule` had been offering
`[*.{md,mdx}]` as its example of widening a matcher.

`.xml` is the one entry where naming the converter is not enough. It needs
`xsltproc` **and** an XSLT stylesheet, and a stylesheet is document-specific, so
there is nothing to ship and installing the program does not make `.xml`
lintable — unlike `asciidoctor`, which genuinely fixes `.adoc`. Vale says so
differently depending on the host, too: `xsltproc not found` where the program is
absent, `no XSLT transform provided` where it is present, and macOS ships
`/usr/bin/xsltproc` while a typical Linux CI image does not. The contract test
now asserts Vale's checker tag, which is the same everywhere, rather than a
substring of the converter name.
94 changes: 94 additions & 0 deletions openspec/changes/engine-reach-in-route/proposal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
## Why

`route.txt` tells an agent to pick an engine and gives it no way to know
what either local engine can read. The engine table is entirely about
evidence _shape_ — one file's tree, a document's words, more than one
file — and says nothing about language reach. The closest it comes is an
Important Note asking the agent to "choose an engine whose availability
you can assert," which asks for an assertion the recipe never equips it
to make.

The reported case is concrete: two GitHub Actions workflow rules were
routed to `create-runtime-rule`, which requires a login, because nothing
said whether ast-grep parses YAML. It does — `Yaml` is one of the 26
languages the pinned `@ast-grep/cli@0.41.0` lists in `sg run -h`.

Nothing in the repository could have answered the question.
`src/generated/ast-grep-rule-schema.json` types `$defs.Language` as a
bare string with no enum; `verify` never validates a rule's `language`,
so any spelling passes locally and fails inside the binary; and
`detect --json` reports the _repository's_ languages in a different
vocabulary (`C++` where ast-grep says `Cpp`). Vale self-reports nothing
at all. The binaries are the only authority.

## What Changes

- **A new `packages/cli/src/rules/capabilities.ts`** carries each local
engine's reach as pure data, pinned to the engine version it was taken
from. Pure is load-bearing: `src/prompts/recipes.ts` is a Worker-safe
surface and `assert-prompts-graph` fails the build if the prompts
chunk reaches a host capability, so the constants cannot be read from
a binary or a file at render time.
- **The recipes state reach through `%(…)s` variables**, resolved from
those constants rather than transcribed into the `.txt`. Transcribed
lists would drift on the next engine bump with nothing to catch them,
and a stale claim about what an engine can read is worse than the
silence it replaces — an agent acts on it.
- **`route.txt` (topic v1 → v2)** gains a reach block in step 4: what
`sg` parses, that an Actions workflow is `Yaml` and `Yaml` is on the
list, Vale's markup / comments-only / plaintext-fallback tiers, and
the converter-dependent formats Vale cannot read at all. It also
states that a language on neither list does **not** route to runtime by
default — `create-legacy-rule` is checked first, because the repo's own
linter may already speak it.
- **`create-vale-rule.txt` (topic v2 → v3)** repeats the reach where a
matcher is actually authored, and stops offering `[*.{md,mdx}]` as the
worked example of widening a glob. That example is a trap: one `.mdx`
file fails the entire Vale pass.
- **Two vendor-contract test blocks** pin the constants to the binaries.
ast-grep is asserted by set-equality against `sg run -h`; Vale, which
self-reports nothing, is probe-measured, with each tier separated by a
construct that distinguishes it from the fallback.

**Delivery is a single PR.** Constants, recipe text, spec delta, and
tests are one reviewable diff, and no slice of it is independently
useful — the constants exist only to be rendered, and the recipe text is
wrong without them.

## Capabilities

### Modified Capabilities

- `cli-rule-routing`: the `route` recipe states each local engine's
language reach, derived from the pinned engine versions rather than
transcribed into the recipe text.

## Impact

- **Added**: `packages/cli/src/rules/capabilities.ts` — pure data, no
imports at all.
- **Modified**: `packages/cli/src/prompts/recipes.ts` — six new entries in
`buildVariables`, all build-time constants. Compatible with the
`TASKLESS_CLI` rule that a recipe variable is an argument and never an
ambient read: nothing here touches `process`.
- **Modified**: `packages/cli/src/agent/route.txt` (topic v1 → v2) and
`packages/cli/src/agent/create-vale-rule.txt` (topic v2 → v3).
- **Modified**: `packages/cli/test/ast-grep-vendor-contract.test.ts` and
`packages/cli/test/vale-vendor-contract.test.ts` each gain a separate
top-level `describe` for the capability constants;
`packages/cli/test/recipe-cross-references.test.ts` gains a block
asserting the constants reach rendered text;
`packages/cli/test/prompts.test.ts` has its `buildVariables` key list
and its prompts-import allowlist extended.
- **Out of scope**: bumping Vale past 3.18.0, where MDX parses natively
and `.mdx` leaves the converter-dependent list. The constants are
shaped so that bump is a one-file edit.
- **Out of scope**: a runtime guard that catches an `E100` before it
takes down a `check`. This change makes the hazard visible at authoring
time; it does not defend against one already committed.
- **Out of scope**: extending `detect --json` with an `engines` block.
`openspec/specs/cli-detect/spec.md` frames `detect` as an offline scan
of the _repository_; engine reach is knowable at build time and does
not need a third subprocess turn in a recipe already spending two.

**Tracking:** taskless/cli#151
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
## ADDED Requirements

### Requirement: Route states each local engine's reach from the pinned engine versions

The `route` recipe SHALL state what each local engine can actually read, so
that an agent choosing between `create-sg-rule`, `create-vale-rule`, and the
runtime destinations is not left to guess whether an engine parses the language
in front of it.

The recipe SHALL state, for the ast-grep engine, the set of languages the
pinned ast-grep release parses, spelled as a rule's `language:` field must
spell them. It SHALL state, for the Vale engine, that Vale treats a file
according to its extension in tiers — markup, comment text only, and a
plaintext fallback that lints an unparsed file as whole-file prose — and that a
set of formats fails rather than lints because Vale delegates their parse to an
external converter this CLI does not ship. The recipe SHALL state that such a
failure aborts the entire Vale pass rather than skipping the offending file.

These statements SHALL be **derived from the pinned engine versions rather
than transcribed into the recipe text**. The recipe SHALL carry substitution
markers resolved at render time from constants that name the engine version
they were taken from, and those constants SHALL be pinned to the engines'
observable behaviour by tests that invoke the engine binaries. A transcribed
list would go stale on the next engine bump with nothing to detect it, and a
stale claim about engine reach is more harmful than no claim, because an agent
acts on it.

The recipe SHALL NOT treat a language absent from both engines' reach as
automatically requiring the runtime tier. It SHALL direct the agent to consider
`create-legacy-rule` first, since a linter the repository already runs may
cover the language, and that destination requires no login.

The recipe SHALL distinguish an engine's reach from an engine's availability.
Reach is a property of the pinned engine version; availability is a property of
the host on which the CLI is running, and a language within reach is still
unusable where the engine's platform binary did not resolve.

#### Scenario: Route names ast-grep's languages

- **WHEN** the rendered `route` recipe is read
- **THEN** it SHALL name the languages the pinned ast-grep release parses, including `Yaml`
- **AND** an agent SHALL be able to conclude from it that a rule over a GitHub Actions workflow is expressible as an `sg` rule

#### Scenario: Route names Vale's tiers and its unreadable formats

- **WHEN** the rendered `route` recipe is read
- **THEN** it SHALL name the extensions Vale parses as markup, the extensions where Vale lints comment text only, and the plaintext fallback that applies to everything else
- **AND** it SHALL name the formats whose parse Vale delegates to an absent external converter
- **AND** it SHALL state that one such file fails the whole Vale pass rather than only itself

#### Scenario: The reach statements carry no unresolved marker

- **WHEN** the `route` recipe is rendered
- **THEN** no `%(…)s` substitution marker SHALL remain in the text an agent receives

#### Scenario: A bumped engine cannot leave the recipe stale

- **WHEN** an engine binary is upgraded to a version whose reach differs from the constants the recipe renders
- **THEN** a vendor-contract test that invokes the engine SHALL fail
- **AND** the recipe SHALL NOT be able to state the superseded reach without that failure

#### Scenario: An unreachable language is not routed to runtime by default

- **WHEN** the rule's language appears in neither engine's reach
- **THEN** `route` SHALL direct the agent to consider `create-legacy-rule` before escalating to a runtime destination
30 changes: 30 additions & 0 deletions openspec/changes/engine-reach-in-route/tasks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
Delivery shape: **single PR**. Constants, recipe text, spec delta, and tests are one diff — the constants exist only to be rendered, and the recipe text is wrong without them, so no slice lands independently.

## 1. Constants

- [x] 1.1 Add `packages/cli/src/rules/capabilities.ts` with the ast-grep language list verbatim from `sg run -h` at the pinned `0.41.0`, and the pinned version beside it
- [x] 1.2 Probe the shipped Vale 3.17.1 binary for its tiers rather than trusting any prior table, and record markup, comment-only, and converter-dependent extensions with the converter each failure names
- [x] 1.3 Keep the module free of imports entirely, so `assert-prompts-graph` still passes when `src/prompts/recipes.ts` reaches it
- [x] 1.4 Add the render helpers that turn each list into recipe prose, so a recipe never joins an array itself

## 2. Recipes

- [x] 2.1 Extend `buildVariables` with `AST_GREP_VERSION`, `AST_GREP_LANGUAGES`, `VALE_VERSION`, `VALE_MARKUP_FORMATS`, `VALE_COMMENT_FORMATS`, and `VALE_CONVERTER_FORMATS`
- [x] 2.2 Bump `route.txt` to topic v2 and add the reach block to step 4, after the engine table's sharpening bullets and before the worked examples
- [x] 2.3 State in `route.txt` that a language on neither list is not automatically runtime, and that `create-legacy-rule` is checked first
- [x] 2.4 Separate reach from availability in `route.txt`'s Important Notes — the existing "availability you can assert" bullet is about the binary resolving on this host, which the new lists do not answer
- [x] 2.5 Bump `create-vale-rule.txt` to topic v3, repeat the reach at the matcher step, and replace the `[*.{md,mdx}]` widening example

## 3. Tests

- [x] 3.1 Add an `ast-grep engine capabilities` block parsing `Supported languages are: [...]` and asserting set-equality, with `Yaml` named explicitly so the issue's answer is an assertion rather than an implication
- [x] 3.2 Add a `Vale engine capabilities` block that probes each tier with a discriminating fixture — a construct only a parser skips for markup, a bare non-comment line for comment-only, a non-zero exit with `E100` for converter-dependent
- [x] 3.3 Assert in `recipe-cross-references.test.ts` that no rendered recipe leaks a `%(…)s` marker and that rendered `route.txt` carries the lists
- [x] 3.4 Extend the `buildVariables` key list and the prompts import allowlist in `prompts.test.ts`

## 4. Verification

- [x] 4.1 `pnpm openspec validate engine-reach-in-route --strict`
- [x] 4.2 `pnpm --filter @taskless/cli build` — the prompts-graph plugin is the real check that `capabilities.ts` stayed pure
- [x] 4.3 `pnpm typecheck`, `pnpm lint`, and `pnpm --filter @taskless/cli test`
- [x] 4.4 Add the changeset
38 changes: 35 additions & 3 deletions packages/cli/src/agent/create-vale-rule.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Topic: create-vale-rule (CLI v%(CLI_VERSION)s / topic v2)
# Topic: create-vale-rule (CLI v%(CLI_VERSION)s / topic v3)

## You are here
This is `create-vale-rule`. It helps you write a Vale rule: a check over
Expand Down Expand Up @@ -271,8 +271,10 @@ it.

- `[*.md]` is a **matcher**: a glob over paths, deciding which files
this rule sees. Match it to the files the rule is actually about,
such as `[*.{md,mdx}]` or `[docs/**/*.md]`. A rule can declare
several matchers if it needs to.
such as `[*.{md,markdown}]` or `[docs/**/*.md]`. A rule can declare
several matchers if it needs to. Before you widen a glob, check the
reach table below — what Vale does to a file it cannot parse is not
"nothing".
- `tskl) rule = <id>` is a breadcrumb Taskless reads to attribute the
matcher back to this rule after assembly interleaves every rule's
matchers into one file. Vale parses the key and ignores it. Write it
Expand All @@ -294,6 +296,36 @@ it.
`[…]` line belongs to no matcher, and Vale ignores it after warning
on stderr.

**What a matcher's glob is allowed to catch.** Vale (v%(VALE_VERSION)s)
treats a file one of four ways, decided by extension. The lists are
rendered from the pinned Vale version, not written out here, so they
track the shipped binary.

- **markup** — the document is prose and the format's own non-prose
constructs are skipped. This is the tier every `scope:` value
assumes; `scope: heading` has nothing to find outside it:
%(VALE_MARKUP_FORMATS)s
- **comment text only** — the comments are linted and the code body
is invisible, which is exactly right for "comments must not say
'obviously'":
%(VALE_COMMENT_FORMATS)s
- **plaintext fallback** — everything else, `.yml` `.toml` `.sh`
`.sql` and every extension not named above included. There is no
parser, so the whole file is linted as prose: a rule matched to
YAML flags key names and values, not just the comments. If that is
not what the rule means, narrow the glob rather than accepting it.
- **cannot be read at all** — Vale supports these formats but shells
out to an external converter to parse them, and this CLI ships none
of those converters:
%(VALE_CONVERTER_FORMATS)s

**A single unreadable file fails the whole Vale pass.** Vale exits 2
with an `E100` runtime error and abandons the run — `--no-exit` does
not suppress it — so every other Vale rule over every other file goes
unreported. `[*.{md,mdx}]` is not a slightly wider `[*.md]`; it is a
matcher that takes `check` down the first time the repo grows an
`.mdx` file. Never put one of those extensions in a glob.

5. **Write the fixtures.** Two directories inside the rule, both flat.
Vale lints the whole fixture tree, so a document nested a level
deeper would be linted and never checked against either bucket, which
Expand Down
Loading