feat(pi): add pi.dev harness + self-driven kata TDD flow with DES enforcement - #75
Open
emdeha wants to merge 21 commits into
Open
feat(pi): add pi.dev harness + self-driven kata TDD flow with DES enforcement#75emdeha wants to merge 21 commits into
emdeha wants to merge 21 commits into
Conversation
Add the nWave DES extension for the pi coding agent (pi.dev) as a thin protocol translator: pi lifecycle events -> Claude Code JSON hook protocol -> existing Python DES adapter (zero adapter fork, mirrors the OpenCode shim). pi has no subagents, so enforcement rides per-tool-call events: tool_call (PreToolUse, can block) and a session_start startup self-test that confirms the DES engine is reachable and gating is live for an activated project. Validated end-to-end against real pi 0.79.9 by a @walking_skeleton @driving_port acceptance test that loads the rendered extension and asserts the "[nWave DES] enforcement active for pi" health line — model-free. Also lands the DISCUSS feature-delta + slice briefs and the SPIKE-0 findings (the tool_call interception + adapter-bridge assumption, proven), and bootstraps the product SSOT (jobs/persona/journey). Step-Id: pi-harness-ws-01 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ce scenarios, RED scaffolds DESIGN (peer-approved): SSOT architecture brief (## Application Architecture, C4 L1+L2) + ADR-pi-001 (D6 Hybrid step-completion model), adr-pi-002 (extension/installer reuse), adr-pi-003 (crafter skill wiring). Zero adapter fork held; SubagentStopService reused unchanged with a commit-boundary trigger. DISTILL (Sentinel-approved): 21 acceptance scenarios across installer-lifecycle, extension-translation, and tdd-enforcement-gates feature files (8 green, 12 RED-for-right-reason, 1 @requires_external skip). RED scaffold pi_des_plugin.py (__SCAFFOLD__, AssertionError). atdd-infrastructure-policy.md bootstrapped. SPIKE-1 (model-free, source inspection): resolved the DESIGN-review HIGH on unproven driving ports — pi 0.79.9 type decls confirm the bash tool_call is the same blockable event class as write, BashToolInput.command is inspectable, and the bash tool_result carries exitCode. R2 downgraded to LOW. Final review gate cleared: Sentinel APPROVED, Eclipse APPROVED (post-fix), Architect CONDITIONALLY_APPROVED with A1–A3 documented as DELIVER action items. Step-Id: pi-harness-distill-01 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…step 01-01 - Acceptance test: installer-plugin-lifecycle.feature (all 8 scenarios GREEN) - Render pi-des-extension.ts.template into PI_CONFIG_DIR, embedding resolved python interpreter + DES lib path via install_paths resolvers - validate_prerequisites/install/verify/uninstall mirror the OpenCode mould; missing pi dir skips with success (R1), missing DES lib refused with guidance - Register PiDESPlugin in PluginRegistry guarded by "pi" in target_platforms, set_dependencies(["des"]) - Refactoring: none (clean; mirrors proven sibling, no speculative abstraction) Step-Id: 01-01 Task-Id: pi-harness Co-Authored-By: Claude <noreply@anthropic.com>
…tions - step 01-02 Extend the pi extension translator (ADR-PI-001 D6 Hybrid): - bash `git commit` tool_call -> subagent-stop step-completion validation - bash test-run tool_result -> post-tool-use suite-state recording (non-blocking) - pin DES_AUDIT_LOG_DIR to project .nwave/des/logs on every spawn - extract relayEngineVerdict so commit + write share the single block decision - Acceptance test: commit->subagent-stop and test-run->post-tool-use translation - Refactoring: L1 (isGitCommitCommand intent name) + L3 (relayEngineVerdict DRY) Step-Id: 01-02 Task-Id: pi-harness Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…te - step 01-03 Turn the two RED step-completion gate scenarios GREEN by driving the unchanged subagent-stop / GitCommitVerifier engine model-free at the commit boundary, and add the A2 durable-trust-boundary scenario. - Drive the gate scenarios via the direct DES protocol (executionLogPath/ projectId/stepId/cwd) with constructed execution-log + git commit state, so the engine -- not the test fixture -- renders the block/allow verdict. - RED-gate observable anchored on the standing execution-log sentinel (pre-write exit 2). - A2: an out-of-harness commit (Step-Id without matching Task-Id) is caught post-hoc by GitCommitVerifier AND-semantics (COMMIT_NOT_VERIFIED). - Relay reconciliation: relayEngineVerdict now surfaces the subagent-stop JSON block signal (decision:block, exit 0) in addition to the pre-write exit-2 signal -- still exactly one block decision, the engine owns it (D5). - Acceptance test: incomplete-cycle commit rejected; completed-cycle trailered commit accepted; out-of-harness commit caught by the verifier - Refactoring: L1-L6 assessed -- code already clean (named helpers/constants, no magic values, no deep nesting); no transformation added value Step-Id: 01-03 Task-Id: pi-harness Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…olution archive All 3 TDD steps GREEN (21 passed / 0 failed / 1 skipped). DES integrity verified (complete RED→GREEN→COMMIT traces). Adversarial review APPROVED, 0 defects, no testing theater. Records DoD check, demo evidence ([nWave DES] enforcement active for pi), and quality-gate outcomes; archives evolution context. Step-Id: pi-harness-finalize Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…DISCUSS→DESIGN→SPIKE→DISTILL)
Driving/bootstrap layer so a single pi agent self-runs the nWave DELIVER TDD
cycle for a kata (no orchestrator/subagents), with the shipped commit gate
verifying. Zero DES engine change (K2).
- DISCUSS: K1 commit-boundary-verified, K2 no engine change, K3 mechanism→DESIGN,
K4 scope=DELIVER cycle; 3 stories/slices reusing job pi-tdd-enforced-crafter.
- DESIGN (ADR-pkt-001): KD1 hybrid (pi registerCommand bootstrap + crafter skill),
KD2 kata-manifest + NN-NN steps, KD3 install-resolved python -m des.cli.*,
KD4 synthesized-transcript activates the unchanged subagent-stop.
- SPIKE: PROVEN — unchanged subagent-stop blocks incomplete / allows complete via
synthesized transcript; src/des untouched. Also found the shipped pi commit gate
currently always-allows (missing transcript context) — this feature activates it.
- DISTILL: 11 scenarios (10 RED-for-right-reason, 1 @requires_external skip) +
RED scaffolds (scripts/install/pi_kata/{bootstrap,transcript_context}.py).
- Review gate cleared: Eclipse + Architect APPROVED; Sentinel conditional → A-PKT-1
(state-delta consistency) documented as DELIVER action item.
Step-Id: pi-kata-tdd-flow-plan
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ifest writer - step 01-01 - Acceptance test: slice-01-bootstrap (bootstrapped/refused/idempotent) - Implement bootstrap_kata_session, read_kata_manifest, advance_step_id; remove __SCAFFOLD__ - Reuse des.cli.init_log UNCHANGED (K2); spawn via resolved interpreter + PYTHONPATH (D-PKT-3) - Atomic manifest write (temp-then-move) + re-read-assert (ADR-PKT-001 H3) - Activation gate on .nwave/local-config.json enabled_for_repo; observable refusal contract - Refactoring: L1-L6 assessed, already clean (no transformation added value) Step-Id: 01-01 Task-Id: pi-kata-tdd-flow Co-Authored-By: Claude <noreply@anthropic.com>
@contract-shape:* tags cannot be registered via addinivalue_line (pytest truncates the marker name at the first colon). Mirror the established pytest_bdd_apply_tag pattern (installer_orphan_sweep) to consume wave-metadata tags without marking. Unblocks pi_kata_tdd_flow collection under the repo's default --strict-markers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ler - step 01-02 - Acceptance test: slice-02 self-driven cycle (3/4 target scenarios GREEN) - build_transcript_content / write_synthesized_transcript / build_subagent_stop_payload assemble the Claude-Code-protocol transcript carrying the four exact DES markers feeding the UNCHANGED subagent-stop engine (ADR-PKT-001 KD4, zero src/des change) - Refactoring: L1 (extracted _VALIDATION_MARKER constant) Step-Id: 01-02 Task-Id: pi-kata-tdd-flow Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…(step 01-02)
The malformed-transcript scenario asserted json.loads(stdout)['decision']=='allow',
but the unchanged subagent-stop engine signals allow/no-context as exit 0 + EMPTY
stdout (same contract as pi-harness _write_allowed) — json.loads('') errored.
Production code (transcript_context.py) verified correct; only the AT assertion
was wrong. Now matches the real engine contract.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… content The 'a later step is committed without recording its cycle' Given called commit_with_trailers on a clean tree (prior step's record_complete_cycle staged the only changes), so git commit failed 'nothing to commit' before the engine was reached. Write a real source change for the skipped step so the commit carries content but no recorded cycle — the boundary then blocks on the incomplete cycle (same engine path proven green by slice-02). Scaffold-only fix; no production code touched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… extension and plugin - step 01-03
- Extension template (EXTEND, pure relay): add a pi.registerCommand("kata")
bootstrap handler delegating to pi_kata.bootstrap via install-resolved
python -c (KD1/KD2/KD3), and a commit-context assembler on the existing bash
git-commit -> subagent-stop route that synthesizes the Claude-Code transcript
(four DES markers via pi_kata.transcript_context) and feeds
{agent_transcript_path, cwd} so the shipped-but-inert commit gate now verifies
with GitCommitVerifier (KD4). Both relay the single engine verdict -- no new
block decision (thin-translator contract preserved: exactly one block: true).
- Plugin (EXTEND): render the new {{KATA_PYTHONPATH}} placeholder (parent of the
importable pi_kata package) and place the crafter-skill pi-runnable
cycle+recording section (software-crafter/SKILL.md, ADR-PI-003) rendered with
the install-resolved python + DES lib path; manifest tracks the skill file;
verify + uninstall updated.
- Acceptance: pi_kata 10 passed / 1 skipped, pi_harness 21 passed / 1 skipped
- A-PKT-1: documented exemption kept (layer-3 @real-io example-only, Mandate 11;
test files are DISTILL-authored -- not edited per SLIM scope)
- ZERO src/des change (K2)
Step-Id: 01-03
Task-Id: pi-kata-tdd-flow
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…og, evolution All 3 steps GREEN (pi_kata 10 passed / 1 skipped; pi-harness 21/1 no regression). DES integrity: complete RED→GREEN→COMMIT traces. Adversarial review APPROVED, 0 defects, no testing theater; 3 orchestrator test-infra fixes judged legitimate. Zero src/des change (K2). Records DoD check, demo evidence (bootstrap → step 01-01), quality gates, and an evolution archive with the DES-hook + colon-tag gotchas. Step-Id: pi-kata-tdd-flow-finalize Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The pi extension placed software-crafter/SKILL.md on disk but never
surfaced it to pi -- the resources_discover handler ADR-PI-003 mandated
was missing. The model therefore never received the des-log-phase
recording contract, so live kata phases were never recorded and
execution-log.json stayed empty (DoD not met).
- Regression test: rendered extension registers a resources_discover
handler returning the installed software-crafter skill dir in
skillPaths; verify asserts discoverability, not just file presence
- Add resources_discover handler returning { skillPaths: [<skill dir>] },
resolved relative to the extension's own location (pi 0.79.9
ResourcesDiscoverResult shape)
- Harden PiDESPlugin.verify to reject an extension that does not expose
resources_discover surfacing the skill dir
- Extension stays a PURE relay (thin-translator contract green); zero
src/des change
Step-Id: 01-01
Task-Id: fix-pi-kata-phase-recording
Co-Authored-By: Claude <noreply@anthropic.com>
…nistic step-id advance - step 01-02 Closes the test-coverage gap that let the empty-execution-log bug ship green. The slice-03 steps recorded phases through a test-rigged interpreter (sys.executable + src on PYTHONPATH), so a broken install-resolved des-log-phase spawn would still have gone green. Now the steps drive recording through the canonical skill-prescribed path (install-resolved python -m des.cli.log_phase), exercising the live wiring exactly as the model runs it. - Acceptance test: slice-03 multi-step kata records each step's cycle in plan order via the skill-prescribed spawn; RED proven by AttributeError on the absent bootstrap.record_phase/current_step_id live-wiring path - bootstrap: add canonical record_phase + current_step_id; extract shared _spawn_des_cli (init-log + log-phase share one install-resolved spawn shape) - advance_step_id wired as the canonical step-advance (was dead code, no caller): each cycle records a FRESH NN-NN so the commit gate never re-validates a step-id and trips the engine's second-attempt-allow (SPIKE constraint 3) — incomplete-cycle block achieved install-side, ZERO src/des change (K2) - Refactoring: L1 (LOG_PHASE_MODULE/PHASE_STATUS_EXECUTED constants) + L3 (_spawn_des_cli dedup), remove dead _run_des_cli test shortcut Step-Id: 01-02 Task-Id: fix-pi-kata-phase-recording Co-Authored-By: Claude <noreply@anthropic.com>
… deterministic step-id Closes the live phase-recording gap found by dogfooding (execution-log stayed empty in a real pi kata run → DoD not met). Corrected RCA (nw-bugfix): - ROOT CAUSE: the pi extension had NO resources_discover handler, so the placed software-crafter skill (carrying the des-log-phase recording contract) was never surfaced to pi → the model never recorded phases. (The signal/non_des theory was a red herring — the signal only enriches audit metadata.) - FIX (zero src/des change): add resources_discover → skillPaths in the extension template; PiDESPlugin.verify asserts discoverability (not just file-exists); wire the previously-dead advance_step_id so each RED gets a fresh NN-NN — which keeps an incomplete step BLOCKED at the commit boundary (a reused step-id would hit the engine's second-attempt-allow; fresh ids never re-validate, so the first-attempt BLOCK stands). Engine unchanged. - Regression tests (the previously-absent live-wiring coverage): skill discoverability + extension-path-driven recording landing in execution-log + incomplete-cycle-blocks-at-commit. Suite 35 passed / 2 skipped. Step-Id: fix-pi-kata-phase-recording Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…g command
The crafter skill instructed the model to record phases with
'{python} -m des.cli.log_phase ...' — a bare invocation that cannot import des
(in dev, and in a real install where des lives at the resolved lib dir, not on
the interpreter's path). So even with the skill delivered (resources_discover),
the model's recording call would fail -> empty execution-log. Surfaced by the
live dogfood run. Fix: render 'PYTHONPATH={lib} {python} -m des.cli.log_phase'
inline + a run-verbatim note. Regression guard asserts every rendered
des.cli.log_phase command carries an inline PYTHONPATH=. This makes phase
recording work out-of-the-box. Zero src/des change. Suite 36 passed / 2 skipped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… step 01-01 The live commit gate fired on the bash git commit tool_call (PRE-execution), then ran GitCommitVerifier which failed (COMMIT_NOT_VERIFIED: 'no commits yet') because the commit did not exist yet. Move the live gate to phase-completeness only (suppress engine commit verification by omitting DES-PROJECT-ROOT marker + cwd); commit-trailer provenance becomes a post-hoc check (step 01-02). Reorder the crafter skill so the COMMIT phase is recorded BEFORE git commit. - Acceptance test: complete cycle ALLOWED + incomplete cycle BLOCKED at the pre-commit boundary before any commit exists (live-faithful regression) - Refactoring: L1 (clear names/docstrings) + L2 (extract _event_types helper) - ZERO src/des change; pi_kata + pi_harness suites green (38 passed, 2 skipped) Step-Id: 01-01 Task-Id: fix-pi-kata-commit-gate-timing Co-Authored-By: Claude <noreply@anthropic.com>
The LIVE gate verifies phase-completeness at the pre-commit tool_call; it cannot verify a not-yet-existing commit. This post-hoc checker closes the recorded-but-not-committed gap: every step that recorded a COMMIT phase must map to a real Step-Id+Task-Id-trailered commit (AND-semantics), by REUSING the UNCHANGED engine GitCommitVerifier. Zero src/des change (K2). - Acceptance test: provenance passes on a faithfully-committed kata; fails on a recorded-but-not-committed step and on wrong/missing trailers - ADR-PKT-001 D-PKT-4 amended: settled live-gate vs post-hoc-provenance split Step-Id: 01-02 Task-Id: fix-pi-kata-commit-gate-timing Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…t-hoc provenance Closes the live commit-gate defect (dogfood: COMMIT_NOT_VERIFIED "no commits yet", commit proceeded anyway). RCA: GitCommitVerifier can't pass at a PRE-commit tool_call (commit doesn't exist yet); COMMIT phase was recorded post-commit; the model-free tests masked it via commit-then-validate order. Settled design (install-side, zero src/des change): - LIVE gate = phase-completeness at the pre-commit tool_call: the commit-context payload omits cwd + the DES-PROJECT-ROOT marker (keeps VALIDATION/PROJECT-ID/ STEP-ID) so the engine skips GitCommitVerifier and blocks purely on missing RED/GREEN/COMMIT phases. - Crafter skill records the COMMIT phase BEFORE git commit (so a genuine cycle passes at pre-commit; an incomplete one blocks on missing phases). - Commit-trailer PROVENANCE moves to a post-hoc backstop (scripts/install/pi_kata/ provenance.py) reusing the UNCHANGED GitCommitVerifier (AND-semantics). - ADR-pkt-001 D-PKT-4 amended with the split; live-faithful pre-commit regression tests + provenance tests added. Suite 41 passed / 2 skipped. Step-Id: fix-pi-kata-commit-gate-timing Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
|
The following files are on the sensitive-file denylist and cannot be
These files are typically managed by the release pipeline or contain This PR remains open for discussion. No mirror branch has been created. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds pi.dev as a fourth nWave harness and a self-driven kata TDD flow on top of it, so a single pi agent (no orchestrator, no subagents) runs the nWave DELIVER cycle under DES enforcement. All install-side — zero
src/des/**change (the Python DES engine is reused unchanged).Feature 1 —
pi-harnesspi_des_plugin.py), mirroring the OpenCode shim.pre-write, bashgit commit→subagent-stop, bash test-run →post-tool-use,session_start→ health line.Feature 2 —
pi-kata-tdd-flow/katabootstrap command +kata-manifest.json+ crafter skill (delivered via the extension'sresources_discover) so a single pi agent self-runs RED→GREEN→COMMIT, recording each phase viades-log-phase.provenance.py, reusingGitCommitVerifier) verifiesStep-Id/Task-Idcommit trailers.claude-opus-4-8: a full strict-TDD step recorded + committed + gate-passed, out of the box.Methodology
Both features ran the full wave flow (DISCUSS→DESIGN→SPIKE→DISTILL→DELIVER) with consolidated reviews; SPIKE-0/1 de-risked pi's
tool_call/tool_resultblockability and the subagent-stop activation. Three dogfooding-driven bugfix cycles (RCA → regression test → fix) closed: missingresources_discover, missing inlinePYTHONPATHon the recording command, and a commit-gate timing defect (gate fired pre-commit against a verifier needing the commit to exist).Tests
pi suites: 41 passed / 2 skipped (
@requires_externallive-model). Zerosrc/des/**change verified at every step.Try it yourself
Prerequisites:
piinstalled (@earendil-works/pi-coding-agent, tested on 0.79.9), and a model configured for pi (e.g. an Anthropic API key — the examples useclaude-opus-4-8). Check out this branch anduv sync.1. Deterministic proof (no model needed)
The whole behavior — bootstrap, phase recording, the pre-commit gate (allow complete / block incomplete), and post-hoc provenance — is covered against the real DES engine:
Expect ~
41 passed / 2 skipped(the skips are the live-model scenarios).2. Live, in real pi (needs a model)
Stage a throwaway kata project with the rendered extension + crafter skill, then drive it:
3. Verify what happened
Follow-ups (non-blocking)
provenance.check_kata_provenanceinto CI (the backstop the live gate can't cover).public:flags for the new pi/pi_kata assets inframework-catalog.yamlbefore any release sync.🤖 Generated with Claude Code