feat: harden native subagent dispatch to standardization-ready — follow-up to #355 - #356
Merged
Merged
Conversation
…structural no-nesting Closes the two standardization blockers from PR #355's DEF-001 / residual risks, using native runtime enforcement, not prompt text. Blocker 1 (DEF-001, workspace isolation): - Every executor gains workspace: isolated_worktree. The orchestrator MUST dispatch with Agent isolation: "worktree" — child works in a clean, locked git worktree at a deterministic revision, never the parent working tree. Audit: observed a probe child land in .claude/worktrees/agent-<id> on a clean HEAD, isolated from the parent branch. Blocker 2 (structural no-nesting on the write path): - Audit finding: a general-purpose child CAN spawn a nested sub-agent (probe did it). Plan/Explore agent types are defined "All tools except Agent, ... Write, Edit" -> structurally cannot nest and cannot write. - Every executor.agent_type is now Plan (was general-purpose for codex). write_capable is false for all. An implementation slice returns a patch + validation commands; the ROOT applies and validates. Root is the only agent holding Agent + Write. Files: PROJECT_CAPABILITIES.yaml (executor: +workspace, agent_type Plan, write_capable false), ai-os-orchestrator/SKILL.md (structural invariants + mandatory isolation + patch-return flow + bounded failure recovery), AGENT_LOOP_PLAYBOOK.md bounds (+ regenerated AIOS_05 + provenance), tests/test_project_capabilities_executor.py (DEF-001 regression + non-nesting + no-write-capable-child invariants). Checks: 6 canonical scripts + provenance audit + pytest (216) pass. Additional observed executions + measurement evidence + decision memo follow. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…rdict STANDARDIZE BOUNDED (conditional) Native enforcement audit + Pilots D/E/F on the hardened dispatch: - Pilot D (runtime failure): Plan child structurally lacks Write -> honest failure, no workaround -> DEF-P D-001 -> bounded recovery (patch-return, root applied + validated), no silent retry. - Pilot E (write path): Plan child in isolated worktree returned a clean 2-file unified diff + validation commands; root applied and validated. subagent_tokens 41558, tool_uses 11, duration 93s. - Pilot F (long multi-hop): 5 owner transitions ai_os->thinking->analytics->codex->ai_os, one execution_id exec-pilotF-2026-09-02, root-only routing, guard-gated repeat, Closure Review. Pilot's own multi-owner verdict: STANDARDIZE BOUNDED WITH CONDITIONS. Measurement evidence: 16 total dispatches (7 hardened), isolated-worktree landing 7/7=100%, structural write refusal 1/1, nested spawn by non-nesting executor 0, silent retries 0, latency captured n=2 (93s/149s), telemetry captured opportunistically only. Risks CLOSED (native, not prompt): DEF-001 workspace isolation (mandatory isolation:"worktree", registry + test + fail-closed SKILL wording); structural no-nesting (all executors Plan = no Agent tool; write-capable child removed; patch-return + root applies). Risks REMAINING: R1 unverified live spawn args (irreducible for a prompt-level controller); R2 thin measurement base; R3 worktree isolates VCS state only; R7 no cost owner. SKILL.md gains fail-closed spawn wording. No AES-record schema change, no ROUTING_RULES/registry-role/authority/gate change, no framework. Recommendation: STANDARDIZE BOUNDED (conditional) on a 4-item punch-list (fail-closed evidence linter, telemetry schema, >=15-dispatch sample, cost owner); else KEEP PILOT-ONLY until it lands. Checks: 6 canonical scripts + provenance audit + pytest (216) pass. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
github-actions
Bot
added
the
needs-human-review
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
sergstack
added a commit
that referenced
this pull request
Sep 2, 2026
…ollow-up to #356 (#358) * feat(orchestrator): commissioning — dispatch-evidence linter, telemetry contract, 16-dispatch sample, cost owner Closes the STANDARDIZE BOUNDED commissioning punch-list (4/4). No architecture change: no new router/state-machine/DB/framework, no AES-schema or ROUTING_RULES change, no authority expansion, no child->child, no write-capable children. 1. schemas/subagent_dispatch_evidence.schema.json + scripts/check_subagent_dispatch_evidence.py (wired into .github/workflows/docs-safety.yml as a blocking step) + tests/test_subagent_dispatch_evidence.py (13 tests). Every executed dispatch must have a machine-checkable record: agent_type (Plan/Explore only), isolation "worktree", workspace observation under .claude/worktrees/agent-, execution/owner linkage, cross-checked against PROJECT_CAPABILITIES.yaml. 2. Telemetry contract: duration_ms / subagent_tokens / tool_uses are required keys, each a number or the literal "not_captured" (never omitted or invented). Runtime emitted no usage footer for the 16 commissioning dispatches -> all recorded not_captured, honestly. 3. docs/evidence/subagent_dispatch_records_2026-09-02.json — 16 hardened dispatches across all 7 owner capabilities: - exec-commissionG: 6-hop long multi-hop + repeat-route guard (bare repeat refused; repeat with evidence_delta allowed); - exec-commissionH: 2 deliberate failures (capability-missing write; missing-input read) both honest + registered (DEF-commissionH-001/002), no silent retry, plus a patch-return recovery the root applied; - exec-commissionI: per-capability coverage; - C-PROBE-1: runtime re-verification (Plan has no Agent/Write; isolation:"worktree" = clean linked worktree; unchanged). Guard-calibration population recorded with a proposal; the four continuation.guards remain unset (schema unchanged). 4. Cost/latency owner = [AI OS], with a "when dispatch is justified vs inline root" rule, recorded in AGENT_LOOP_PLAYBOOK.md. QA: exact no-nesting wording ("a Plan child cannot use the native Agent tool"; Bash-mediated external spawn = residual risk pending separate evidence); runtime-upgrade re-verify bound added. Status stays STANDARDIZE BOUNDED (conditional) pending the Judge review; flips to STANDARDIZED BOUNDED only on judge_verdict: pass. Checks: 6 canonical + provenance + dispatch-evidence linter + pytest (229) pass. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(commissioning): apply code-review findings to the dispatch-evidence linter - acceptance gate now applies by filename match (subagent_dispatch_records*.json), not only when generated_for contains "commissioning" — a rename can no longer silently drop the >=15/owner/scenario enforcement; - registry_executors() tolerates a capability with no executor block (degrades to {} -> clean FAIL message instead of an uncaught KeyError); - missing jsonschema now raises a distinct EnvironmentError_ -> exit code 2 ("CANNOT CHECK"), not a spurious validation FAIL (exit 1); - check_file returns (problems, doc); main() no longer re-parses the file; - tests: drop unused `import copy`; drop an absolute local path from a fixture; add coverage for the filename-based gate and the (problems, doc) return. Checks: 6 canonical + provenance + dispatch-evidence linter + pytest (231) pass. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(commissioning): Judge round-1 revise — correct defect-status language Judge (adversarial) round 1 returned `revise`: the memo/status claimed the two deliberate-failure defects were "registered / in the AES defect lifecycle" with no FAILURE_REGISTRY.md entry or AES record — status inflation — and def-/DEF- casing did not match `defect_ref`. - IDs normalised to def-commission-h-001 / def-commission-h-002 in the JSON defect_ref, JSON notes, and the memo; - language corrected: the two induced failures are recorded in subagent_dispatch_records_2026-09-02.json (outcome:refused + defect_ref) as deliberate probes of the failure-handling path, NOT FAILURE_REGISTRY.md regression cases; FAILURE_REGISTRY.md and the AES schema stay unchanged; - acceptance table row 3 reworded to match. Round-2 Judge re-review result folded in next. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * feat(orchestrator): Judge round-2 pass — flip native subagent dispatch to STANDARDIZED BOUNDED Judge re-review (exec-commissionj J2, adversarial, Plan child in isolated worktree, telemetry captured 41801 tok / 15 tools / 84062 ms): round-1 gap closed, no regression on criteria 1/2/4/5/6. Verdict: pass. - dispatch records: +J1 (round-1 revise) +J2 (round-2 pass) = 18 hardened dispatches across 7 owners; J2 is the first commissioning-campaign dispatch with a captured Agent usage footer; - memo: Judge section filled from the observed result; counts 16 -> 18; telemetry capture status 1/18; §1 detail updated for the review fixes; - status flipped conditional -> STANDARDIZED BOUNDED in CURRENT_STATUS.md, MASTER_STATUS.md, docs/evidence/README.md, and AGENT_LOOP_PLAYBOOK.md (bounded, pilot-scoped; NOT a default/unrestricted standard; general promotion + guard thresholds remain separate owner decisions). judge_verdict: pass. No AES-schema / ROUTING_RULES / authority change; no child->child; no write-capable children; rollback still a scoped git revert. Checks: 6 canonical + provenance + dispatch-evidence linter + pytest (231) pass. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * docs(commissioning): sync linter docstring with the filename-based acceptance gate Follow-up to the code-review fix in e2ba3c2: the module docstring still said the acceptance gate keys only on `generated_for` containing "commissioning". It now also applies by the `subagent_dispatch_records*.json` filename (`acceptance_gate_applies`). Docstring only; no behaviour change. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Code <sstegancev@gmail.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #355 (merged as
0a7ee18). No new functionality — this closes the two standardization blockers from #355's residual risks using native runtime enforcement, not prompt text, runs 3 more observed multi-owner executions, adds measurement evidence, and delivers a decision memo.Verdict:
STANDARDIZE BOUNDED— conditional on a 4-item punch-list; elseKEEP PILOT-ONLYuntil it lands. Full memo:docs/evidence/NATIVE_SUBAGENT_DISPATCH_STANDARDIZATION_2026-09-02.md.Native enforcement audit (before any change)
Agent isolation: "worktree"is real — probe child landed in a clean, locked.claude/worktrees/agent-<id>at a deterministic revision, not the stale parent branch.general-purposechild can nest (probe spawned a sub-agent, got "PONG"). Only prompt text stopped it → insufficient.Plan/ExploreareAll tools except Agent, …, Write, Edit→ structurally cannot nest or write. No built-in has Write-without-Agent; a tracked custom type is out of scope (.claude/agentsgitignored).→ Blocker 1 closable natively. Blocker 2 closable only by removing the write-capable child.
Changes (minimal)
PROJECT_CAPABILITIES.yaml— everyexecutor:agent_type: Plan(wasgeneral-purposeforcodex),write_capable: false(all), newworkspace: isolated_worktree(all).ai-os-orchestrator/SKILL.md— two structural invariants (noAgenttool ⇒ no nesting; noWrite⇒ patch-return + root applies); mandatoryisolation: "worktree"; fail-closed spawn wording (omitting isolation/agent_type is an AEScontractdefect, not a silent default; a dispatch with no recorded workspace observation is not counted as executed).AGENT_LOOP_PLAYBOOK.mdbounds updated (hub-and-spoke "structurally enforced"; mandatory isolation; root is the only writer) + regeneratedAIOS_05bundle + provenance.tests/test_project_capabilities_executor.py— DEF-001 regression (workspace == isolated_worktreefor all), non-nesting (agent_type ∈ {Plan, Explore}), no write-capable child.docs/evidence/README.md,MASTER_STATUS.md,CURRENT_STATUS.md.Not touched: AES record schema,
ROUTING_RULES.md, registry role, root-only routing, authority/merge/deploy/production gates, child→child ban, framework ban. No runtime/DB/service, no.gitignorechange.3 more observed executions (all dispatched with
isolation: "worktree")Planchild asked to write → structurally cannot, refused honestly, nosed -iworkaround →DEF-P D-001→ bounded recovery (patch-return, root applied + validated), no silent retryPlanchild in isolated worktree returned a clean 2-file unified diff + validation commands; root applied + validated.subagent_tokens 41558,tool_uses 11,93 sai_os → thinking → analytics → codex → ai_os, 5 owner transitions, oneexecution_idexec-pilotF-2026-09-02, root-only routing, guard-gated repeat visit, Closure Review vs original goal. Pilot's own multi-owner verdict: STANDARDIZE BOUNDED WITH CONDITIONSMeasurement evidence (nothing invented; gaps marked)
16 total dispatches (7 hardened) · isolated-worktree landing 7/7 = 100% · structural write refusal 1/1 · nested spawn by non-nesting executor 0 · silent retries 0 · 3 defects all handled · latency captured n=2 (93 s / 149 s) · telemetry captured opportunistically only · no cost owner assigned.
Risks
Closed (native, not prompt): DEF-001 workspace isolation; structural no-nesting (write-capable child removed).
Remaining: R1 unverified live spawn args — irreducible for a prompt-level controller; mitigated by fail-closed wording + registry-owned values, carried as a reviewed risk. R2 thin measurement base. R3 worktree isolates VCS state only. R7 no cost owner.
Strict
STANDARDIZE BOUNDEDcriteria: 9/10 met (…#4 borderline)deterministic isolated workspace ✔ · write child can't nest ✔ (removed) · runtime failure → AES defect ✔ · ≥6 executions ✔ (borderline — 6 reached bounded terminal state) · long multi-hop keeps one
execution_id+ provenance ✔ · root-only router ✔ · Closure Review on original goal ✔ · no silent retry / no inflation ✔ · bounded rollback ✔ · no authority expansion ✔Punch-list for an unconditional STANDARDIZE BOUNDED
schemas/subagent_dispatch_evidence.schema.json+scripts/check_subagent_dispatch_evidence.py, wired todocs-safety.yml) — every recorded dispatch row carriesagent_type+isolation: "worktree"+ workspace observation. (Does not touch the AES schema.)continuation.guardsthresholds.Rollback
git revertthe 2 commits.executorblock is inert once the SKILL subsection reverts. No schema migration, no runtime store, no.gitignorechange.AES statuses
execution_state: completed·overall_delivery: pass·qa_status: pass(6 checks + provenance + pytest 216) ·judge_verdict: not_run·authority_status: owner_review_pending·merge_status: open·production_status: not_applicable🤖 Generated with Claude Code