Feature/capr cohort def from narrative - #83
Merged
Conversation
- The new ACP/MCP flow now has prompt assets, scope gating, concept review modes, deterministic Capr emission, and Capr→Circe validation. - Concept-set policy is preserved: descendant, mapped, and exclusion flags now survive into Circe rather than being silently dropped. - Added explicit era collapse support (era_days) for 365-day collapse cohorts. - Added fail-closed rejection for mixed-domain concept sets; cohort 858 will not be approximated as Condition-only. - R validation now rejects process, filesystem, network, and reflective-execution constructs before running source. An OS-level sandbox remains a follow-up hardening item. Focused suite: 12 passed. Training-reference status: - Compiles and validates with reviewed concept policies: 710, 794, 743, 222, 1340, 1341, 1345, 1346, 1347. - Explicitly not yet supported: 858 (Condition + Observation entry) and 63 (ordered temporal follow-up plus washout). One review flag: cohort 222’s narrative says “earliest,” while its retained reference Circe uses primary-criteria limit All; the flow correctly requires the caller’s confirmed scope to resolve that semantic choice rather than inheriting the reference ambiguity.
- Typed v1 scope contract at the ACP boundary; unsupported or malformed fields return needs_clarification / invalid_scope before emission. - Prompt bundle constrained to supported deterministic modes. - Serialized managed-stdio MCP operations, proposal LLM calls, and per-process R/Circe validation to avoid resource contention under concurrent requests. - Documented the concurrency behavior and Capr feature roadmap.
Completed:
- Public ACP route: POST /flows/phenotype_make_computable
- Typed request, scope, reviewed-concept, and proposal-plan contracts
- confirmed_scope and concept-review modes:
- required
- propose
- provided_only
- Deterministic function-form Capr emitter and R/Circe validator
- Supported v1 patterns:
- Condition entry
- Condition/Visit overlap
- Temporal follow-up with observation/washout
- Fixed or observation-period exit
- Era collapse
- Fail-closed handling for unsupported temporal logic, invalid scope, malformed concepts, and mixed-domain entry.
- Cohort 63’s corrected 365-day observation requirement.
- Cohort 858’s explicit mixed-domain clarification path.
- Prompt bundle, feature-gap documentation, deployment-configured mcp.r runtime, concurrency serialization, and the LLM proposal smoke.
- Retained doit workflow smoke covers all 11 training cohorts and passes using your configured R runtime.
What remains before calling the sprint’s v1 complete:
- Review the live LLM proposal-task result under the newly strict proposal contract; it is intentionally slow and separate from deterministic artifact testing.
- Decide whether to keep the proposal smoke in the default run_smoke_suite or make it explicitly opt-in, given the model latency and verbose logging.
- Document/implement a stronger isolation strategy for arbitrary R source. Current flow emits deterministic source and statically screens validation input, but functional bubblewrap
isolation is unavailable here.
- Keep 858 unresolved until a clinical owner chooses the intended Condition/Observation policy.
- Begin the separate concept-building capability evaluation: broad narratives such as “advanced liver disease” need richer grounded concept-set assistance. That is the right point
to compare a no-new-dependencies baseline against a Groundworkers/OMOP stack branch.
- Add future Capr capabilities deliberately—drug exposure/era, measurement, observation, and richer boolean criteria are documented as out of v1 scope.
- The deferred session to integrate the earlier phenotype_definition Circe contract remains separate from this direct-narrative flow.
The core build-and-validation objective is implemented; the remaining work is hardening, integration, and the next capability tier.
… work on the concept-set creation step
- review_delivery: "auto" returns inline data for ≤10 candidates and a compact, immutable session response for larger reviews. - Large responses contain review_id, counts, expiry, and URLs for paged candidates, CSV download, and proposal retrieval. - CSV is served by ACP; the coding agent downloads it only after user permission. - LLM assessment now covers direct search candidates only; PHOEBE-expanded candidates remain in the full human-review ledger as retrieval context. - Added direct-assessment validation, session TTL support, paged routes, and CSV regression tests. - Focused suite passes: 41 tests. Skill validation passes.
- The final provided_only submission is not session-bound. Once the CSV, manifest, narrative, scope, and reviewed policies are available, an expired review session does not prevent
resubmission. Your successful run already demonstrated the essential stateless path.
- The current CSV-review helper supports one reviewed concept set in one OMOP domain. The ACP emitter itself supports the currently implemented multi-set patterns, but the CSV/
manifest workflow needs a deliberate v2 design for multiple named concept sets and, especially, Condition-plus-Visit overlap. That is the main untested next capability.
- The six “redundant descendant” findings were valuable. They showed the LLM proposal included specific children already covered by an included-descendants ancestor. The human-
reviewed submission remained technically valid, but still carried some redundant inclusions. We should add a review preflight that reports and offers to remove only semantically
redundant positive children before approval.
- Similarly, the explicit cholecystitis/dacryocystitis exclusions likely do not alter this particular concept set: they are unrelated to the included urinary-cystitis descendants.
Explicit exclusions should be reserved for concepts that would otherwise enter through an ancestor or mapped expansion. A future preflight can label such rows “policy has no
effect” rather than treating them as harmful.
- The final phenotype is technically valid and reproducible, but the concept set has meaningful clinical review choices: chronic forms, pregnancy-specific cystitis, and hematuria-
associated concepts. The workflow correctly kept those decisions human-owned.
The natural next increment is a deterministic concept-set preflight: hierarchy simplification, no-effect exclusion detection, and then multi-concept-set CSV review design.
- Multi-set review: explicit criterion_domains now produce named review lanes; a CSV can carry separate Condition and Visit concept sets for cohort 222 and the parser emits both
sets correctly.
- Clinical-domain expansion: validated direct entry now supports Condition, Drug exposure, Procedure, Measurement, Observation, Visit, and Device. Eras, value predicates,
attributes, and arbitrary cross-domain logic remain fail-closed. Specimen is unavailable in this Capr version.
- Condition-or-Observation: cohort 858 now emits earliest qualifying entry across both domains only when multi_domain_entry_policy: "any_qualifying_domain" is explicit.
The demo skill now explains multi-set CSV handling and renders set grouping before approval.
Validation completed successfully, including the retained 11-case structural comparison. Cohort 63’s intentional 365-day continuous-observation variant remains documented rather
than being normalized to the reference.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
This was referenced Aug 29, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.