Skip to content

feat: add structured artifact-contract diff reporting - #89

Closed
stacknil wants to merge 6 commits into
mainfrom
stacknil/artifact-contract-diff
Closed

feat: add structured artifact-contract diff reporting#89
stacknil wants to merge 6 commits into
mainfrom
stacknil/artifact-contract-diff

Conversation

@stacknil

Copy link
Copy Markdown
Owner

Summary

  • add a deterministic artifact-contract-diff/v1 report for missing, extra, and changed reviewer artifacts
  • expose bounded JSON/JSONL structure, schema-version markers, and validated run-manifest digest changes without copying artifact bodies or checkout roots
  • keep renderer-dependent binary artifacts presence-only and preserve the authoritative regeneration gate

Tracks #85.

Design decision

The package comparator is the authoritative semantic core; the CLI is a thin human/JSON projection. Text-like artifacts share one strict UTF-8 and CRLF/CR-to-LF normalization policy with regenerate_artifacts.py. The comparator hashes normalized bytes first and parses structure only for changed, missing, or extra JSON/JSONL. Invalid digest shapes, unsafe paths, symlinks, special files, resource-limit violations, and report destinations inside either input tree fail closed with exit 2.

Main risk

The main compatibility risk is line-ending equivalence: the existing regeneration gate now treats lone CR the same way the new triage comparator does. A parity regression locks LF, CRLF, and lone-CR behavior. The larger implementation surface also warrants delayed review and either reduction or an independently verifiable stack before this PR becomes ready.

Compatibility impact

Additive reviewer tooling only: no committed artifact schema, demo behavior, SIEM integration, or release output changes. artifact-contract-diff/v1 is a new strict report contract. Existing binary comparison semantics remain presence-only. The regeneration gate gains lone-CR normalization parity; all currently committed strict artifacts still reproduce byte-contract-equivalently.

Rollback path

Revert this branch's six semantic commits. There is no data migration, generated-artifact update, release tag, or persisted state to unwind; the prior python scripts/regenerate_artifacts.py --check workflow remains the fallback.

Validation

  • python -m pytest -q tests/test_artifact_contract_diff.py tests/test_artifact_regeneration_check.py — 26 passed, 1 platform-conditional skip
  • python scripts/regenerate_artifacts.py --check — 29 strict artifacts matched; 6 visual snapshots regenerated presence-only
  • real data/processed/richer_sample self-compare — 4 comparable unchanged; 3 binaries presence-only; JSON validated against artifact-contract-diff/v1; privacy/timestamp scan passed
  • python scripts/check_release_contract.py — 8 schema tests passed; full suite 238 passed, 1 platform-conditional skip
  • git diff --check origin/main...HEAD — passed
  • changed-file privacy/secret scan — passed

Review state

Draft: 9 files and 1,831 added lines exceed the lightweight-review threshold. Do not merge until the size boundary is reduced or split, CI is green, bot feedback is resolved, and the required delayed compatibility review is complete.

@stacknil

Copy link
Copy Markdown
Owner Author

Post-CI final diff review completed at head 54ee36b.

  • Design decision: keep regenerate_artifacts.py --check authoritative; the package comparator owns normalized identity and report invariants, while human/JSON CLI surfaces are projections only.
  • Main risk: this is still a 9-file / 1,831-added-line draft. It must be reduced or split and receive delayed compatibility review plus external clean-clone reproduction before becoming ready.
  • Compatibility impact: additive candidate CLI/schema; no committed artifact shape, demo behavior, runtime dependency, or release change. The existing gate now shares lone-CR normalization with triage, locked by parity tests.
  • Rollback path: revert the six semantic commits; no data migration, generated artifact update, tag, release, or persisted state is involved.

Final evidence: all four remote checks passed; origin/main remains at the reviewed merge base 467262f; git diff --check and changed-file privacy/secret scans passed; no bot comments or reviews are outstanding. The PR remains draft and is not approved for merge.

@stacknil

Copy link
Copy Markdown
Owner Author

Superseded by the policy-compliant stacked replacement:

  1. feat: add bounded artifact diff core #90 — bounded comparison core + human CLI (432 substantive behavior lines; 3 semantic commits)
  2. feat: classify structured artifact changes #91 — bounded JSON/JSONL structure/schema/provenance classification (234 substantive behavior lines; 3 semantic commits)
  3. feat: publish deterministic artifact diff report #92 — atomic deterministic JSON projection + strict artifact-contract-diff/v1 schema (382 substantive behavior/schema lines; 4 semantic commits; closes Add structured artifact-contract diff reporting #85 when merged)

Each replacement is independently scoped below the 500-line substantive gate, has dedicated compatibility/error-path tests, a documented design/risk/compatibility/rollback boundary, and four green remote checks. The full stacked head also passed an external clean-clone reproduction: 29 strict artifacts, 6 visual snapshots, 8 existing schema tests, and 249 passed / 1 skipped.

Closing this oversized draft without merge. The branch and this discussion remain available as implementation provenance; no branch is deleted.

@stacknil stacknil closed this Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant