Skip to content

feat: publish deterministic artifact diff report - #92

Draft
stacknil wants to merge 4 commits into
stacknil/artifact-diff-structured-summaryfrom
stacknil/artifact-diff-json-report
Draft

feat: publish deterministic artifact diff report#92
stacknil wants to merge 4 commits into
stacknil/artifact-diff-structured-summaryfrom
stacknil/artifact-diff-json-report

Conversation

@stacknil

@stacknil stacknil commented Aug 23, 2026

Copy link
Copy Markdown
Owner

Summary

  • add deterministic artifact-contract-diff/v1 serialization for the shared report model
  • add --json-out with output-root isolation and atomic replacement
  • publish a strict Draft 2020-12 schema with status/reason/snapshot cross-field constraints
  • document report semantics, privacy boundaries, and rollback behavior

This is layer 3 of the split replacement for #89 and is stacked on #91.

Design decision

Keep compare_artifact_trees() as the sole semantic core. Human output and JSON are projections of the same immutable report. JSON output contains stable relative identities and bounded summaries only—no roots, bodies, timestamps, or ambient metadata. The destination resolves outside both input trees and is replaced atomically only after a complete synchronized write.

Main risk

A destination-path validation or write failure must never modify an input artifact or destroy a previous report. Root-containment, symlink-root, comparison-failure preservation, simulated os.replace failure, temporary-file cleanup, and deterministic-byte tests cover this boundary. The strict schema may also reject hand-authored payloads that contradict the core invariants; that is intentional.

Compatibility impact

Additive optional CLI flag and new artifact-contract-diff/v1 schema. Existing human CLI behavior, regeneration authority, committed artifacts, evidence schemas, and release labels remain unchanged. Consumers that adopt the JSON report should validate against the versioned schema.

Rollback path

Revert be7c323, f775814, 9b38243, then 77872f6. #91 remains an independently usable human comparator with structured reason classification and no machine-readable output contract.

Validation

  • .venv\Scripts\python.exe -m pytest -q tests/test_artifact_contract_diff_core.py tests/test_artifact_contract_diff_structured.py tests/test_artifact_contract_diff_json_report.py tests/test_artifact_regeneration_check.py — 37 passed, 1 skipped
  • .venv\Scripts\python.exe -m pytest -q tests/test_artifact_contract_diff_json_report.py tests/test_markdown_links.py tests/test_reviewer_docs.py — 31 passed
  • Draft 2020-12 schema self-check — passed
  • .venv\Scripts\python.exe scripts/check_release_contract.py — 29 strict artifacts matched, 6 visual artifacts regenerated, 8 existing schema tests passed, full suite 249 passed / 1 skipped
  • real data/processed JSON report generated twice — identical SHA-256, schema-valid summary (14 total / 8 comparable / 6 presence-only), no local path, smoke report removed
  • git diff --check — passed
  • privacy scan of added lines — no local usernames, OneDrive/absolute machine paths, credentials, or school identifiers

Review state

Draft for the required schema compatibility and delayed review. Relative to #91 this PR changes 6 files and contains 382 substantive behavior/schema lines (125 Python + 257 schema), below the 500-line split gate.

Closes #85.

@stacknil

Copy link
Copy Markdown
Owner Author

Post-CI diff review completed against be7c323 (remote head matches the reviewed local head).

  • Design decision: compare_artifact_trees() remains the only semantic core; JSON is a deterministic projection with a versioned Draft 2020-12 schema and no roots, bodies, timestamps, or ambient metadata.
  • Main risk: output-path or atomic-write failures must not mutate either input tree or destroy a previous report. Containment, symlink-root, comparison-failure, simulated replace-failure, and cleanup tests cover this boundary.
  • Compatibility impact: additive optional --json-out plus artifact-contract-diff/v1. Existing artifacts, evidence schemas, regeneration authority, human output, and release labels remain unchanged.
  • Rollback path: revert be7c323, f775814, 9b38243, then 77872f6; feat: classify structured artifact changes #91 remains independently usable without machine-readable output.

Reviewed relative scope is 6 files and 382 substantive behavior/schema lines (125 Python + 257 schema). All four remote checks are green; no bot or reviewer comments are outstanding.

External clean-clone reproduction also passed at this exact head: fresh venv install, 29 strict artifacts, 6 visual snapshots, 8 existing schema tests, full suite 249 passed / 1 skipped, and a schema-valid real report with SHA-256 5BD27339A4F7A3884C315CC09A9B3B7E10EEF51B54B44A1A30F884D53BEC2B80. The clone and smoke report were removed after verification.

Keeping the PR draft for delayed schema compatibility review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant