Skip to content

Add machine-readable JSON report (00BiocCheck.json) - #258

Open
seandavi wants to merge 2 commits into
develfrom
json-output
Open

Add machine-readable JSON report (00BiocCheck.json)#258
seandavi wants to merge 2 commits into
develfrom
json-output

Conversation

@seandavi

Copy link
Copy Markdown

Motivation

BiocCheck results are currently only available as text. Anything that wants
to act on them — a CI gate, an editor, a bot that triages submissions — has to
parse 00BiocCheck.log, which is fragile and breaks whenever a message is
reworded.

Nearly everything needed is already collected internally: handleCondition()
records the originating check function via sys.call, handleCheck() records
the check title, and add() knows the severity. They are just flattened into
text at render time. This PR exposes them.

What this adds

Every run now writes 00BiocCheck.json next to 00BiocCheck.log in the
<PackageName>.BiocCheck folder:

{
  "metadata": { "Package": "MyPackage", "PackageVersion": "0.99.0", "...": "" },
  "summary": { "error": 1, "warning": 2, "note": 5 },
  "status": "error",
  "entries": [
    {
      "severity": "warning",
      "checkFun": "checkFormatting",
      "check": "Checking formatting of DESCRIPTION, NAMESPACE, ...",
      "message": "Consider shorter lines; 3 lines (1%) are > 80 characters long.",
      "help_text": "First few lines:",
      "details": ["R/foo.R#L12 x <- some_very_long_call( ..."],
      "locations": [ { "file": "R/foo.R", "line": 12 } ]
    }
  ],
  "text": ["* Checking formatting of DESCRIPTION, NAMESPACE, ...", "..."]
}
  • summary / status give the tallies and the most severe condition
    (error, warning, note, or ok), so a gate is one line:
    jq -e '.summary.error == 0' MyPackage.BiocCheck/00BiocCheck.json
  • entries has one record per condition raised. checkFun is the stable
    identifier to key on; check is the human-readable title, which is free to
    change between releases.
  • locations are parsed inside BiocCheck from the two location formats the
    checks already emit (.lineReport and
    sprintf("%s (line %d, column %d)"), with or without a leading symbol
    name), so consumers never parse them. Chunk-relative vignette locations are
    deliberately omitted rather than reported as file lines.
  • text is the log file, line for line — the JSON is a superset of the text
    report, nothing to cross-reference.

The same data is on the returned object: bc$getStatus(), bc$entries, and
bc$toJSON() (returns the JSON string when no file is given).

Implementation notes

  • New entries field on the BiocCheck class, appended in add() and
    filtered in zero(). The existing log / composeReport / report text
    path is untouched, so there is no change to 00BiocCheck.log.
  • toJSON / fromJSON were doubly encoding the payload (toJSON() to a
    string, then write_json() of that string) and are rewritten. They were not
    called anywhere in the package, so no on-disk format was in use.
  • Field types are stable: details is always an array, help_text and
    locations are null when absent, metadata is always an object.
  • jsonlite moves from Suggests to Imports. The alternative — an opt-in
    flag — would need to be set correctly in every CI job to be useful, and it
    is a small, dependency-free package. Happy to switch it to a dot-option if
    you would rather keep it optional.

Two judgment calls worth your review

  1. The JSON is always written, with no opt-out (see above).
  2. Four coding-practice checks reported bare file names (bad_coding.R) while
    every other check reports package-relative paths (R/bad_coding.R). They
    now use .getDirFiles() so locations.file resolves consistently. This
    changes existing text output, so it is easy to drop if you consider it
    out of scope.

Checklist

  • Update the NEWS file
  • Update the vignette file (new "Machine-readable output" section; also a
    short README section)
  • Add unit tests — inst/tinytest/test_json.R, 36 assertions covering the
    schema, summary/status, both location formats, the round trip, and the
    files written by report()
  • R CMD build & R CMD check — full local suite passes (221
    assertions); an end-to-end run on testpkg0 yields 49 entries matching
    13 errors / 17 warnings / 19 notes with 113 parsed locations

@LiNk-NY @lshep for review.

🤖 Generated with Claude Code

https://claude.ai/code/session_0153nSC2TGmUwxcXt8iCvweV

Write '00BiocCheck.json' alongside '00BiocCheck.log' in the
'<PackageName>.BiocCheck' folder. The report carries the full text output
plus structured fields so that CI jobs, editors, and other tools can act on
BiocCheck results without parsing text:

  * 'summary' with the error, warning, and note counts and an overall
    'status' ('error', 'warning', 'note', or 'ok')
  * 'entries', one record per condition raised, with the originating check
    function ('checkFun') as the stable identifier, the check title, the
    message, the reported files, and parsed file 'locations'
  * 'text', the plain report line by line

The check function name and the check title were already recorded; they are
now exposed rather than being flattened into the text log. Also fix the
doubly encoded 'toJSON'/'fromJSON' methods, have 'toJSON' return the JSON
when no file is given, and report package-relative file paths in the coding
practice checks so that locations resolve consistently.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0153nSC2TGmUwxcXt8iCvweV
@codecov

codecov Bot commented Aug 31, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.10145% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.06%. Comparing base (20f99c6) to head (4edf283).
⚠️ Report is 2 commits behind head on devel.

Files with missing lines Patch % Lines
R/BiocCheck-class.R 96.92% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##            devel     #258      +/-   ##
==========================================
+ Coverage   69.17%   70.06%   +0.89%     
==========================================
  Files          15       15              
  Lines        3046     3117      +71     
==========================================
+ Hits         2107     2184      +77     
+ Misses        939      933       -6     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

The added roxygen prose and vignette example exceeded 80 characters once
expanded into Rd, which BiocCheck flags on its own source.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0153nSC2TGmUwxcXt8iCvweV
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant