refactor: add a JSON-LD deserializer symmetric with the serializer - #110
Open
ehennestad wants to merge 2 commits into
Open
refactor: add a JSON-LD deserializer symmetric with the serializer#110ehennestad wants to merge 2 commits into
ehennestad wants to merge 2 commits into
Conversation
Serialization had an architecture and deserialization had a function. loadInstances read files, stripped a hardcoded vocabulary prefix, dispatched types, built instances and wired links between them, with a hand-rolled traversal that had no cycle detection and no extension point for another format. BaseDeserializer is the counterpart to BaseSerializer. Subclasses implement parseToStructs for one format; the base class handles type dispatch, instance construction and link wiring. JsonLdDeserializer implements the JSON-LD case and accepts several documents at once. Link wiring moves to LinkWiringVisitor, built on the traversal core introduced earlier in this stack, so it inherits cycle detection and positional replacement rather than reimplementing traversal. A node that cannot be turned into an instance was reported with warning(ME.message), one warning per node, which lost the error identifier and told the caller nothing about which node was lost or how much of the document was missing. Such nodes are now collected and reported once, by identifier, under an identified warning. Callers that cannot use a partial result can ask for an error instead. Resolving the type stays outside that handling. A type in an unknown namespace means the document was written for a different model version, which is a document-level problem, and reporting it per node would leave the caller with an empty result and a warning rather than a failure. loadInstances is now file reading and deserializer selection, and goes from 144 lines to 48. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Test Results (R2022a)756 tests +6 754 ✅ +6 2m 30s ⏱️ -10s For more details on these failures, see this check. Results for commit d7b0925. ± Comparison against base commit a5ac5dd. ♻️ This comment has been updated with latest results. |
ehennestad
force-pushed
the
add-jsonld-deserializer
branch
from
August 28, 2026 01:00
fa853e0 to
d7b0925
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## fix-controlled-term-array-construction #110 +/- ##
==========================================================================
+ Coverage 79.20% 79.28% +0.08%
==========================================================================
Files 419 422 +3
Lines 4077 4089 +12
==========================================================================
+ Hits 3229 3242 +13
+ Misses 848 847 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
ehennestad
force-pushed
the
add-jsonld-deserializer
branch
2 times, most recently
from
August 28, 2026 12:59
34f1334 to
d7b0925
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Serialization had an architecture; deserialization had a function. This adds the missing half.
What loadInstances was doing
A single 144-line function read the files, stripped a hardcoded vocabulary prefix, dispatched each node to a type, built the instances, and then wired the references between them with a hand-rolled recursive traversal. That traversal had no cycle detection, no extension point for another format, and no way to configure any of it.
BaseDeserializer
The counterpart to
BaseSerializer. Subclasses implement one method:The base class owns what does not depend on the format: dispatching each node to its type, constructing the instances, and wiring the links.
JsonLdDeserializerimplements the JSON-LD case and accepts several documents in one call, which is what loading a folder of one-file-per-instance needs.Link wiring moves onto the traversal core
LinkWiringVisitorreplaces the recursiveresolveLinkslocal function. Building it onBaseVisitormeans it inherits cycle detection and positional replacement rather than reimplementing traversal for the third time in the codebase.Reporting instead of dropping
A node that could not be turned into an instance was previously handled like this:
That loses the error identifier, makes the warning unsuppressable by identifier, garbles the message if it contains a
%, and produces one warning per node with no indication of which node was lost. A node with no@typewas skipped even more quietly, under acontinue % Todo: Why skip?.Unreadable nodes are now collected and reported once, by identifier, under
openMINDS:Deserializer:UnreadableNodes, so it is possible to tell how much of a document did not survive. A caller that cannot work with a partial result can construct the deserializer withUnreadableNodePolicy = "error".Resolving the type is deliberately left outside that handling. A type in an unknown namespace means the whole document was written for a different model version. Treating that as a per-node problem would hand the caller an empty collection and a warning where a hard failure is more useful, and the fixture test for legacy namespace documents asserts exactly that.
Result
loadInstancesis now file reading and deserializer selection, 48 lines instead of 144.Tests
New
DeserializerTestcovers reading a collection document, wiring links between nodes, wiring links across separate documents, terminating on a circular document, reporting an untyped node while still returning the readable ones, and the error policy.Not addressed here
jsonld2structstill strips a hardcodedopenminds.ebrains.euvocabulary prefix, which is wrong for documents written in expanded form under a v4 model. That belongs with the version-tolerant loading work later in this stack.🤖 Generated with Claude Code