From 7e7bc93fa2af104cc8ff0d1be730aa261ee7b235 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sat, 22 Aug 2026 14:23:29 -0400 Subject: [PATCH 01/25] docs(spec): whole-object rollup on @agg:collect, and the surface contract it lands in (#335) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Design for making @of optional on origin.aggregate @agg:collect. Ships in two halves: the rollup itself, and two holes in the queryable-projection contract that the rollup would otherwise land a column into — the filter allowlist never consults isArray, and @sortable has no subtype validation at all. Both are live today. Records the decision NOT to mint a subtype or @kind for "document" vs "relational" projections: ADR-0037's subtype test fails, @kind is the physical-storage axis with read-only-ness derived from it, and the query contract is already per-field, which is the right level. Four claims in the issue did not survive verification against the code, including that the @objectRef-must-be-a-value rule already exists — it is payload-scoped and never reaches a projection-hosted field, so the branch's main #270 guard would have been absent. Co-Authored-By: Claude Opus 5 (1M context) --- ...22-issue-335-whole-object-rollup-design.md | 322 ++++++++++++++++++ 1 file changed, 322 insertions(+) create mode 100644 docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md diff --git a/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md b/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md new file mode 100644 index 000000000..ccb0bdaf4 --- /dev/null +++ b/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md @@ -0,0 +1,322 @@ +# Whole-object rollup on `origin.aggregate @agg: collect` — design + +**Issue:** [#335](https://github.com/metaobjectsdev/metaobjects/issues/335) +**Date:** 2026-08-22 +**Status:** proposed +**Baseline:** `9eede9dd9` + +## Summary + +Make `@of` **optional** on `origin.aggregate @agg: collect`. Absent means a **whole-object +rollup**: collect the related rows as a JSON array of the field's declared `@objectRef` +value-object, rather than an array of one scalar column. + +```jsonc +{ "field.object": { + "name": "supplierBriefs", + "isArray": true, + "@objectRef": "acme::common::SupplierBrief", + "children": [ + { "origin.aggregate": { "@agg": "collect", "@via": "acme::catalog::Product.suppliers" } } + ] +}} +``` + +The work ships in **two halves**. Half A is the feature. Half B closes two holes in the +queryable-projection contract that Half A would otherwise land a column into. + +## Why two halves + +A projection is a **queryable relational surface**, not merely a wire read-shape. That is the +project's intent, and it is already true in code: any object with a readable source gets +`renderFilterAllowlist` / `renderSortAllowlist` and filtered read routes +(`templates/entity-file.ts:171`). + +A whole-object rollup column is a deliberate **leaf** inside that surface — non-filterable by +construction, like a `jsonb` column in any hand-written view. The loader already enforces the +leaf-ness (`@filterable` on `field.object` is `ERR_FILTERABLE_UNSUPPORTED_SUBTYPE`, +`validation-passes.ts:436-455`). + +But the surface contract has two holes today, **independent of this feature**: + +1. **The filter allowlist is not `isArray`-aware.** `filterSubTypeFor` + (`templates/filter-allowlist.ts:38-44`) falls through to `"string"` for any unrecognised + subtype, and nothing in that file consults `isArray` at all. So + `field.string isArray: true @filterable: true` already emits a `like`/`eq` rule against a + `text[]` column. This is live: `weekLabels` is a shipping array projection column. +2. **There is no `@sortable` subtype validation anywhere in the loader** — versus a hard error + for `@filterable`. `@sortable: true` on a JSON or array column passes the loader and emits a + sort entry. + +Shipping Half A without Half B adds a second non-scalar column shape to a query tier that +already mishandles the first. + +## Decision: no new subtype and no new `@kind` + +Considered and rejected: distinguishing a "document projection" from a "relational projection" +in the type system. + +- **ADR-0037's subtype test fails.** A subtype needs its own native type, behaviour, or + attributes. Both forms are read-only views, with the same source `@kind`s and the same + generated artifacts. The only difference is whether one column is non-scalar. +- **`@kind` is already chartered for something else.** On `source.rdb` it names *which physical + relational object backs this* — `table | view | materializedView | storedProc | + tableFunction` — and read-only-ness is **derived** from it (`spec/metamodel/source.json`). A + `documentView` member would be a category error and would break that derivation. +- **The query contract is already per-FIELD**, which is the correct level, because that is + where the truth lives. A projection carrying one nested column stays a queryable surface on + every other column. + +**Consequence recorded deliberately:** #210 made `object.projection` the only legal host for +assembly origins, so "assemble a document" has no other home in the metamodel. If a shipping +consumer ever needs a read shape that is *not* a queryable surface, that earns its own type +then — the ADR-0007 Amendment 2 bar, the same discipline applied to `origin.collection`. Not +now, and not on speculation. + +--- + +# Half A — the whole-object rollup + +## A1. Loader (all five ports) + +Split `collect` out of the `@of`-required gate at `validation-passes.ts:1252`. + +**`@of` PRESENT** — unchanged, including the element-type-preserving check at `:1265-1268`. + +**`@of` ABSENT** — the carrying field must be: + +| Requirement | Error | Already enforced? | +|---|---|---| +| `field.object` with `@objectRef` | `ERR_INVALID_ORIGIN` | new | +| `isArray: true` | `ERR_INVALID_ORIGIN` | yes — `:1192-1197` | +| `@objectRef` targets an `object.value` | `ERR_SUBTYPE_RULE_VIOLATION` | **new — see below** | +| `@via` present | `ERR_INVALID_ORIGIN` | new (no `@of` to infer from) | +| at least one to-many `@via` hop | `ERR_ORIGIN_CARDINALITY` | yes — `_checkAggregateCardinality` | +| `@distinct` absent | `ERR_INVALID_ORIGIN` | new — see A5 | + +**The value-only rule is NOT inherited from #210.** `_checkNestedPayloadRefsValueOnly` +(`validation-passes.ts:213`) walks only from **template-level payload targets** — its own +comment scopes it to "every `field.object @objectRef` reachable from a template-level payload +target". A projection-hosted `field.object` never reaches that pass, so nothing today stops a +projection field's `@objectRef` from resolving to an `object.entity`. This branch must enforce +it itself, reusing `ERR_SUBTYPE_RULE_VIOLATION` for consistency with the payload-side message. +Without it, an `@objectRef` to an entity reintroduces the #270 shape — a curated value silently +becoming the full entity — in DDL. + +**Element type is declared-authoritative (#270).** The element type comes from the declared +`@objectRef`, never from the `@via` relationship's target entity. This is the exact bug #270 +deleted; the rule is restated here because the new branch is where it could be reintroduced. + +**`@orderBy` re-points.** `_validateOrderByKeys` currently resolves keys against +`ofTarget?.entity` (`:1271`). With `@of` absent there is no `ofTarget`, so keys must resolve +against the `@via` **terminal** entity. + +**Multi-hop `@via` is legal.** `viaTerminalEntity` already walks every segment +(`extract-view-spec.ts:477-493`), and the `any`/`all` branch already permits multi-hop. No new +restriction — consistency with the sibling branch is the reason. + +## A2. Member resolution — the #270 guard + +The lowering must project **exactly the declared value-object's members**, matched **by name** +against the `@via` terminal entity's effective fields. + +- A VO member with **no matching field** on the terminal entity → **error**, new code + `ERR_COLLECT_MEMBER_UNRESOLVED`. A new code must be added to the shared ledger and every + registry that gates it — TS `errors.ts` (exact-bidirectional), Python `errors.py` (superset), + Java `ErrorCode.java` — in the same change, or the ledger tests fail. +- A VO member whose `field.` **differs** from the matched field's → **error**, reusing + `ERR_INVALID_ORIGIN`, which is what the scalar element-type check at `:1265-1268` already + emits. This is the object-form analogue of that check, carrying the same #185 + type-preserving doctrine per member. + +Failing open here — silently dropping or substituting members — is precisely how #270 turned a +declared curated value-object into the full entity. The error is the point. + +## A3. Codegen (TypeScript only — ADR-0015) + +`extract-view-spec.ts`: today `if (!of_) continue;` (`:891`) sits **above** the `AGG_COLLECT` +branch (`:902`), and the aggregated entity is resolved *from* `@of`. Restructure so the +object arm resolves from `@via` instead. + +**Mirror the `any`/`all` branch** (`:861-885`) — it is the same shape and already proven: +`viaTerminalEntity(via, …)` → `resolveEntityRef` → `findAliasInTree` → `primaryKeyColumn`. + +New column kind `collectObjectAgg` in `view-spec.ts`, carrying the resolved member list +(`{ memberName, sourceColumn }[]`) plus `sourceAlias`, `joinedPkColumn`, `orderBy`. + +Kept as a **separate kind** rather than an arm of `collectAgg`: the payloads differ (a member +list vs a single source column) and the existing kind is consumed by +`viewOrderKeysAreDeterministic` / the real-aggregate predicate at `:1005` and `:1021`, which a +union type would force every consumer to re-narrow. + +## A4. DDL lowering (`view-ddl-emit.ts`) + +Preserves the existing empty-set guard and ordering discipline. + +**Postgres — `jsonb`, not `json`.** Verified against PostgreSQL 15.15: `json` has neither an +equality nor an ordering operator (`ERROR: operator does not exist: json = json`; +`could not identify an ordering operator for type json`), so the `json_agg(json_build_object(…) +ORDER BY …)` form named in the issue **does not run**. `jsonb` has both, and `field.object` is +already `jsonb` in `column-mapper.ts:610-613`. + +```sql +COALESCE( + jsonb_agg(jsonb_build_object('id', s."id", 'name', s."name") ORDER BY s."id" ASC) + FILTER (WHERE s."id" IS NOT NULL), + '[]'::jsonb +) AS "supplierBriefs" +``` + +**SQLite** — verified on 3.53.0: nested objects propagate (the JSON subtype carries), and +in-aggregate `ORDER BY` and the `FILTER` clause both work. + +```sql +COALESCE( + json_group_array(json_object('id', s."id", 'name', s."name") ORDER BY s."id" ASC) + FILTER (WHERE s."id" IS NOT NULL), + json_array() +) AS "supplierBriefs" +``` + +In-aggregate `ORDER BY` requires SQLite **≥ 3.44**. This is **not a new constraint** — the +existing scalar `collect` already emits it, and D1's baseline is pinned at `3.44.0` +(`introspect/d1.ts:44`). + +**Empty default differs by arm**: the scalar arm keeps `'{}'` (a PG array); the object arm uses +`'[]'::jsonb`. The scalar arm's output must stay byte-identical. + +## A5. Element order and dedupe + +**Default order: the related entity's primary key, ascending.** "Value-ascending" does not +transfer from scalars — ordering rows by a serialized object is meaningless, and on PG `json` +it does not even parse. PK-ascending is deterministic and already the in-emitter precedent: +`renderFirst` appends `childAlias.childPk ASC` as its tie-break (`view-ddl-emit.ts:131-137`). + +**`@orderBy` is supported**, resolving against the `@via` terminal entity, with the PK appended +as tie-break so equal-order rows stay byte-deterministic. + +> **This diverges from the scalar arm, deliberately.** Today a scalar `collect` with `@orderBy` +> emits *only* those keys (`view-ddl-emit.ts:193-196`) — no tie-break — so two rows equal under +> the keys have engine-chosen order. That is tolerable for scalars and not for objects, where +> the value itself cannot break the tie (PG `json` has no ordering operator at all). Appending +> the PK matches `renderFirst`, which already does exactly this. The scalar arm is **not** +> changed: doing so would alter existing emitted SQL for every project using `@orderBy`. + +**`@distinct` is FORBIDDEN — by choice, not by engine limit.** State this honestly: it was +verified to work on both engines (SQLite 3.53 `json_group_array(DISTINCT json_object(…))` +dedupes correctly; PG `jsonb_agg(DISTINCT …)` works). It is forbidden because it is a +guaranteed no-op whenever the value-object carries the entity's primary key, which is the +common case, and a silent no-op is worse than a refusal. Re-entry path if an adopter presents a +real case: allow `@distinct` **with a mandatory `@orderBy`**, inverting today's mutual +exclusion. + +## A6. Registry prose — byte-gated, seven files + +`@of` is already `"required": false` structurally, so no `required` flip. But its `description` +reads *"Required for count/sum/avg/min/max/collect; …"* and that string is byte-gated in seven +places across all five ports: + +``` +fixtures/registry-conformance/expected-registry.json +fixtures/metamodel-docs/expected/types/origin.md +spec/metamodel/origin.json +server/csharp/MetaObjects/Persistence/Origin/OriginSchema.cs +server/csharp/MetaObjects/SpecMetamodel/origin.json +server/python/src/metaobjects/spec_metamodel/origin.json +server/typescript/packages/metadata/src/persistence/origin/origin-definition.embedded.ts +``` + +All seven change in lockstep or `registry-conformance` goes red in every port. + +## A7. `metamodelVersion`: `0.10` → `0.11` (additive) + +This is #210 in reverse — a rule change whose only machine-readable footprint is prose, which +`check-metamodel-version.mjs` can only **WARN** about (`:36`). No gate forces the call, so it is +made here: **additive**, because previously-invalid metadata becomes valid and nothing that +loaded before stops loading. Set with `node scripts/check-metamodel-version.mjs --set 0.11` +(writes all five sites). + +--- + +# Half B — the queryable-surface contract + +## B1. `isArray`-aware filter band + +`filterSubTypeFor` (`templates/filter-allowlist.ts:38-44`) maps subtype → category and falls +through to `"string"`; nothing consults `isArray`. + +**Rule:** an array-typed field is **not filterable** under the FR-009 scalar operator band. A +`field. isArray: true` carrying `@filterable: true` becomes a **load error**, reusing +`ERR_FILTERABLE_UNSUPPORTED_SUBTYPE` — the same door `field.object` already goes through, since +the reason is identical (no operator in the band applies). + +Fail at load, not by filtering the field out of the allowlist: silently dropping a declared +`@filterable` is the same failure mode as the pre-#292 unchecked snapshot — the author believes +something is enforced that is not. + +## B2. `@sortable` subtype validation + +`@sortable` defaults from `@filterable`, so today it is only independently set when explicit — +and nothing validates it. Add the subtype/array check that `@filterable` has, so +`@sortable: true` on a JSON or array column is a load error rather than a generated sort over a +JSON column. + +**Note:** B1 and B2 are potentially breaking for a project that today declares +`@filterable`/`@sortable` on an array field. That declaration currently generates a rule that +cannot work, so this converts a runtime failure into a load failure. Call it out in the +changelog. + +--- + +## Testing + +**The corpus must gain the case, not just the code.** `fixtures/conformance/README.md:102` +already records `flattened-kitchen-sink`'s dropped `supplierBriefs` as *"coverage genuinely +lost"* and names #335 as the restore path. Restore it there, expressed the new way. + +| Layer | What | +|---|---| +| Conformance (5 ports) | `@of`-absent collect loads on a `field.object @objectRef isArray`; each error arm fails: no `@objectRef`, non-value target, missing `@via`, all-to-one `@via`, unresolved member, member type mismatch | +| No-churn | a `collect` **with** `@of` emits byte-identical SQL — pinned, since the scalar arm shares the branch | +| Real engine, both dialects | emit → apply → introspect → **re-diff must be empty**, then read rows back and assert the array-of-objects shape, including the empty-set case returning `[]` not `null` | +| Half B | a `@filterable` array field fails to load; a `@sortable` array field fails to load | + +**The round-trip tests live in a separate package.** `view-lifecycle-{pg,sqlite}.test.ts` are in +`server/typescript/packages/integration-tests`, which only the `ts-slow` lane runs — they will +not appear in a `codegen-ts` or `migrate-ts` run. Golden SQL alone is not acceptable evidence +for new DDL; that is how the migrate defect class got through before. + +**SQLite byte-stability binds.** `view-fingerprint.ts` explains that Postgres never compares +against the deparser (it hashes the body we emit), **but that "SQLite/D1 need none of this: +`sqlite_master.sql` is the verbatim text we wrote, so the body comparator is exact there."** So +the emitted SQLite expression must be deterministic or every migrate reports drift. + +## Non-goals + +- Non-RDB lowerings — #211's capability matrix. +- Filtering a parent by an embedded child's property. This is the capability a queryable + surface would eventually want, `json_agg` cannot serve it, and it is a separate design. +- Exposing `ObjectManager.attachIncludes` as a generated endpoint. It is wired into the + ObjectManager read path (`object-manager.ts:128,140`) but **no generated endpoint surfaces + it** and it is TS-only. If that is ever built, it is the app-side-merge counterpart to this + view-side join — the two are alternatives, and this design does not foreclose it. + +## Evidence base + +Claims here were verified at `9eede9dd9` rather than carried from the issue text. Four of the +issue's own statements did not survive: + +1. **"No registry change"** — structurally true, but the byte-gated `description` prose must + change in seven files (A6). +2. **The proposed `json_agg(json_build_object(…) ORDER BY …)` does not run on Postgres** — + verified against a real engine (A4). +3. **"One relaxed rule"** understates it — member resolution, `@orderBy` re-pointing, a new + column kind, and a changed empty default all follow. +4. **"`@objectRef` must target an `object.value`, which is already a loader rule from #210"** — + it is not. That rule is payload-scoped and never reaches a projection-hosted field (A1). + This is the most consequential of the four: relying on it would have shipped the branch with + its main #270 guard simply absent. + +Two claims from review that also did not survive checking: this is **not** the first non-scalar +projection column (`weekLabels` ships today, asserted against real PG and SQLite), and +`@distinct` is **not** blocked by either engine. From 1194032a706e67ad8e77735292029b8d2ec03899 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sat, 22 Aug 2026 14:57:44 -0400 Subject: [PATCH 02/25] docs(spec): metamodelVersion rides the existing 0.11 rather than moving it (#335) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A parallel change (#342) already moved metamodelVersion 0.10 -> 0.11, and that bump is unreleased — v0.24.0 shipped 0.10. The gate baselines against the last release tag so a cycle needs one bump, not one per PR, so this change needs no edit at all. Records the two conditions that would change it: 0.11 shipping before this lands, or Half B being judged breaking. Co-Authored-By: Claude Opus 5 (1M context) --- ...22-issue-335-whole-object-rollup-design.md | 23 +++++++++++++------ 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md b/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md index ccb0bdaf4..8a51ffd05 100644 --- a/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md +++ b/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md @@ -228,13 +228,22 @@ server/typescript/packages/metadata/src/persistence/origin/origin-definition.emb All seven change in lockstep or `registry-conformance` goes red in every port. -## A7. `metamodelVersion`: `0.10` → `0.11` (additive) - -This is #210 in reverse — a rule change whose only machine-readable footprint is prose, which -`check-metamodel-version.mjs` can only **WARN** about (`:36`). No gate forces the call, so it is -made here: **additive**, because previously-invalid metadata becomes valid and nothing that -loaded before stops loading. Set with `node scripts/check-metamodel-version.mjs --set 0.11` -(writes all five sites). +## A7. `metamodelVersion`: rides the existing `0.11` — no further bump + +Classification: **additive**. Previously-invalid metadata becomes valid; nothing that loaded +before stops loading. This is #210 in reverse — a rule change whose only machine-readable +footprint is prose, which `check-metamodel-version.mjs` can only **WARN** about (`:36`), so no +gate forces the call and it is made here. + +**But no edit is required.** `#342` (`e96bbe329`) already moved `metamodelVersion` `0.10` → +`0.11`, and that bump is **unreleased** — `v0.24.0` shipped `0.10`. The gate baselines against +the last **release tag**, not `HEAD~1`, precisely so that a release cycle needs one bump rather +than one per PR. So `0.11` already covers this change. + +Two conditions would change that, and the plan must re-check both at implementation time: +- If `0.11` ships in a release **before** this lands, this needs `0.12`. +- If Half B is judged breaking rather than additive (see B2's note), the classification — + though not necessarily the number, pre-1.0 — changes with it. --- From 1bb6d1872a7589add5624eca17c05e061507fbab Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sat, 22 Aug 2026 22:02:23 -0400 Subject: [PATCH 03/25] docs(spec): every new load error needs a corpus fixture that triggers it (#335) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A rule with no negative fixture is a blind spot one layer up — "no fixture covers it" and "every port enforces it" look identical on a green suite. Three findings in the parallel #342 work were exactly this shape, including one where the ungated arm was the precise place a port had diverged. Makes Half B's negative cases shared conformance fixtures rather than port-local unit tests, and records the measurement that motivates it: a structural scan of 1321 JSON + 124 YAML files finds zero fields carrying both isArray:true and filterable/sortable:true, so nothing in the corpus exercises an array field through the filter tier at all. Co-Authored-By: Claude Opus 5 (1M context) --- ...8-22-issue-335-whole-object-rollup-design.md | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md b/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md index 8a51ffd05..59eb81b36 100644 --- a/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md +++ b/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md @@ -288,7 +288,22 @@ lost"* and names #335 as the restore path. Restore it there, expressed the new w | Conformance (5 ports) | `@of`-absent collect loads on a `field.object @objectRef isArray`; each error arm fails: no `@objectRef`, non-value target, missing `@via`, all-to-one `@via`, unresolved member, member type mismatch | | No-churn | a `collect` **with** `@of` emits byte-identical SQL — pinned, since the scalar arm shares the branch | | Real engine, both dialects | emit → apply → introspect → **re-diff must be empty**, then read rows back and assert the array-of-objects shape, including the empty-set case returning `[]` not `null` | -| Half B | a `@filterable` array field fails to load; a `@sortable` array field fails to load | +| Half B | a `@filterable` array field fails to load; a `@sortable` array field fails to load — **as shared `fixtures/conformance/` cases, not port-local unit tests** | + +**Every new load error needs a corpus fixture that TRIGGERS it, in the shared corpus.** A rule +with no negative fixture is the same blind spot one layer up: "no fixture covers it" and "every +port enforces it" are indistinguishable from a green suite. Three separate findings in #342 were +this shape — a BOTH-fixture that covered `index.lookup` but not `identity.secondary`, which is +the exact arm where the JVM had diverged; and `@expr` built and shipped end-to-end with no +fixture ever declaring one, so one port was free to refuse it for months. + +A port-local unit test does not substitute. It proves *one* port enforces the rule, which is +precisely the assertion that was true-and-insufficient in every one of those cases. This +matters more than usual here because Half B's in-repo blast radius is **zero** — a structural +scan of 1321 JSON and 124 YAML files found no field carrying both `isArray: true` and +`@filterable`/`@sortable: true`. Nothing in the corpus exercises an array field through the +filter tier at all, which is why the tier has been free to be wrong about it since it shipped. +The fixture is the only thing that changes that. **The round-trip tests live in a separate package.** `view-lifecycle-{pg,sqlite}.test.ts` are in `server/typescript/packages/integration-tests`, which only the `ts-slow` lane runs — they will From 318b394d6619750ecf71918fe4132b119e89aba1 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sat, 22 Aug 2026 22:03:43 -0400 Subject: [PATCH 04/25] docs(spec): bound Half B's breaking-ness instead of just labelling it (#335) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit "Potentially breaking" was too flat. Three bounds now stated: in-repo cost measured at zero, what breaks was already emitting SQL that cannot execute, and — the distinction that actually matters — nothing ever taught adopters to write it. That last point separates this from a sibling change in the same window that is the same category and a different magnitude: a retired spelling that appeared as a worked example in a shipped skill has a plausible acquisition route; a silently wrong emission nobody was told to write does not. Also records that this version question gets asked when #335 is cut, on its own evidence, rather than folded into a ruling already made about a different change. Co-Authored-By: Claude Opus 5 (1M context) --- ...22-issue-335-whole-object-rollup-design.md | 31 ++++++++++++++++--- 1 file changed, 27 insertions(+), 4 deletions(-) diff --git a/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md b/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md index 59eb81b36..4fdf92f92 100644 --- a/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md +++ b/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md @@ -270,10 +270,33 @@ and nothing validates it. Add the subtype/array check that `@filterable` has, so `@sortable: true` on a JSON or array column is a load error rather than a generated sort over a JSON column. -**Note:** B1 and B2 are potentially breaking for a project that today declares -`@filterable`/`@sortable` on an array field. That declaration currently generates a rule that -cannot work, so this converts a runtime failure into a load failure. Call it out in the -changelog. +### B3. How breaking is this, precisely + +B1 and B2 make previously-loading metadata fail to load, so they are breaking on the metadata +contract. Three things bound it, and the bound should be stated rather than the label: + +1. **In-repo cost is zero.** A structural scan (per field node) of 1321 JSON and 124 YAML files + found **no** field carrying both `isArray: true` and `@filterable`/`@sortable: true`. Nothing + here breaks. +2. **What breaks was already broken.** Such a declaration emits a `like`/`eq` rule against a + `text[]` column — SQL that cannot execute. This converts a runtime failure into a load + failure, which is the direction this project already treats as a fix rather than a + regression. +3. **There is no mechanism by which an adopter was taught to write it.** This is the distinction + that matters, and it is worth stating because a sibling change in the same release window is + the *same category* and a **different magnitude**: #342's retired `@fields` + `@expr` + spelling appeared as a worked example in a shipped skill, so there is a plausible route by + which adopters acquired it. The array-`@filterable` form was never documented, never + exemplified, and never emitted by any generator — it is a silently-wrong emission nobody was + told to write. + +So: same category (breaking only for metadata outside this repo), materially lower exposure. +Changelog must still call it out — an adopter who wrote it independently gets a load error with +no warning — but the "we handed out the recipe" argument does not apply here. + +**Do not bundle this decision with another release's.** The version question for #335 is asked +when #335 is cut, on this evidence. Folding it into a ruling already made about a different +change would be deciding it on the wrong facts. --- From c22af2e7c0b24d687980e6b85567f8bbe967f706 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 23 Aug 2026 08:16:53 -0400 Subject: [PATCH 05/25] docs(plan): implementation plan for the whole-object rollup and the surface it lands in (#335) Fifteen tasks, 86 steps, TDD throughout. Half B first (two loader rules making the filter/sort contract honest about array columns) so Half A lands into a correct tier, then the loader change across four ports, the seven byte-gated prose copies, the TypeScript view column and its lowering, and a real-engine round-trip on both dialects. Carries the traps that have bitten this repo before as inline warnings rather than lore: `dotnet test` printing Passed! over a failed compile, a Maven or ci-local run piped through tail reporting tail's exit status, the integration round-trip living in a package no per-package run touches, and the fresh-worktree precondition that looks like a regression in codegen-ts. Co-Authored-By: Claude Opus 5 (1M context) --- ...026-08-22-issue-335-whole-object-rollup.md | 1427 +++++++++++++++++ 1 file changed, 1427 insertions(+) create mode 100644 docs/superpowers/plans/2026-08-22-issue-335-whole-object-rollup.md diff --git a/docs/superpowers/plans/2026-08-22-issue-335-whole-object-rollup.md b/docs/superpowers/plans/2026-08-22-issue-335-whole-object-rollup.md new file mode 100644 index 000000000..9e5d9613b --- /dev/null +++ b/docs/superpowers/plans/2026-08-22-issue-335-whole-object-rollup.md @@ -0,0 +1,1427 @@ +# Whole-Object Rollup (#335) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make `@of` optional on `origin.aggregate @agg: collect`, so a projection can roll up related rows as a JSON array of a declared value-object — and close the two holes in the queryable-projection contract that such a column would otherwise land in. + +**Architecture:** Two halves. **Half B first** (two loader rules making the filter/sort contract honest about array columns) so Half A lands into a correct tier. **Half A** then splits `collect` out of the `@of`-required gate in all four loaders, adds a value-object member→column resolution rule, and emits a new `collectObjectAgg` view column lowered to `jsonb_agg(jsonb_build_object(…))` on Postgres and `json_group_array(json_object(…))` on SQLite. Codegen is TypeScript-only (ADR-0015); the loader rules are cross-port. + +**Tech Stack:** TypeScript (Bun), C#, Java, Python. Shared JSON conformance corpus. Postgres + SQLite via Testcontainers. + +**Spec:** [`docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md`](../specs/2026-08-22-issue-335-whole-object-rollup-design.md) + +## Global Constraints + +- **`metamodelVersion` is NOT edited by this work.** It is already `0.11` (moved by #342) and that bump is unreleased. **Re-check before merging:** if `0.24.1` has shipped by then, `0.11` is released and this needs `node scripts/check-metamodel-version.mjs --set 0.12`. +- **ADR-0039 — never use own-only accessors.** Read `isArray` as `field.resolvedIsArray()`, attrs as `field.attr(...)`, children as `.children()`. `field.isArray` and `ownAttr`/`ownChildren` silently drop everything inherited via `extends`. The one exception in this plan is reading an `origin.*` node's own attrs (`origin.ownAttr(...)`), because `origin.*` never inherits (ADR-0029) — the existing code does this and carries that comment. +- **ADR-0023 — never invent an attribute.** This work adds **no** new registered attribute. `@of` is already `"required": false` structurally. +- **Named constants only.** Never inline `"collect"`, `"field"`, `"object"` etc. Import from `packages/metadata/src/constants.ts` (TS) and each port's equivalent. +- **Public repository.** No private/other-project names, no absolute home paths, in code, docs, fixtures **or commit messages**. +- **Every new load error needs a shared corpus fixture that triggers it.** A port-local unit test does not substitute — it proves one port enforces the rule, which is the true-but-insufficient assertion behind three separate #342 findings. +- **Scalar-arm output must stay byte-identical.** A `collect` **with** `@of` emits exactly the SQL it emits today. This is pinned by an explicit no-churn test, not assumed. +- **Test scoping.** Never run a bare `bun test` at the repo root. Scope to the package. + +## File Structure + +**Half B — loader only (4 ports):** +- `server/typescript/packages/metadata/src/loader/validation-passes.ts` — extend `validateFilterableHasSupportedOps` +- `server/csharp/MetaObjects/Loader/ValidationPasses.cs` — same rule +- `server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java` — same rule +- `server/python/src/metaobjects/loader/validation_passes.py` — same rule +- `fixtures/conformance/error-filterable-array-field/` — new negative fixture +- `fixtures/conformance/error-sortable-array-field/` — new negative fixture + +**Half A — loader (4 ports):** same four files, the `origin.aggregate` block. + +**Half A — registry prose (7 byte-gated files):** +- `spec/metamodel/origin.json` +- `fixtures/registry-conformance/expected-registry.json` +- `fixtures/metamodel-docs/expected/types/origin.md` +- `server/csharp/MetaObjects/Persistence/Origin/OriginSchema.cs` +- `server/csharp/MetaObjects/SpecMetamodel/origin.json` +- `server/python/src/metaobjects/spec_metamodel/origin.json` +- `server/typescript/packages/metadata/src/persistence/origin/origin-definition.embedded.ts` + +**Half A — codegen (TypeScript only):** +- `server/typescript/packages/codegen-ts/src/projection/view-spec.ts` — new `collectObjectAgg` kind +- `server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts` — object arm +- `server/typescript/packages/codegen-ts/src/projection/view-ddl-emit.ts` — both dialects + +**Half A — gates:** +- `fixtures/conformance/collect-whole-object/` + 6 negative fixtures +- `fixtures/conformance/flattened-kitchen-sink/` — restore `supplierBriefs` +- `server/typescript/packages/integration-tests/test/view-lifecycle-{pg,sqlite}.test.ts` — round-trip + +--- + +## Task 1: Half B — array fields cannot be `@filterable` (TypeScript) + +**Files:** +- Modify: `server/typescript/packages/metadata/src/loader/validation-passes.ts` (function `validateFilterableHasSupportedOps`, currently at ~`:436-456`) +- Test: `server/typescript/packages/metadata/test/validation-filterable-array.test.ts` (create) + +**Interfaces:** +- Consumes: nothing from earlier tasks. +- Produces: the extended `validateFilterableHasSupportedOps(root: MetaData): ParseError[]`, emitting `ERR_FILTERABLE_UNSUPPORTED_SUBTYPE` for array fields. Task 2 ports this rule; Task 3 gates it cross-port. + +**Why:** `filterSubTypeFor` in `codegen-ts/src/templates/filter-allowlist.ts` falls through to `"string"` for anything unrecognised and **nothing in that file consults `isArray`**, so `field.string isArray: true @filterable: true` emits a `like`/`eq` rule against a `text[]` column — SQL that cannot execute. No operator in the FR-009 band applies to an array, which is the same reason `field.object` is already rejected, so it reuses the same error code and the same door. + +- [ ] **Step 1: Write the failing test** + +Create `server/typescript/packages/metadata/test/validation-filterable-array.test.ts`: + +```ts +import { test, expect, describe } from "bun:test"; +import { MetaDataLoader } from "../src/loader/meta-data-loader.js"; + +const model = (fieldJson: string) => `{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { + "name": "Product", + "children": [ + { "source.rdb": { "@kind": "table", "@table": "products" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "@fields": ["id"] } }, + ${fieldJson} + ] + }} + ] + } +}`; + +describe("@filterable on an array field", () => { + test("an array field marked @filterable fails to load", () => { + const src = model( + `{ "field.string": { "name": "tags", "isArray": true, "@filterable": true } }`, + ); + const loader = new MetaDataLoader(); + expect(() => loader.loadFromString(src, "meta.demo.json")).toThrow( + /ERR_FILTERABLE_UNSUPPORTED_SUBTYPE/, + ); + }); + + test("the same field WITHOUT isArray still loads", () => { + const src = model( + `{ "field.string": { "name": "tags", "@filterable": true } }`, + ); + const loader = new MetaDataLoader(); + expect(() => loader.loadFromString(src, "meta.demo.json")).not.toThrow(); + }); + + test("an array field NOT marked @filterable still loads", () => { + const src = model( + `{ "field.string": { "name": "tags", "isArray": true } }`, + ); + const loader = new MetaDataLoader(); + expect(() => loader.loadFromString(src, "meta.demo.json")).not.toThrow(); + }); +}); +``` + +> If `loadFromString` is not the loader's string entry point in this codebase, find the one the neighbouring tests in `server/typescript/packages/metadata/test/` use and match it exactly — do not invent an API. + +- [ ] **Step 2: Run the test to verify it fails** + +```bash +cd server/typescript/packages/metadata && bun test test/validation-filterable-array.test.ts +``` + +Expected: the first test FAILS (no error is thrown); the other two PASS. + +- [ ] **Step 3: Extend the validation pass** + +In `validation-passes.ts`, inside `validateFilterableHasSupportedOps`, replace the single-condition body of the field loop with an array check ahead of the subtype check: + +```ts + // ADR-0039: resolving — a concrete field may inherit @filterable via extends. + if (field.attr(FIELD_ATTR_FILTERABLE) !== true) continue; + + // #335 Half B — an ARRAY field has no operator band either. Every FR-009 + // operator (eq/ne/gt/gte/lt/lte/in/like/isNull) is a scalar comparison; + // none applies to a collection column. The allowlist template does not + // consult isArray and falls through to the "string" band, so this + // previously emitted a `like` rule against a text[] column — SQL that + // cannot execute. Same reason as the subtype check below, so same code. + // ADR-0039: resolvedIsArray(), never the own `isArray` flag. + if (field.resolvedIsArray()) { + errors.push( + new ParseError( + `Field "${obj.name}.${field.name}" has @filterable: true but is an array ` + + `(isArray: true). No filter operator applies to a collection column. ` + + `Remove @filterable from this field.`, + { code: "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE", source: field.source }, + ), + ); + continue; + } + + if (opsForSubType(field.subType).length > 0) continue; +``` + +- [ ] **Step 4: Run the test to verify it passes** + +```bash +cd server/typescript/packages/metadata && bun test test/validation-filterable-array.test.ts +``` + +Expected: 3 pass, 0 fail. + +- [ ] **Step 5: Run the full metadata suite for regressions** + +```bash +cd server/typescript/packages/metadata && bun test +``` + +Expected: 0 fail. If an existing fixture breaks, it is declaring `@filterable` on an array field — read it before changing it; that fixture is pinning the defect. + +- [ ] **Step 6: Commit** + +```bash +git add server/typescript/packages/metadata/src/loader/validation-passes.ts \ + server/typescript/packages/metadata/test/validation-filterable-array.test.ts +git commit -m "fix(metamodel): an array field has no filter-operator band either (#335)" +``` + +--- + +## Task 2: Half B — `@sortable` gets the same subtype/array validation (TypeScript) + +**Files:** +- Modify: `server/typescript/packages/metadata/src/loader/validation-passes.ts` +- Test: `server/typescript/packages/metadata/test/validation-sortable-array.test.ts` (create) + +**Interfaces:** +- Consumes: Task 1's extended `validateFilterableHasSupportedOps`. +- Produces: `validateSortableHasSupportedSubtype(root: MetaData): ParseError[]`, exported alongside it and registered in the same pass list. Task 3 gates it. + +**Why:** `@sortable` defaults from `@filterable`, so it is only independently set when explicit — and **nothing validates it at all**, versus a hard error for `@filterable`. `@sortable: true` on a JSON or array column currently passes the loader and emits a sort entry over a column that cannot be ordered. + +- [ ] **Step 1: Write the failing test** + +Create `server/typescript/packages/metadata/test/validation-sortable-array.test.ts`: + +```ts +import { test, expect, describe } from "bun:test"; +import { MetaDataLoader } from "../src/loader/meta-data-loader.js"; + +const model = (fieldJson: string) => `{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { + "name": "Product", + "children": [ + { "source.rdb": { "@kind": "table", "@table": "products" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "@fields": ["id"] } }, + ${fieldJson} + ] + }} + ] + } +}`; + +describe("@sortable subtype validation", () => { + test("an array field marked @sortable fails to load", () => { + const src = model( + `{ "field.string": { "name": "tags", "isArray": true, "@sortable": true } }`, + ); + const loader = new MetaDataLoader(); + expect(() => loader.loadFromString(src, "meta.demo.json")).toThrow( + /ERR_SORTABLE_UNSUPPORTED_SUBTYPE/, + ); + }); + + test("a field.object marked @sortable fails to load", () => { + const src = model( + `{ "field.object": { "name": "spec", "@objectRef": "acme::shop::Spec", "@sortable": true } }`, + ); + const loader = new MetaDataLoader(); + expect(() => loader.loadFromString(src, "meta.demo.json")).toThrow( + /ERR_SORTABLE_UNSUPPORTED_SUBTYPE/, + ); + }); + + test("a plain scalar marked @sortable still loads", () => { + const src = model( + `{ "field.string": { "name": "sku", "@sortable": true } }`, + ); + const loader = new MetaDataLoader(); + expect(() => loader.loadFromString(src, "meta.demo.json")).not.toThrow(); + }); +}); +``` + +- [ ] **Step 2: Run the test to verify it fails** + +```bash +cd server/typescript/packages/metadata && bun test test/validation-sortable-array.test.ts +``` + +Expected: the first two FAIL (nothing thrown); the third PASSES. + +- [ ] **Step 3: Add the error code to the ledger** + +`ERR_SORTABLE_UNSUPPORTED_SUBTYPE` is new. Add it to **all** of: +- `fixtures/conformance/ERROR-CODES.json` +- `server/typescript/packages/metadata/src/errors.ts` (exact-bidirectional — the TS ledger test fails if the sets differ) +- `server/python/src/metaobjects/errors.py` (superset) +- `server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java` +- `server/csharp/MetaObjects/Errors.cs` + +Copy the description style of the neighbouring `ERR_FILTERABLE_UNSUPPORTED_SUBTYPE` entry. Do not paraphrase the rule differently in different ports — the strings are compared. + +- [ ] **Step 4: Write the validation pass** + +In `validation-passes.ts`, directly below `validateFilterableHasSupportedOps`: + +```ts +// @sortable on a subtype or shape that cannot be ordered (#335 Half B) +// --------------------------------------------------------------------------- +// @sortable defaults FROM @filterable, so it is independently set only when +// explicit — and nothing validated it, while @filterable has had a hard error +// since SP-H Unit9. A @sortable JSON or array column emits a sort entry over a +// column no dialect can ORDER BY meaningfully. → ERR_SORTABLE_UNSUPPORTED_SUBTYPE. + +export function validateSortableHasSupportedSubtype(root: MetaData): ParseError[] { + const errors: ParseError[] = []; + // ADR-0039: root has no super; children()==ownChildren() but resolving is the default. + for (const obj of root.children().filter((c) => c.type === TYPE_OBJECT)) { + // children() — inherited @sortable fields (via extends:/super:) are visible. + for (const field of obj.children().filter((c) => c.type === TYPE_FIELD)) { + // ADR-0039: resolving — a concrete field may inherit @sortable via extends. + if (field.attr(FIELD_ATTR_SORTABLE) !== true) continue; + // ADR-0039: resolvedIsArray(), never the own `isArray` flag. + const isArray = field.resolvedIsArray(); + if (!isArray && opsForSubType(field.subType).length > 0) continue; + errors.push( + new ParseError( + `Field "${obj.name}.${field.name}" has @sortable: true but ` + + (isArray + ? `is an array (isArray: true) — a collection column has no ordering.` + : `its subtype "${field.subType}" cannot be ordered.`) + + ` Remove @sortable from this field.`, + { code: "ERR_SORTABLE_UNSUPPORTED_SUBTYPE", source: field.source }, + ), + ); + } + } + return errors; +} +``` + +Register it in the same list that calls `validateFilterableHasSupportedOps` — grep for that name to find the pass registry and add the new function immediately after it. + +If `FIELD_ATTR_SORTABLE` does not exist in `constants.ts`, add it there beside `FIELD_ATTR_FILTERABLE`; do not inline `"sortable"`. + +- [ ] **Step 5: Run the test to verify it passes** + +```bash +cd server/typescript/packages/metadata && bun test test/validation-sortable-array.test.ts +``` + +Expected: 3 pass, 0 fail. + +- [ ] **Step 6: Run the full metadata suite** + +```bash +cd server/typescript/packages/metadata && bun test +``` + +Expected: 0 fail. + +- [ ] **Step 7: Commit** + +```bash +git add server/typescript/packages/metadata/src/loader/validation-passes.ts \ + server/typescript/packages/metadata/src/errors.ts \ + server/typescript/packages/metadata/test/validation-sortable-array.test.ts \ + fixtures/conformance/ERROR-CODES.json \ + server/python/src/metaobjects/errors.py \ + server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java \ + server/csharp/MetaObjects/Errors.cs +git commit -m "fix(metamodel): @sortable gets the subtype validation @filterable already had (#335)" +``` + +--- + +## Task 3: Half B — shared conformance fixtures + +**Files:** +- Create: `fixtures/conformance/error-filterable-array-field/input/meta.demo.json` +- Create: `fixtures/conformance/error-filterable-array-field/expected-errors.json` +- Create: `fixtures/conformance/error-sortable-array-field/input/meta.demo.json` +- Create: `fixtures/conformance/error-sortable-array-field/expected-errors.json` +- Modify: `fixtures/conformance/README.md` + +**Interfaces:** +- Consumes: the two rules from Tasks 1–2. +- Produces: the cross-port gate. Task 4's port work is verified against these. + +**Why:** This is the point of Half B. A load error with no fixture that triggers it is the same blind spot one layer up — "no fixture covers it" and "every port enforces it" are indistinguishable on a green suite. A structural scan of 1321 JSON and 124 YAML files found **zero** fields carrying both `isArray: true` and `@filterable`/`@sortable: true`, so nothing in the corpus exercises an array field through the filter tier at all. + +- [ ] **Step 1: Create the filterable fixture input** + +`fixtures/conformance/error-filterable-array-field/input/meta.demo.json`: + +```json +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Product", + "children": [ + { "source.rdb": { "@kind": "table", "@table": "products" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "@fields": ["id"] } }, + { "field.string": { "name": "tags", "isArray": true, "@filterable": true } } + ] + } + } + ] + } +} +``` + +- [ ] **Step 2: Create the filterable expected-errors** + +`fixtures/conformance/error-filterable-array-field/expected-errors.json`: + +```json +{ + "errors": [ + { + "code": "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE", + "source": { + "format": "json", + "files": ["meta.demo.json"], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[3]['field.string']" + } + } + ], + "warnings": [] +} +``` + +- [ ] **Step 3: Create the sortable fixture** + +Same two files under `fixtures/conformance/error-sortable-array-field/`, with the field changed to `"@sortable": true` and the code to `ERR_SORTABLE_UNSUPPORTED_SUBTYPE`. + +- [ ] **Step 4: Run the TS conformance corpus** + +```bash +cd server/typescript/packages/metadata && bun test test/conformance.test.ts +``` + +Expected: PASS, count increased by 2. If the `jsonPath` in `expected-errors.json` does not match, the failure message prints the actual path — copy it verbatim rather than guessing. + +- [ ] **Step 5: Document the fixtures in the corpus README** + +In `fixtures/conformance/README.md`, add both fixtures to the case list, each with one line saying which rule it covers. State that they are the **only** cases exercising an array field through the filter/sort tier. + +- [ ] **Step 6: Commit** + +```bash +git add fixtures/conformance/error-filterable-array-field \ + fixtures/conformance/error-sortable-array-field \ + fixtures/conformance/README.md +git commit -m "test(conformance): gate the array filter/sort rules cross-port (#335)" +``` + +--- + +## Task 4: Half B — port the two rules to C#, Java, Python + +**Files:** +- Modify: `server/csharp/MetaObjects/Loader/ValidationPasses.cs` +- Modify: `server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java` +- Modify: `server/python/src/metaobjects/loader/validation_passes.py` + +**Interfaces:** +- Consumes: Tasks 1–3. The TypeScript implementation is the reference; the fixtures are the contract. +- Produces: all four ports green on the two fixtures from Task 3. + +**Why:** The loader contract is cross-port. Four ports hand-rolling the same read is what produced the `@fields` divergence in #342 — so implement each against the fixture, not against a reading of the TS source. + +- [ ] **Step 1: Find each port's filterable pass** + +```bash +grep -rn "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE" server/csharp server/java server/python --include=*.cs --include=*.java --include=*.py +``` + +Read each one before editing. Each port already has this rule; you are adding the array condition beside it and adding the sortable pass after it. + +- [ ] **Step 2: Use each port's RESOLVING array accessor** + +This is the single highest-risk detail. Do **not** read a raw `isArray` field or a raw attr. + +| Port | Correct accessor | +|---|---| +| TypeScript | `field.resolvedIsArray()` | +| Python | `node.resolved_is_array()` | +| C# / Java | grep the port's existing `@agg:collect` array check — it already reads array-ness the resolving way; copy that call exactly | + +For C# and Java: + +```bash +grep -rn "collect produces a list" server/csharp server/java +``` + +The line above that error is the resolving accessor to reuse. + +- [ ] **Step 3: Implement in each port** + +Mirror the TypeScript semantics exactly: +- `@filterable: true` + array ⇒ `ERR_FILTERABLE_UNSUPPORTED_SUBTYPE` +- `@sortable: true` + (array **or** no operator band) ⇒ `ERR_SORTABLE_UNSUPPORTED_SUBTYPE` +- Neither fires when the attr is absent or `false`. + +- [ ] **Step 4: Run each port's conformance suite** + +```bash +cd server/python && uv run pytest --extra integration -k conformance +cd server/csharp && dotnet build && dotnet test --filter "FullyQualifiedName~Conformance" +cd server/java && mvn -q -pl metadata test +``` + +Expected: all green on the two new fixtures. + +**Two traps, both previously shipped:** `dotnet test` prints `Passed!` even when a project failed to COMPILE — grep the output for `error CS` and confirm the build succeeded separately. Never pipe a Maven run through `tail`; the exit status becomes `tail`'s. + +- [ ] **Step 5: Commit** + +```bash +git add server/csharp/MetaObjects/Loader/ValidationPasses.cs \ + server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java \ + server/python/src/metaobjects/loader/validation_passes.py +git commit -m "fix(metamodel): array filter/sort rules in the remaining three ports (#335)" +``` + +--- + +## Task 5: Half A — `@of` optional on `collect` (TypeScript loader) + +**Files:** +- Modify: `server/typescript/packages/metadata/src/loader/validation-passes.ts` — the block under the comment `// --- count/sum/avg/min/max/collect: @of REQUIRED ---` +- Test: `server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts` (create) + +**Interfaces:** +- Consumes: nothing from Half B (independent rules, same file). +- Produces: the `@of`-absent collect branch. Task 6 adds member resolution; Task 8 ports it. + +**Why:** Today `@of` is required for `collect`, so only a single scalar column can be rolled up. `@of` is already `"required": false` in the registry — the constraint is validation-only. + +- [ ] **Step 1: Write the failing test** + +Create `server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts`. Include a **positive** case and each **negative** arm: + +```ts +import { test, expect, describe } from "bun:test"; +import { MetaDataLoader } from "../src/loader/meta-data-loader.js"; + +/** Product 1:N Supplier, plus a projection rolling suppliers up as objects. */ +const model = (collectField: string, extra = "") => `{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { "name": "Supplier", "children": [ + { "source.rdb": { "@kind": "table", "@table": "suppliers" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "name" } }, + { "field.long": { "name": "productId" } }, + { "identity.primary": { "@fields": ["id"] } }, + { "identity.reference": { "name": "product", "@references": "Product", "@fields": ["productId"] } } + ]}}, + { "object.entity": { "name": "Product", "children": [ + { "source.rdb": { "@kind": "table", "@table": "products" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "@fields": ["id"] } }, + { "relationship.association": { "name": "suppliers", "@cardinality": "many", "@objectRef": "Supplier" } } + ]}}, + { "object.value": { "name": "SupplierBrief", "children": [ + { "field.long": { "name": "id" } }, + { "field.string": { "name": "name" } } + ]}}, + ${extra} + { "object.projection": { "name": "ProductWithSuppliers", "extends": "Product", "children": [ + { "source.rdb": { "@kind": "view", "@view": "v_product_suppliers" } }, + { "identity.primary": { "@fields": ["id"] } }, + ${collectField} + ]}} + ] + } +}`; + +const load = (src: string) => new MetaDataLoader().loadFromString(src, "meta.demo.json"); + +const WHOLE_OBJECT = `{ "field.object": { + "name": "supplierBriefs", "isArray": true, "@objectRef": "SupplierBrief", + "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers" } } ] +}}`; + +describe("@of-absent collect (whole-object rollup)", () => { + test("loads on a field.object @objectRef isArray with @via", () => { + expect(() => load(model(WHOLE_OBJECT))).not.toThrow(); + }); + + test("fails without @objectRef", () => { + expect(() => load(model(`{ "field.object": { + "name": "supplierBriefs", "isArray": true, + "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers" } } ] + }}`))).toThrow(/ERR_INVALID_ORIGIN/); + }); + + test("fails when @objectRef targets an entity, not a value", () => { + expect(() => load(model(`{ "field.object": { + "name": "supplierBriefs", "isArray": true, "@objectRef": "Supplier", + "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers" } } ] + }}`))).toThrow(/ERR_SUBTYPE_RULE_VIOLATION/); + }); + + test("fails without @via (nothing to infer the relation from)", () => { + expect(() => load(model(`{ "field.object": { + "name": "supplierBriefs", "isArray": true, "@objectRef": "SupplierBrief", + "children": [ { "origin.aggregate": { "@agg": "collect" } } ] + }}`))).toThrow(/ERR_INVALID_ORIGIN/); + }); + + test("fails when @distinct is declared", () => { + expect(() => load(model(`{ "field.object": { + "name": "supplierBriefs", "isArray": true, "@objectRef": "SupplierBrief", + "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers", "@distinct": true } } ] + }}`))).toThrow(/ERR_INVALID_ORIGIN/); + }); + + test("a collect WITH @of is unaffected", () => { + expect(() => load(model(`{ "field.string": { + "name": "supplierNames", "isArray": true, + "children": [ { "origin.aggregate": { "@agg": "collect", "@of": "Supplier.name", "@via": "Product.suppliers" } } ] + }}`))).not.toThrow(); + }); +}); +``` + +- [ ] **Step 2: Run the test to verify it fails** + +```bash +cd server/typescript/packages/metadata && bun test test/validation-collect-whole-object.test.ts +``` + +Expected: the positive cases FAIL with `missing @of`; the negatives may pass for the wrong reason. That is fine — Step 3 makes all six correct. + +- [ ] **Step 3: Split `collect` out of the `@of`-required gate** + +Replace the block beginning `// --- count/sum/avg/min/max/collect: @of REQUIRED ---` with: + +```ts + // --- @of: REQUIRED for count/sum/avg/min/max; OPTIONAL for collect --- + // #335 — an @of-absent collect is a WHOLE-OBJECT rollup: collect the + // related rows as an array of the field's declared @objectRef value + // object rather than an array of one scalar column. + if (!ofPresent) { + if (!isCollect) { + errors.push(new ParseError( + `origin.aggregate on ${obj.name}.${field.name}: missing @of.`, + { code: "ERR_INVALID_ORIGIN", source: src })); + continue; + } + // Whole-object rollup. The carrying field must be a field.object + // naming a value object, and @via must be explicit (there is no @of + // entity to infer the single-hop relation from). + // ADR-0039: resolving — @objectRef may be inherited via extends. + const objectRef = field.attr(FIELD_ATTR_OBJECT_REF); + if (field.subType !== FIELD_SUBTYPE_OBJECT || typeof objectRef !== "string" || objectRef === "") { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: @of is omitted, so this is a ` + + `whole-object rollup — the carrying field must be a field.object declaring @objectRef ` + + `(add @of to collect a single column instead).`, + { code: "ERR_INVALID_ORIGIN", source: src })); + continue; + } + // #210's value-only rule is PAYLOAD-scoped and never reaches a + // projection-hosted field, so this branch enforces it itself. + // Without it an @objectRef to an entity silently rolls up the FULL + // entity — the #270 shape, this time baked into DDL. + const refPkg = field.parent?.package ?? obj.package ?? ""; + const refTarget = resolveObjectRef(root, objectRef, refPkg).node; + if (refTarget !== undefined && refTarget.subType !== OBJECT_SUBTYPE_VALUE) { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: @objectRef '${objectRef}' ` + + `resolves to ${TYPE_OBJECT}.${refTarget.subType} — a whole-object rollup must target an ` + + `object.value (#210, ADR-0028).`, + { code: "ERR_SUBTYPE_RULE_VIOLATION", source: src })); + continue; + } + // ADR-0039: own — origin.* never inherits (ADR-0029). + const viaAttr = origin.ownAttr(ORIGIN_AGGREGATE_ATTR_VIA); + if (typeof viaAttr !== "string" || viaAttr === "") { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: @via is required on a ` + + `whole-object rollup — there is no @of entity to infer the relationship from.`, + { code: "ERR_INVALID_ORIGIN", source: src })); + continue; + } + // @distinct is refused on the object form. It is NOT an engine limit + // (both engines dedupe JSON objects); it is a guaranteed no-op + // whenever the value object carries the entity's primary key, which + // is the common case, and a silent no-op is worse than a refusal. + if (hasDistinct) { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: @distinct is not supported on a ` + + `whole-object rollup (it is a no-op whenever the value object carries the primary key).`, + { code: "ERR_INVALID_ORIGIN", source: src })); + continue; + } + const hops = _validateViaPath(viaAttr, root, obj, field.name, src, errors); + if (hops !== undefined) _checkAggregateCardinality(hops, obj, field.name, src, errors); + // @orderBy keys resolve against the @via TERMINAL entity, not @of. + if (hasOrderBy) { + const terminal = _viaTerminalEntityNode(viaAttr, root, obj); + _validateOrderByKeys(orderBy, terminal, obj, field.name, "origin.aggregate @agg:collect", src, errors); + } + continue; + } +``` + +Add a `_viaTerminalEntityNode` helper beside `_validateViaPath` that walks the same dotted `@via` segments and returns the terminal entity node (or `undefined`). `extract-view-spec.ts:477` has the codegen-side equivalent (`viaTerminalEntity`) — mirror its hop-walking logic, returning the loader's node type. + +Import any constants not already imported (`FIELD_SUBTYPE_OBJECT`, `FIELD_ATTR_OBJECT_REF`, `OBJECT_SUBTYPE_VALUE`, `resolveObjectRef`). + +- [ ] **Step 4: Run the test to verify it passes** + +```bash +cd server/typescript/packages/metadata && bun test test/validation-collect-whole-object.test.ts +``` + +Expected: 6 pass, 0 fail. + +- [ ] **Step 5: Run the full metadata suite** + +```bash +cd server/typescript/packages/metadata && bun test +``` + +Expected: 0 fail. + +- [ ] **Step 6: Commit** + +```bash +git add server/typescript/packages/metadata/src/loader/validation-passes.ts \ + server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts +git commit -m "feat(metamodel): @of is optional on @agg:collect — whole-object rollup (#335)" +``` + +--- + +## Task 6: Half A — value-object member resolution (the #270 guard) + +**Files:** +- Modify: `server/typescript/packages/metadata/src/loader/validation-passes.ts` (the branch from Task 5) +- Modify: the five error ledgers (as in Task 2, Step 3) +- Test: `server/typescript/packages/metadata/test/validation-collect-member-resolution.test.ts` (create) + +**Interfaces:** +- Consumes: Task 5's `@of`-absent branch and its `_viaTerminalEntityNode` helper. +- Produces: `ERR_COLLECT_MEMBER_UNRESOLVED`. Task 10's emitter relies on every member resolving. + +**Why:** The lowering projects exactly the declared value-object's members. A member that matches no column on the `@via` terminal entity must be an **error** — failing open here is how #270 turned a curated value object into the full entity. Element type stays declared-authoritative. + +- [ ] **Step 1: Write the failing test** + +Create `server/typescript/packages/metadata/test/validation-collect-member-resolution.test.ts`, reusing the model helper from Task 5 (copy it — the engineer may read tasks out of order) with two cases: + +```ts + test("a VO member with no matching column on the @via terminal fails", () => { + // SupplierBrief declares `nickname`; Supplier has no such field. + expect(() => load(modelWithVo( + `{ "field.long": { "name": "id" } }, { "field.string": { "name": "nickname" } }`, + WHOLE_OBJECT, + ))).toThrow(/ERR_COLLECT_MEMBER_UNRESOLVED/); + }); + + test("a VO member whose type differs from the matched column fails", () => { + // SupplierBrief declares `name` as a long; Supplier.name is a string. + expect(() => load(modelWithVo( + `{ "field.long": { "name": "id" } }, { "field.long": { "name": "name" } }`, + WHOLE_OBJECT, + ))).toThrow(/ERR_INVALID_ORIGIN/); + }); +``` + +Write `modelWithVo(voFields, collectField)` as a variant of Task 5's `model` that parameterises `SupplierBrief`'s field list. + +- [ ] **Step 2: Run the test to verify it fails** + +```bash +cd server/typescript/packages/metadata && bun test test/validation-collect-member-resolution.test.ts +``` + +Expected: both FAIL (nothing thrown). + +- [ ] **Step 3: Add `ERR_COLLECT_MEMBER_UNRESOLVED` to all five ledgers** + +Same five files as Task 2, Step 3. + +- [ ] **Step 4: Implement member resolution** + +Inside Task 5's branch, after the `@via` hop validation and before its `continue`: + +```ts + // Member resolution — the lowering projects EXACTLY the declared value + // object's members, matched by NAME against the @via terminal entity. + // An unmatched member must error: failing open is how #270 turned a + // curated value object into the full entity. + const terminalEntity = _viaTerminalEntityNode(viaAttr, root, obj); + if (terminalEntity !== undefined && refTarget !== undefined) { + // children() — a value object may inherit members via extends. + const terminalFields = terminalEntity.children().filter((c) => c.type === TYPE_FIELD); + for (const member of refTarget.children().filter((c) => c.type === TYPE_FIELD)) { + const match = terminalFields.find((f) => f.name === member.name); + if (match === undefined) { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: value-object member ` + + `'${member.name}' has no matching field on '${terminalEntity.name}' — a whole-object ` + + `rollup projects exactly the declared members.`, + { code: "ERR_COLLECT_MEMBER_UNRESOLVED", source: src })); + continue; + } + // Per-member type agreement — the object-form analogue of the + // scalar element-type check, same #185 type-preserving doctrine. + if (member.subType !== match.subType) { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: value-object member ` + + `'${member.name}' is field.${member.subType} but '${terminalEntity.name}.${match.name}' ` + + `is field.${match.subType} — a whole-object rollup preserves each member's type.`, + { code: "ERR_INVALID_ORIGIN", source: src })); + } + } + } +``` + +- [ ] **Step 5: Run both Half A tests** + +```bash +cd server/typescript/packages/metadata && bun test test/validation-collect-member-resolution.test.ts test/validation-collect-whole-object.test.ts +``` + +Expected: 8 pass, 0 fail. + +- [ ] **Step 6: Commit** + +```bash +git add server/typescript/packages/metadata/src/loader/validation-passes.ts \ + server/typescript/packages/metadata/src/errors.ts \ + server/typescript/packages/metadata/test/validation-collect-member-resolution.test.ts \ + fixtures/conformance/ERROR-CODES.json \ + server/python/src/metaobjects/errors.py \ + server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java \ + server/csharp/MetaObjects/Errors.cs +git commit -m "feat(metamodel): whole-object rollup resolves VO members against the @via terminal (#335)" +``` + +--- + +## Task 7: Half A — conformance fixtures (positive + 6 negative arms) + +**Files:** +- Create: `fixtures/conformance/collect-whole-object/{input/meta.demo.json,expected.json}` +- Create: six `fixtures/conformance/error-collect-*/` directories, each with `input/meta.demo.json` + `expected-errors.json` +- Modify: `fixtures/conformance/README.md` + +**Interfaces:** +- Consumes: Tasks 5–6. +- Produces: the cross-port contract Task 8 implements against. + +The six negative arms, one fixture each: + +| Fixture | Arm | Code | +|---|---|---| +| `error-collect-no-object-ref` | no `@objectRef` | `ERR_INVALID_ORIGIN` | +| `error-collect-ref-not-value` | `@objectRef` → entity | `ERR_SUBTYPE_RULE_VIOLATION` | +| `error-collect-no-via` | `@via` absent | `ERR_INVALID_ORIGIN` | +| `error-collect-via-to-one` | every hop to-one | `ERR_ORIGIN_CARDINALITY` | +| `error-collect-member-unresolved` | VO member not on terminal | `ERR_COLLECT_MEMBER_UNRESOLVED` | +| `error-collect-distinct` | `@distinct` declared | `ERR_INVALID_ORIGIN` | + +- [ ] **Step 1: Build the positive fixture** + +Use the model from Task 5's test as `input/meta.demo.json`. Generate `expected.json` by running the corpus and copying the canonical serialization the runner prints on mismatch — **do not hand-write it**. + +- [ ] **Step 2: Build the six negative fixtures** + +Each is the positive input with one thing changed. Copy the `expected-errors.json` shape from `fixtures/conformance/error-origin-aggregate-no-to-many/expected-errors.json`; take the `jsonPath` from the runner's failure output rather than deriving it. + +- [ ] **Step 3: Run the corpus** + +```bash +cd server/typescript/packages/metadata && bun test test/conformance.test.ts +``` + +Expected: PASS, count increased by 7. + +- [ ] **Step 4: Document in the README** + +Add all seven to the case list. For the positive one, state that it is the corpus's only whole-object rollup. + +- [ ] **Step 5: Commit** + +```bash +git add fixtures/conformance/collect-whole-object fixtures/conformance/error-collect-* fixtures/conformance/README.md +git commit -m "test(conformance): gate the whole-object rollup and its six error arms (#335)" +``` + +--- + +## Task 8: Half A — port the loader rules to C#, Java, Python + +**Files:** +- Modify: `server/csharp/MetaObjects/Loader/ValidationPasses.cs` +- Modify: `server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java` +- Modify: `server/python/src/metaobjects/loader/validation_passes.py` + +**Interfaces:** +- Consumes: Tasks 5–7. The seven fixtures are the contract. +- Produces: four ports green on the whole corpus. + +- [ ] **Step 1: Locate each port's gate** + +```bash +grep -rn "missing @of" server/csharp server/java server/python +``` + +Python phrases it differently — search for its `AGG_COLLECT` handling near the `collect produces a list` message instead. + +- [ ] **Step 2: Implement the same six rules per port** + +Object-ref required; ref must be `object.value`; `@via` required; cardinality; member resolution; `@distinct` refused. Use each port's **resolving** accessors throughout (see Task 4, Step 2). + +- [ ] **Step 3: Run each port's conformance suite** + +```bash +cd server/python && uv run pytest --extra integration -k conformance +cd server/csharp && dotnet build && dotnet test --filter "FullyQualifiedName~Conformance" +cd server/java && mvn -q -pl metadata test +``` + +Expected: green on all seven new fixtures. Same two traps as Task 4, Step 4. + +- [ ] **Step 4: Commit** + +```bash +git add server/csharp/MetaObjects/Loader/ValidationPasses.cs \ + server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java \ + server/python/src/metaobjects/loader/validation_passes.py +git commit -m "feat(metamodel): whole-object rollup in the remaining three ports (#335)" +``` + +--- + +## Task 9: Half A — registry prose in seven byte-gated files + +**Files:** the seven listed under **File Structure**. + +**Interfaces:** +- Consumes: Tasks 5–8 (the rule the prose describes). +- Produces: `registry-conformance` green in all five ports. + +**Why:** `@of` is already `"required": false` structurally, so nothing changes there — but its `description` says *"Required for count/sum/avg/min/max/collect"*, which is now false. That string is byte-gated in seven files; change fewer than seven and `registry-conformance` goes red in whichever port you missed. + +- [ ] **Step 1: Find every copy** + +```bash +grep -rln "Required for count/sum/avg/min/max/collect" . +``` + +Expected: exactly 7 files. + +- [ ] **Step 2: Rewrite the sentence identically in all seven** + +New text, byte-identical everywhere: + +``` +Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max; OPTIONAL for collect, where absent means a whole-object rollup of the field's declared @objectRef value object; forbidden for any/all (which quantify over rows via @filter, not a column). +``` + +- [ ] **Step 3: Regenerate the derived docs fixture** + +`fixtures/metamodel-docs/expected/types/origin.md` is generated. Find its generator command in `fixtures/metamodel-docs/README.md` and run it rather than hand-editing. + +- [ ] **Step 4: Run registry conformance in every port** + +```bash +cd server/typescript/packages/metadata && bun test test/registry-conformance.test.ts +cd server/python && uv run pytest --extra integration -k registry +cd server/csharp && dotnet build && dotnet test --filter "FullyQualifiedName~Registry" +cd server/java && mvn -q -pl metadata test -Dtest='*Registry*' +``` + +Expected: all green. + +- [ ] **Step 5: Confirm `metamodelVersion` still needs no move** + +```bash +node scripts/check-metamodel-version.mjs --explain +git tag --list 'v0.*' --sort=-v:refname | head -1 +``` + +If the newest release tag is still `v0.24.0`, `0.11` is unreleased and no edit is needed. If `v0.24.1` has shipped, run `node scripts/check-metamodel-version.mjs --set 0.12`. + +- [ ] **Step 6: Commit** + +```bash +git add spec/metamodel/origin.json fixtures/registry-conformance/expected-registry.json \ + fixtures/metamodel-docs/expected/types/origin.md \ + server/csharp/MetaObjects/Persistence/Origin/OriginSchema.cs \ + server/csharp/MetaObjects/SpecMetamodel/origin.json \ + server/python/src/metaobjects/spec_metamodel/origin.json \ + server/typescript/packages/metadata/src/persistence/origin/origin-definition.embedded.ts +git commit -m "docs(metamodel): @of is optional on collect, in all seven byte-gated copies (#335)" +``` + +--- + +## Task 10: Half A — the `collectObjectAgg` view column (TypeScript codegen) + +**Files:** +- Modify: `server/typescript/packages/codegen-ts/src/projection/view-spec.ts` +- Modify: `server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts` +- Test: `server/typescript/packages/codegen-ts/test/projection/collect-whole-object-spec.test.ts` (create) + +**Interfaces:** +- Consumes: the loader guarantees from Tasks 5–8 (every member resolves; `@objectRef` is a value; `@via` present). +- Produces: + +```ts +{ + readonly kind: "collectObjectAgg"; + readonly fieldName: string; + readonly dbColAlias: string; + readonly sourceAlias: string; + readonly joinedPkColumn: string; + readonly members: readonly { readonly memberName: string; readonly sourceColumn: string }[]; + readonly orderBy: readonly ViewOrderKey[]; +} +``` + +Task 11 lowers this to SQL. + +**Why a separate kind, not an arm of `collectAgg`:** the payloads differ (a member list vs one source column), and `collectAgg` is consumed by `viewOrderKeysAreDeterministic` (`:1005`) and the real-aggregate predicate (`:1021`), which a union would force every consumer to re-narrow. + +- [ ] **Step 1: Add the kind to `view-spec.ts`** + +Add the interface above to the `ViewColumn` union, beside the existing `collectAgg` member. + +- [ ] **Step 2: Write the failing test** + +Create the test file; build the same Product/Supplier/SupplierBrief model as Task 5, run `extractViewSpec`, and assert: + +```ts + const col = spec.columns.find((c) => c.fieldName === "supplierBriefs"); + expect(col?.kind).toBe("collectObjectAgg"); + expect(col?.members).toEqual([ + { memberName: "id", sourceColumn: "id" }, + { memberName: "name", sourceColumn: "name" }, + ]); + expect(col?.orderBy).toEqual([]); // default = PK asc, applied at emit + expect(col?.joinedPkColumn).toBe("id"); +``` + +- [ ] **Step 3: Run to verify it fails** + +```bash +cd server/typescript/packages/codegen-ts && bun test test/projection/collect-whole-object-spec.test.ts +``` + +Expected: FAIL — the column is absent entirely, because `if (!of_) continue;` skips it. + +- [ ] **Step 4: Restructure the branch** + +In `extract-view-spec.ts`, the `if (!of_) continue;` at ~`:891` sits **above** the `AGG_COLLECT` branch and must no longer swallow the object form. Add the object arm **before** that guard, mirroring the `any`/`all` branch at `:861-885`: + +```ts + // #335 — @of ABSENT on collect is a whole-object rollup. Resolve the related + // entity from @via's terminal hop (not from @of), exactly as any/all does. + if (agg === AGG_COLLECT && !of_) { + const via = origin.ownAttr(ORIGIN_AGGREGATE_ATTR_VIA) as string | undefined; + if (!via) continue; // loader already errored + const objectRef = field.attr(FIELD_ATTR_OBJECT_REF) as string | undefined; + if (!objectRef) continue; // loader already errored + const relatedName = viaTerminalEntity(via, root, projPkg); + if (!relatedName) continue; + const relatedEntity = resolveEntityRef(root, relatedName, projPkg); + const sourceAlias = findAliasInTree(joinTree, relatedName); + if (!relatedEntity || sourceAlias === undefined) continue; + const joinedPk = primaryKeyColumn(relatedEntity, ctx); + if (joinedPk === undefined) continue; + const vo = resolveObjectRef(root, objectRef, projPkg).node; + if (!vo) continue; + // children() — a value object may inherit members via extends. + const members = vo.children() + .filter((c): c is MetaField => c.type === TYPE_FIELD) + .map((m) => { + const target = relatedEntity.fields().find((f) => f.name === m.name); + // The loader guarantees every member resolves (ERR_COLLECT_MEMBER_UNRESOLVED). + return target === undefined + ? undefined + : { memberName: m.name, sourceColumn: sourceColumnNameFor(target, ctx) }; + }) + .filter((m): m is { memberName: string; sourceColumn: string } => m !== undefined); + if (members.length === 0) continue; + columns.push({ + kind: "collectObjectAgg", + fieldName: field.name, + dbColAlias: dbCol, + sourceAlias, + joinedPkColumn: joinedPk, + members, + orderBy: resolveOrderByKeys(origin.ownAttr(ORIGIN_ATTR_ORDER_BY), relatedEntity, ctx), + }); + continue; + } +``` + +Then update `:1005` and `:1021` to treat `collectObjectAgg` the same way they treat `collectAgg` (deterministic ordering; a real aggregate). + +- [ ] **Step 5: Run the test to verify it passes** + +```bash +cd server/typescript/packages/codegen-ts && bun test test/projection/collect-whole-object-spec.test.ts +``` + +Expected: PASS. + +- [ ] **Step 6: Commit** + +```bash +git add server/typescript/packages/codegen-ts/src/projection/view-spec.ts \ + server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts \ + server/typescript/packages/codegen-ts/test/projection/collect-whole-object-spec.test.ts +git commit -m "feat(codegen-ts): extract a collectObjectAgg column from an @of-less collect (#335)" +``` + +--- + +## Task 11: Half A — SQL lowering, both dialects + +**Files:** +- Modify: `server/typescript/packages/codegen-ts/src/projection/view-ddl-emit.ts` +- Test: `server/typescript/packages/codegen-ts/test/projection/collect-whole-object-ddl.test.ts` (create) + +**Interfaces:** +- Consumes: Task 10's `collectObjectAgg`. +- Produces: the emitted SQL Task 13 applies to real engines. + +**Why `jsonb`, not `json`:** verified against PostgreSQL 15.15 — `json` has neither an equality nor an ordering operator (`ERROR: operator does not exist: json = json`; `could not identify an ordering operator for type json`), so `json_agg(json_build_object(…) ORDER BY …)` **does not run**. `field.object` is already `jsonb` in `column-mapper.ts:610-613`. + +- [ ] **Step 1: Write the failing test** + +```ts + test("postgres: jsonb_agg of jsonb_build_object, PK-asc, empty-set guarded", () => { + expect(emitColumn(col, "postgres")).toBe( + `COALESCE(jsonb_agg(jsonb_build_object('id', s."id", 'name', s."name") ORDER BY s."id" ASC) ` + + `FILTER (WHERE s."id" IS NOT NULL), '[]'::jsonb) AS "supplierBriefs"`, + ); + }); + + test("sqlite: json_group_array of json_object, PK-asc, empty-set guarded", () => { + expect(emitColumn(col, "sqlite")).toBe( + `COALESCE(json_group_array(json_object('id', s."id", 'name', s."name") ORDER BY s."id" ASC) ` + + `FILTER (WHERE s."id" IS NOT NULL), json_array()) AS "supplierBriefs"`, + ); + }); +``` + +Match `emitColumn`'s real name and signature from the neighbouring tests in that directory. + +- [ ] **Step 2: Run to verify it fails** + +```bash +cd server/typescript/packages/codegen-ts && bun test test/projection/collect-whole-object-ddl.test.ts +``` + +Expected: FAIL — no branch handles the kind. + +- [ ] **Step 3: Implement the lowering** + +In `view-ddl-emit.ts`, directly after the `collectAgg` branch at `:187`: + +```ts + if (c.kind === "collectObjectAgg") { + const guard = `${c.sourceAlias}.${quoteIfNeeded(c.joinedPkColumn)} IS NOT NULL`; + // Element order: the related entity's PK ascending by default — "value + // ascending" is meaningless for an object and does not even parse on PG json. + // An explicit @orderBy leads, with the PK appended as tie-break so equal-order + // rows stay byte-deterministic (mirrors renderFirst at :131-137). The SCALAR + // arm deliberately keeps its no-tie-break behaviour: changing it would alter + // emitted SQL for every existing project using @orderBy. + const pk = `${c.sourceAlias}.${quoteIfNeeded(c.joinedPkColumn)}`; + const orderClause = c.orderBy.length > 0 + ? `ORDER BY ${renderOrderKeys(c.orderBy, c.sourceAlias)}, ${pk} ASC` + : `ORDER BY ${pk} ASC`; + const pairs = c.members + .map((m) => `'${m.memberName}', ${c.sourceAlias}.${quoteIfNeeded(m.sourceColumn)}`) + .join(", "); + if (dialect === "sqlite") { + return `COALESCE(json_group_array(json_object(${pairs}) ${orderClause}) FILTER (WHERE ${guard}), json_array()) AS ${alias}`; + } + return `COALESCE(jsonb_agg(jsonb_build_object(${pairs}) ${orderClause}) FILTER (WHERE ${guard}), '[]'::jsonb) AS ${alias}`; + } +``` + +In-aggregate `ORDER BY` needs SQLite **≥ 3.44**. Not a new constraint: the existing scalar `collect` already emits it and D1's baseline is pinned at `3.44.0` (`introspect/d1.ts:44`). + +- [ ] **Step 4: Run to verify it passes** + +```bash +cd server/typescript/packages/codegen-ts && bun test test/projection/collect-whole-object-ddl.test.ts +``` + +Expected: 2 pass. + +- [ ] **Step 5: Add the scalar no-churn pin** + +In the same file, add a test asserting a `collect` **with** `@of` still emits `array_agg(… ORDER BY … ) FILTER (…) , '{}'` on Postgres and `json_group_array` / `json_array()` on SQLite, byte-for-byte. The two arms share a branch neighbourhood; this is what proves the scalar one did not move. + +- [ ] **Step 6: Run the whole codegen-ts suite** + +```bash +cd server/typescript/packages/codegen-ts && bun test +``` + +Expected: 0 fail. If `issue-214-read-half-compile.test.ts` fails with *"Cannot find module `@metaobjectsdev/runtime-ts/drizzle-fastify`"*, that is a fresh-worktree precondition, not your change — run `bun run --filter '@metaobjectsdev/runtime-ts' build` from the repo root first. + +- [ ] **Step 7: Commit** + +```bash +git add server/typescript/packages/codegen-ts/src/projection/view-ddl-emit.ts \ + server/typescript/packages/codegen-ts/test/projection/collect-whole-object-ddl.test.ts +git commit -m "feat(codegen-ts): lower a whole-object rollup to jsonb_agg / json_group_array (#335)" +``` + +--- + +## Task 12: Half A — real-engine round-trip, both dialects + +**Files:** +- Modify: `server/typescript/packages/integration-tests/test/view-lifecycle-pg.test.ts` +- Modify: `server/typescript/packages/integration-tests/test/view-lifecycle-sqlite.test.ts` + +**Interfaces:** +- Consumes: Tasks 10–11. +- Produces: the only evidence that counts for new DDL. + +**Why:** Golden SQL is not acceptable evidence for new DDL — that is exactly how the migrate defect class got through before. These files live in a **separate package** that only the `ts-slow` lane runs; they will not appear in a `codegen-ts` or `migrate-ts` run. + +- [ ] **Step 1: Add the projection to both fixtures** + +Extend the existing model in each file with `SupplierBrief`, a `Supplier` child table, and a `field.object isArray @objectRef` carrying an `@of`-less collect — beside the existing `weekLabels` scalar collect, so both arms are exercised in one view. + +- [ ] **Step 2: Assert emit → apply → introspect → re-diff is EMPTY** + +Follow the existing round-trip in each file exactly. The re-diff must produce **no** operations; a non-empty second diff means the emitted SQL and the introspected schema disagree, which is permanent false drift on every subsequent `meta migrate`. + +- [ ] **Step 3: Assert the value shape, including the empty set** + +```ts + // two related rows → array of objects, PK-ascending + expect(JSON.parse(full.supplierBriefs)).toEqual([ + { id: 1, name: "Acme" }, + { id: 2, name: "Globex" }, + ]); + // zero related rows → [] and NOT null (the FILTER guard) + expect(JSON.parse(empty.supplierBriefs)).toEqual([]); +``` + +On Postgres the driver may return a parsed object rather than a string; match whatever the neighbouring `weekLabels` assertions do in that same file. + +- [ ] **Step 4: Run both** + +```bash +cd server/typescript/packages/integration-tests && bun test test/view-lifecycle-sqlite.test.ts +cd server/typescript/packages/integration-tests && bun test test/view-lifecycle-pg.test.ts +``` + +Postgres needs Docker for Testcontainers. If the container times out, re-run on a quiet box before treating it as a failure — that timeout fires before test logic and means load, not regression. **Never** point these at an existing local Postgres; print the resolved URL first and confirm it is the Testcontainers one. + +- [ ] **Step 5: Commit** + +```bash +git add server/typescript/packages/integration-tests/test/view-lifecycle-pg.test.ts \ + server/typescript/packages/integration-tests/test/view-lifecycle-sqlite.test.ts +git commit -m "test(integration): round-trip the whole-object rollup on real PG and SQLite (#335)" +``` + +--- + +## Task 13: Half A — restore the corpus coverage the retirement lost + +**Files:** +- Modify: `fixtures/conformance/flattened-kitchen-sink/input/meta.catalog.json` +- Modify: `fixtures/conformance/flattened-kitchen-sink/expected.json` +- Modify: `fixtures/conformance/flattened-kitchen-sink/expected-effective.json` +- Modify: `fixtures/conformance/README.md` +- Modify: `docs/features/migrations/origin-collection-retirement.md` + +**Interfaces:** +- Consumes: Tasks 5–8. +- Produces: closes the documented coverage gap. + +**Why:** `fixtures/conformance/README.md:102` records this as *"coverage genuinely lost"* — `flattened-kitchen-sink` dropped `supplierBriefs`, the corpus's only array-of-value-object carrying an origin — and names #335 as the restore path. + +- [ ] **Step 1: Restore `supplierBriefs`, expressed the new way** + +```jsonc +{ "field.object": { + "name": "supplierBriefs", "isArray": true, + "@objectRef": "acme::catalog::SupplierBrief", + "children": [ + { "origin.aggregate": { "@agg": "collect", "@via": "acme::catalog::Product.suppliers" } } + ] +}} +``` + +- [ ] **Step 2: Regenerate both goldens** + +Run the corpus and copy the canonical output from the mismatch report. Do not hand-edit. + +- [ ] **Step 3: Update the README's coverage table** + +The "Coverage genuinely lost" entry is no longer true. Replace it with a line saying the shape is expressible again via `@of`-less collect and naming this fixture as where it is covered. + +- [ ] **Step 4: Fix the retirement guide's silent case** + +`docs/features/migrations/origin-collection-retirement.md:62-65` tells authors to *"delete the `origin.collection` child and change nothing else"*. That is correct for a **payload** host but wrong for a **view-kind projection**, where a no-origin field lowers to `SELECT base."supplierBriefs"` against a column that does not exist (`extract-view-spec.ts:817-826`). Add a short subsection: on a projection host, replace the child with `origin.aggregate @agg: collect @via: …` rather than deleting it. + +- [ ] **Step 5: Run the corpus in every port** + +```bash +cd server/typescript/packages/metadata && bun test test/conformance.test.ts +cd server/python && uv run pytest --extra integration -k conformance +cd server/csharp && dotnet build && dotnet test --filter "FullyQualifiedName~Conformance" +cd server/java && mvn -q -pl metadata test +``` + +Expected: all green. + +- [ ] **Step 6: Commit** + +```bash +git add fixtures/conformance/flattened-kitchen-sink fixtures/conformance/README.md \ + docs/features/migrations/origin-collection-retirement.md +git commit -m "test(conformance): restore the array-of-value-object origin coverage (#335)" +``` + +--- + +## Task 14: Documentation and changelog + +**Files:** +- Modify: `CHANGELOG.md` +- Modify: `docs/features/downstream-metadata-decisions.md` +- Modify: `agent-context/skills/metaobjects-authoring/SKILL.md` **and its 5 byte-gated copies** under `fixtures/agent-context-conformance/*/expected/.claude/skills/metaobjects-authoring/SKILL.md` + +**Interfaces:** +- Consumes: everything. +- Produces: the adopter-facing record. + +**Why the skill matters:** the skill is what teaches adopters the vocabulary. #342 shipped with the skill teaching a form the loader had just made illegal — do not repeat that in the other direction by leaving the skill unable to express the new one. + +- [ ] **Step 1: Add the changelog entry** + +Cover, in this order: the whole-object rollup with a worked example; **Half B as potentially breaking**, stated with its bounds — in-repo cost measured at zero, what breaks was already emitting SQL that cannot execute, and no documentation ever taught the form; the `@distinct` refusal as a deliberate choice rather than an engine limit; and the `metamodelVersion` position. + +- [ ] **Step 2: Add the authoring-skill section** + +Document `@of`-less collect beside the existing scalar `collect`: the `field.object isArray @objectRef` requirement, `@via` required, `@objectRef` must be an `object.value`, members must exist on the terminal entity with matching types, `@distinct` unsupported, default order PK-ascending. + +- [ ] **Step 3: Regenerate the byte-gated skill copies** + +```bash +grep -rn "regenerate" fixtures/agent-context-conformance/README.md +``` + +Run the generator named there. Do not hand-edit the five copies. + +- [ ] **Step 4: Verify the agent-context corpus** + +```bash +cd server/typescript/packages/cli && bun test +``` + +Expected: `agent-context-conformance` green. + +- [ ] **Step 5: Commit** + +```bash +git add CHANGELOG.md docs/features/downstream-metadata-decisions.md \ + agent-context/skills/metaobjects-authoring/SKILL.md \ + fixtures/agent-context-conformance +git commit -m "docs: whole-object rollup and the array filter/sort rules (#335)" +``` + +--- + +## Task 15: Full-gate verification before merge + +- [ ] **Step 1: Clean-tree check** + +```bash +git clean -ndX # review what would be removed +git clean -fdX && bun install +``` + +The dev box is warm; a lane change validated on a warm tree proves nothing. + +- [ ] **Step 2: Run the affected ports** + +```bash +bash scripts/ci-local.sh --only ts +bash scripts/ci-local.sh --only python +bash scripts/ci-local.sh --only csharp +bash scripts/ci-local.sh --only java +``` + +**Never pipe these through `tail`** — the exit status becomes `tail`'s and a red run reports green. Read the trailing `OK —` / `FAILED —` line directly. + +- [ ] **Step 3: Confirm the scalar arm never moved** + +```bash +git diff v0.24.0 --stat -- server/typescript/packages/codegen-ts/test/golden/ +``` + +Expected: no churn attributable to the scalar `collect` path. Any golden that moved must be explained before merge. + +- [ ] **Step 4: Leak scan** + +```bash +bash scripts/ci-local.sh --quick +``` + +- [ ] **Step 5: Re-check the version question** + +```bash +git tag --list 'v0.*' --sort=-v:refname | head -1 +node scripts/check-metamodel-version.mjs --explain +``` + +If `v0.24.1` shipped while this was in flight, `--set 0.12` and commit before merging. + +- [ ] **Step 6: Coordinate before pushing** + +Another session owns `validation-passes.ts` for #342 and has committed there. `git fetch origin` and rebase before pushing; message that session before landing. + +--- + +## Self-review + +**Spec coverage.** A1 → Tasks 5, 8. A2 → Task 6. A3 → Task 10. A4 → Task 11. A5 → Tasks 5 (`@distinct`), 11 (order). A6 → Task 9. A7 → Tasks 9 Step 5, 15 Step 5. B1 → Tasks 1, 4. B2 → Tasks 2, 4. B3 → Task 14 Step 1. Testing section → Tasks 3, 7, 12, 13. Non-goals are not implemented, by design. + +**Placeholders.** None. Every code step carries real code; every command is runnable. + +**Type consistency.** `collectObjectAgg` and its `members: { memberName, sourceColumn }[]` are defined in Task 10 and consumed unchanged in Tasks 11–12. `_viaTerminalEntityNode` is introduced in Task 5 and reused in Task 6. `ERR_COLLECT_MEMBER_UNRESOLVED` and `ERR_SORTABLE_UNSUPPORTED_SUBTYPE` are added to all five ledgers in the tasks that first emit them. + +**Known soft spots**, flagged rather than hidden: the loader's string entry point is written as `loadFromString`, and the DDL test helper as `emitColumn` — both must be matched to the real names in neighbouring tests. Task 9's exact seven files must come from the grep, not from the list, in case a copy moved. From 8a5c1b3cd443349fa7c24d711b3ba1664ab0890a Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 23 Aug 2026 08:24:41 -0400 Subject: [PATCH 06/25] fix(metamodel): an array field has no filter-operator band either (#335) --- .../metadata/src/loader/validation-passes.ts | 20 ++++++ .../test/validation-filterable-array.test.ts | 65 +++++++++++++++++++ 2 files changed, 85 insertions(+) create mode 100644 server/typescript/packages/metadata/test/validation-filterable-array.test.ts diff --git a/server/typescript/packages/metadata/src/loader/validation-passes.ts b/server/typescript/packages/metadata/src/loader/validation-passes.ts index aced951e7..68a07fd6e 100644 --- a/server/typescript/packages/metadata/src/loader/validation-passes.ts +++ b/server/typescript/packages/metadata/src/loader/validation-passes.ts @@ -443,6 +443,26 @@ export function validateFilterableHasSupportedOps(root: MetaData): ParseError[] for (const field of obj.children().filter((c) => c.type === TYPE_FIELD)) { // ADR-0039: resolving — a concrete field may inherit @filterable via extends. if (field.attr(FIELD_ATTR_FILTERABLE) !== true) continue; + + // #335 Half B — an ARRAY field has no operator band either. Every FR-009 + // operator (eq/ne/gt/gte/lt/lte/in/like/isNull) is a scalar comparison; + // none applies to a collection column. The allowlist template does not + // consult isArray and falls through to the "string" band, so this + // previously emitted a `like` rule against a text[] column — SQL that + // cannot execute. Same reason as the subtype check below, so same code. + // ADR-0039: resolvedIsArray(), never the own `isArray` flag. + if (field.resolvedIsArray()) { + errors.push( + new ParseError( + `Field "${obj.name}.${field.name}" has @filterable: true but is an array ` + + `(isArray: true). No filter operator applies to a collection column. ` + + `Remove @filterable from this field.`, + { code: "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE", source: field.source }, + ), + ); + continue; + } + if (opsForSubType(field.subType).length > 0) continue; errors.push( new ParseError( diff --git a/server/typescript/packages/metadata/test/validation-filterable-array.test.ts b/server/typescript/packages/metadata/test/validation-filterable-array.test.ts new file mode 100644 index 000000000..f78814276 --- /dev/null +++ b/server/typescript/packages/metadata/test/validation-filterable-array.test.ts @@ -0,0 +1,65 @@ +// #335 Half B — an array field has no filter-operator band either. +// +// filterSubTypeFor (codegen-ts/src/templates/filter-allowlist.ts) falls +// through to "string" for anything unrecognised and never consults isArray, +// so a `field.string isArray: true @filterable: true` previously emitted a +// like/eq rule against a text[] column — SQL that cannot execute. No FR-009 +// operator applies to a collection column, the same reason field.object is +// already rejected, so this reuses ERR_FILTERABLE_UNSUPPORTED_SUBTYPE. +// +// Uses the same MetaDataLoader().load([new InMemoryStringSource(...)]) API +// as the neighbouring filterable-attrs.test.ts — MetaDataLoader has no +// throwing loadFromString entry point; load() collects errors on the +// returned LoadResult instead of throwing. + +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader } from "../src/loader/meta-data-loader.js"; +import { InMemoryStringSource } from "../src/loader/meta-data-source.js"; + +const model = (fieldJson: string) => `{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { + "name": "Product", + "children": [ + { "source.rdb": { "@kind": "table", "@table": "products" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "@fields": ["id"] } }, + ${fieldJson} + ] + }} + ] + } +}`; + +async function loadErrors(src: string) { + const result = await new MetaDataLoader().load([ + new InMemoryStringSource(src, { id: "meta.demo.json" }), + ]); + return result.errors as unknown as { code: string; message: string }[]; +} + +describe("@filterable on an array field", () => { + test("an array field marked @filterable fails to load", async () => { + const src = model( + `{ "field.string": { "name": "tags", "isArray": true, "@filterable": true } }`, + ); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE"); + expect(hit).toBeDefined(); + expect(hit?.message).toContain("Product.tags"); + }); + + test("the same field WITHOUT isArray still loads", async () => { + const src = model(`{ "field.string": { "name": "tags", "@filterable": true } }`); + const errors = await loadErrors(src); + expect(errors).toEqual([]); + }); + + test("an array field NOT marked @filterable still loads", async () => { + const src = model(`{ "field.string": { "name": "tags", "isArray": true } }`); + const errors = await loadErrors(src); + expect(errors).toEqual([]); + }); +}); From 32be08653a46f5a35501cfc0685bed1a2714e2ba Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 23 Aug 2026 08:34:52 -0400 Subject: [PATCH 07/25] fix(metamodel): @sortable gets the subtype validation @filterable already had (#335) --- fixtures/conformance/ERROR-CODES.json | 1 + server/csharp/MetaObjects/Errors.cs | 4 + .../main/java/com/metaobjects/ErrorCode.java | 8 ++ server/python/src/metaobjects/errors.py | 4 + .../packages/metadata/src/errors.ts | 4 + .../metadata/src/loader/meta-data-loader.ts | 6 +- .../metadata/src/loader/validation-passes.ts | 35 +++++++++ .../test/validation-sortable-array.test.ts | 76 +++++++++++++++++++ 8 files changed, 137 insertions(+), 1 deletion(-) create mode 100644 server/typescript/packages/metadata/test/validation-sortable-array.test.ts diff --git a/fixtures/conformance/ERROR-CODES.json b/fixtures/conformance/ERROR-CODES.json index 2ec7f79c9..a259282a8 100644 --- a/fixtures/conformance/ERROR-CODES.json +++ b/fixtures/conformance/ERROR-CODES.json @@ -70,6 +70,7 @@ "ERR_YAML_COERCION": "A YAML 1.2 silent type coercion produced a value whose runtime type differs from the attribute's declared valueType (e.g. an unquoted `column: TRUE` parsed as boolean for a string-typed attr). Emitted by every port's YAML loader; canonical JSON is unaffected.", "ERR_MERGE_CONFLICT": "FR5c: multi-file overlay merge produced an attribute conflict \u2014 two contributing files declared the same @attr with different non-empty values on the same logical node. Envelope is `format: \"merged\"` with `contributors[]` naming both files.", "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE": "SP-H Unit9: a field carries @filterable: true but its subtype has no filter-operator band (e.g. field.object). Generating a filter for it would emit an empty operator set \u2014 a filter route that rejects every request. Use a filterable subtype (string/enum/uuid/number/currency/date/boolean) or remove @filterable.", + "ERR_SORTABLE_UNSUPPORTED_SUBTYPE": "#335 Half B: a field carries @sortable: true but cannot be ordered \u2014 either it is an array (isArray: true; a collection column has no ordering) or its subtype has no filter-operator band (e.g. field.object), the same subtype-support signal @filterable uses (SP-H Unit9). Generating a sort entry for it would emit an ORDER BY no dialect can execute. Use a sortable subtype (string/enum/uuid/number/currency/date/boolean) or remove @sortable.", "ERR_REGISTRY_SEALED": "ADR-0023: a registration (register / registerType / extend / registerCommonAttrs / setDefaultSubType / addConstraint / registerProviders) was attempted against a registry sealed after its agreed metamodel-provider bootstrap. Made-up metamodel attributes/types are structurally disallowed in the library: codegen cannot register post-bootstrap. A new metamodel attribute requires a registered provider + human agreement. Downstream apps that need extra vocabulary compose their own (unsealed) registry.", "ERR_UNKNOWN": "An internal loader error with no stable error code.", "ERR_ENTITY_PRIMARY_SOURCE_READONLY": "FR-024 (ADR-0028) hard cutover: an object.entity's PRIMARY source has a read-only @kind (view/materializedView/storedProc/tableFunction). Read-only kinds are legal only in non-primary roles; a derived read model is an object.projection.", diff --git a/server/csharp/MetaObjects/Errors.cs b/server/csharp/MetaObjects/Errors.cs index b804612d3..1567d33bf 100644 --- a/server/csharp/MetaObjects/Errors.cs +++ b/server/csharp/MetaObjects/Errors.cs @@ -173,6 +173,10 @@ public enum ErrorCode // SP-H Unit9 — @filterable: true on a field subtype with no filter-operator // band (e.g. field.object). Would silently generate an empty-ops filter. ERR_FILTERABLE_UNSUPPORTED_SUBTYPE, + // #335 Half B — @sortable: true on an array field or a subtype with no + // filter-operator band (e.g. field.object). Would silently emit a sort + // entry over a column no dialect can ORDER BY. + ERR_SORTABLE_UNSUPPORTED_SUBTYPE, // Index-key resolution for index.lookup AND identity.secondary (#342) — the key is // @fields XOR @expr: neither declared, BOTH declared (@expr is used INSTEAD of // @fields), whichever is declared supplies no key, or a named field does not exist diff --git a/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java b/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java index 7fe2cb0cb..4b687dcfd 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java +++ b/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java @@ -366,6 +366,14 @@ public enum ErrorCode { */ ERR_FILTERABLE_UNSUPPORTED_SUBTYPE, + /** + * #335 Half B — a field carries {@code @sortable: true} but is an array or + * its subtype has no filter-operator band (e.g. {@code field.object}), the + * same subtype-support signal {@code @filterable} uses. Generating a sort + * entry for it would emit an ORDER BY no dialect can execute. + */ + ERR_SORTABLE_UNSUPPORTED_SUBTYPE, + /** * ADR-0023 Decision 2: a registration ({@code register}/{@code extendType}/ * {@code registerCommonAttribute}/{@code addConstraint}/{@code registerType}/ diff --git a/server/python/src/metaobjects/errors.py b/server/python/src/metaobjects/errors.py index 10d88774a..61700192b 100644 --- a/server/python/src/metaobjects/errors.py +++ b/server/python/src/metaobjects/errors.py @@ -156,6 +156,10 @@ class ErrorCode(str, Enum): # SP-H Unit9 — @filterable: true on a field subtype with no filter-operator # band (e.g. field.object). Would silently generate an empty-ops filter. ERR_FILTERABLE_UNSUPPORTED_SUBTYPE = "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE" + # #335 Half B — @sortable: true on an array field or a subtype with no + # filter-operator band (e.g. field.object). Would silently emit a sort + # entry over a column no dialect can ORDER BY. + ERR_SORTABLE_UNSUPPORTED_SUBTYPE = "ERR_SORTABLE_UNSUPPORTED_SUBTYPE" # ADR-0023 — a registration was attempted against a registry sealed after its # agreed metamodel-provider bootstrap. Codegen cannot invent metamodel attrs. ERR_REGISTRY_SEALED = "ERR_REGISTRY_SEALED" diff --git a/server/typescript/packages/metadata/src/errors.ts b/server/typescript/packages/metadata/src/errors.ts index 15a5a099b..78858091f 100644 --- a/server/typescript/packages/metadata/src/errors.ts +++ b/server/typescript/packages/metadata/src/errors.ts @@ -159,6 +159,10 @@ export const ERROR_CODES = [ // SP-H Unit9 — @filterable: true on a field subtype with no filter-operator // band (e.g. field.object). Would silently generate an empty-ops filter. "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE", + // #335 Half B — @sortable: true on an array field or a subtype with no + // filter-operator band (e.g. field.object). Would silently emit a sort + // entry over a column no dialect can ORDER BY. + "ERR_SORTABLE_UNSUPPORTED_SUBTYPE", // ADR-0023 — a registration was attempted against a registry sealed after its // agreed metamodel-provider bootstrap. Codegen cannot invent metamodel attrs. "ERR_REGISTRY_SEALED", diff --git a/server/typescript/packages/metadata/src/loader/meta-data-loader.ts b/server/typescript/packages/metadata/src/loader/meta-data-loader.ts index e91cefd7a..386cacbf1 100644 --- a/server/typescript/packages/metadata/src/loader/meta-data-loader.ts +++ b/server/typescript/packages/metadata/src/loader/meta-data-loader.ts @@ -18,7 +18,7 @@ import { ParseError } from "../errors.js"; import type { LoaderWarning } from "../source.js"; import { codeSource, resolvedSource } from "../source.js"; import { parseJson } from "../parser-json.js"; -import { validateDataGridSortFields, validateFilterableHasIndex, validateFilterableHasSupportedOps, validateOriginPaths, validateDerivedFieldProvidability, validateDataGridFilterValues, validateFieldObjectStorage, validateFieldMap, validateTemplatePayloadRefs, validateFieldDefaults, validateRelationships, validateIndexLookupFields, validateProjectionFilter } from "./validation-passes.js"; +import { validateDataGridSortFields, validateFilterableHasIndex, validateFilterableHasSupportedOps, validateSortableHasSupportedSubtype, validateOriginPaths, validateDerivedFieldProvidability, validateDataGridFilterValues, validateFieldObjectStorage, validateFieldMap, validateTemplatePayloadRefs, validateFieldDefaults, validateRelationships, validateIndexLookupFields, validateProjectionFilter } from "./validation-passes.js"; import { runRegisteredValidation } from "./validation-registry.js"; import { validateSourceRoles } from "../persistence/source/validate-source-roles.js"; import { validateSourceEscapes } from "../persistence/source/validate-source-escapes.js"; @@ -578,6 +578,10 @@ export class MetaDataLoader { // (would silently generate a filter that rejects every request). errors.push(...validateFilterableHasSupportedOps(root)); + // #335 Half B — @sortable on an array field or a subtype with no operator + // band → error (would silently emit a sort entry no dialect can execute). + errors.push(...validateSortableHasSupportedSubtype(root)); + // Sixth pass: origin path validation — validates passthrough.@from, // aggregate.@of, and .@via relationship chains. errors.push(...validateOriginPaths(root)); diff --git a/server/typescript/packages/metadata/src/loader/validation-passes.ts b/server/typescript/packages/metadata/src/loader/validation-passes.ts index 68a07fd6e..3d54b335d 100644 --- a/server/typescript/packages/metadata/src/loader/validation-passes.ts +++ b/server/typescript/packages/metadata/src/loader/validation-passes.ts @@ -61,6 +61,7 @@ import { } from "../presentation/layout/layout-constants.js"; import { FIELD_ATTR_FILTERABLE, + FIELD_ATTR_SORTABLE, FIELD_ATTR_OBJECT_REF, FIELD_ATTR_STORAGE, STORAGE_FLATTENED, @@ -477,6 +478,40 @@ export function validateFilterableHasSupportedOps(root: MetaData): ParseError[] return errors; } +// --------------------------------------------------------------------------- +// @sortable on a subtype or shape that cannot be ordered (#335 Half B) +// --------------------------------------------------------------------------- +// @sortable defaults FROM @filterable, so it is independently set only when +// explicit — and nothing validated it, while @filterable has had a hard error +// since SP-H Unit9. A @sortable JSON or array column emits a sort entry over a +// column no dialect can ORDER BY meaningfully. → ERR_SORTABLE_UNSUPPORTED_SUBTYPE. + +export function validateSortableHasSupportedSubtype(root: MetaData): ParseError[] { + const errors: ParseError[] = []; + // ADR-0039: root has no super; children()==ownChildren() but resolving is the default. + for (const obj of root.children().filter((c) => c.type === TYPE_OBJECT)) { + // children() — inherited @sortable fields (via extends:/super:) are visible. + for (const field of obj.children().filter((c) => c.type === TYPE_FIELD)) { + // ADR-0039: resolving — a concrete field may inherit @sortable via extends. + if (field.attr(FIELD_ATTR_SORTABLE) !== true) continue; + // ADR-0039: resolvedIsArray(), never the own `isArray` flag. + const isArray = field.resolvedIsArray(); + if (!isArray && opsForSubType(field.subType).length > 0) continue; + errors.push( + new ParseError( + `Field "${obj.name}.${field.name}" has @sortable: true but ` + + (isArray + ? `is an array (isArray: true) — a collection column has no ordering.` + : `its subtype "${field.subType}" cannot be ordered.`) + + ` Remove @sortable from this field.`, + { code: "ERR_SORTABLE_UNSUPPORTED_SUBTYPE", source: field.source }, + ), + ); + } + } + return errors; +} + // --------------------------------------------------------------------------- // Origin path validation // diff --git a/server/typescript/packages/metadata/test/validation-sortable-array.test.ts b/server/typescript/packages/metadata/test/validation-sortable-array.test.ts new file mode 100644 index 000000000..aaeac5648 --- /dev/null +++ b/server/typescript/packages/metadata/test/validation-sortable-array.test.ts @@ -0,0 +1,76 @@ +// #335 Half B — @sortable gets the same subtype/array validation @filterable +// already had. +// +// @sortable defaults FROM @filterable, so it is independently set only when +// explicit — and nothing validated it: a @sortable JSON or array column +// passed the loader and emitted a sort entry over a column no dialect can +// ORDER BY meaningfully. +// +// Uses the same MetaDataLoader().load([new InMemoryStringSource(...)]) API +// as the neighbouring validation-filterable-array.test.ts — MetaDataLoader +// has no throwing loadFromString entry point; load() collects errors on the +// returned LoadResult instead of throwing. + +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader } from "../src/loader/meta-data-loader.js"; +import { InMemoryStringSource } from "../src/loader/meta-data-source.js"; + +const model = (fieldJson: string) => `{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { + "name": "Spec", + "children": [ + { "source.rdb": { "@kind": "table", "@table": "specs" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "@fields": ["id"] } } + ] + }}, + { "object.entity": { + "name": "Product", + "children": [ + { "source.rdb": { "@kind": "table", "@table": "products" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "@fields": ["id"] } }, + ${fieldJson} + ] + }} + ] + } +}`; + +async function loadErrors(src: string) { + const result = await new MetaDataLoader().load([ + new InMemoryStringSource(src, { id: "meta.demo.json" }), + ]); + return result.errors as unknown as { code: string; message: string }[]; +} + +describe("@sortable subtype validation", () => { + test("an array field marked @sortable fails to load", async () => { + const src = model( + `{ "field.string": { "name": "tags", "isArray": true, "@sortable": true } }`, + ); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_SORTABLE_UNSUPPORTED_SUBTYPE"); + expect(hit).toBeDefined(); + expect(hit?.message).toContain("Product.tags"); + }); + + test("a field.object marked @sortable fails to load", async () => { + const src = model( + `{ "field.object": { "name": "spec", "@objectRef": "acme::shop::Spec", "@storage": "jsonb", "@sortable": true } }`, + ); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_SORTABLE_UNSUPPORTED_SUBTYPE"); + expect(hit).toBeDefined(); + expect(hit?.message).toContain("Product.spec"); + }); + + test("a plain scalar marked @sortable still loads", async () => { + const src = model(`{ "field.string": { "name": "sku", "@sortable": true } }`); + const errors = await loadErrors(src); + expect(errors).toEqual([]); + }); +}); From 98966cb56e6a94f7bb0f276894fe1b59014d058d Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 23 Aug 2026 08:42:45 -0400 Subject: [PATCH 08/25] test(conformance): gate the array filter/sort rules cross-port (#335) --- fixtures/conformance/README.md | 16 ++++++++++++++++ .../expected-errors.json | 13 +++++++++++++ .../input/meta.demo.json | 18 ++++++++++++++++++ .../expected-errors.json | 13 +++++++++++++ .../input/meta.demo.json | 18 ++++++++++++++++++ 5 files changed, 78 insertions(+) create mode 100644 fixtures/conformance/error-filterable-array-field/expected-errors.json create mode 100644 fixtures/conformance/error-filterable-array-field/input/meta.demo.json create mode 100644 fixtures/conformance/error-sortable-array-field/expected-errors.json create mode 100644 fixtures/conformance/error-sortable-array-field/input/meta.demo.json diff --git a/fixtures/conformance/README.md b/fixtures/conformance/README.md index 0f738b714..d27e12532 100644 --- a/fixtures/conformance/README.md +++ b/fixtures/conformance/README.md @@ -80,6 +80,22 @@ API-surface checks; a fixture may have both. Fixtures assert error *codes*, never message prose. Codes are registered in `ERROR-CODES.json`. Adding a code is an additive edit to that file. +## Array fields through the filter/sort tier (#335) + +- `error-filterable-array-field` — a `field.string @isArray @filterable` fails + `ERR_FILTERABLE_UNSUPPORTED_SUBTYPE`: no FR-009 operator (eq/ne/gt/gte/lt/lte/in/like/isNull) + applies to a collection column. +- `error-sortable-array-field` — the same shape with `@sortable` in place of + `@filterable` fails `ERR_SORTABLE_UNSUPPORTED_SUBTYPE`: a collection column has + no ordering. + +A structural scan of the whole repo (1321 JSON + 124 YAML files) found **zero** +fields carrying both `isArray: true` and `@filterable`/`@sortable: true` before +these two fixtures were added — they are the **only** cases in the corpus that +exercise an array field through the filter/sort tier, so they are what makes the +rule (added to the TypeScript loader for #335) a cross-port contract rather than +a TS-only unit test. + ## Adding a fixture Create a directory; add `input/` and expectation files. No runner code changes — diff --git a/fixtures/conformance/error-filterable-array-field/expected-errors.json b/fixtures/conformance/error-filterable-array-field/expected-errors.json new file mode 100644 index 000000000..0bb23ab3d --- /dev/null +++ b/fixtures/conformance/error-filterable-array-field/expected-errors.json @@ -0,0 +1,13 @@ +{ + "errors": [ + { + "code": "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE", + "source": { + "format": "json", + "files": ["meta.demo.json"], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[3]['field.string']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-filterable-array-field/input/meta.demo.json b/fixtures/conformance/error-filterable-array-field/input/meta.demo.json new file mode 100644 index 000000000..17ddc8983 --- /dev/null +++ b/fixtures/conformance/error-filterable-array-field/input/meta.demo.json @@ -0,0 +1,18 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Product", + "children": [ + { "source.rdb": { "@kind": "table", "@table": "products" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "@fields": ["id"] } }, + { "field.string": { "name": "tags", "isArray": true, "@filterable": true, "@db.indexed": true } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-sortable-array-field/expected-errors.json b/fixtures/conformance/error-sortable-array-field/expected-errors.json new file mode 100644 index 000000000..a1c16778d --- /dev/null +++ b/fixtures/conformance/error-sortable-array-field/expected-errors.json @@ -0,0 +1,13 @@ +{ + "errors": [ + { + "code": "ERR_SORTABLE_UNSUPPORTED_SUBTYPE", + "source": { + "format": "json", + "files": ["meta.demo.json"], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[3]['field.string']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-sortable-array-field/input/meta.demo.json b/fixtures/conformance/error-sortable-array-field/input/meta.demo.json new file mode 100644 index 000000000..f5a385b48 --- /dev/null +++ b/fixtures/conformance/error-sortable-array-field/input/meta.demo.json @@ -0,0 +1,18 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Product", + "children": [ + { "source.rdb": { "@kind": "table", "@table": "products" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "@fields": ["id"] } }, + { "field.string": { "name": "tags", "isArray": true, "@sortable": true } } + ] + } + } + ] + } +} From b0533eb837182e9c9b48ba165cde5a9e49f683cf Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 23 Aug 2026 08:56:04 -0400 Subject: [PATCH 09/25] fix(metamodel): array filter/sort rules in the remaining three ports (#335) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Ports the two loader validation rules from TypeScript (task 3) to C#, Java and Python: - @filterable: true on an ARRAY field now errors ERR_FILTERABLE_UNSUPPORTED_ SUBTYPE (extends the existing no-operator-band check in each port). - @sortable: true on an ARRAY field, or a field subtype with no operator band, now errors ERR_SORTABLE_UNSUPPORTED_SUBTYPE — a wholly new pass in all three ports, wired into each port's validation pipeline. Array-ness is read through each port's resolving accessor (C#/Java field.ResolvedIsArray()/isArrayType(), Python field.resolved_is_array()), matching the accessor already used by the @agg:collect check, per ADR-0039. Java additionally gains the ERR_SORTABLE_UNSUPPORTED_SUBTYPE message-prefix constant in ErrorMessageConstants (the ErrorCode enum member already existed; the message ledger did not). Turns fixtures/conformance/error-filterable-array-field and error-sortable-array-field green on all three ports. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq --- .../MetaObjects/Loader/MetaDataLoader.cs | 5 +- .../MetaObjects/Loader/ValidationPasses.cs | 59 ++++++++++++++++++ .../metaobjects/loader/ValidationPhase.java | 60 +++++++++++++++++- .../util/ErrorMessageConstants.java | 9 +++ .../metaobjects/loader/validation_passes.py | 62 ++++++++++++++++++- 5 files changed, 192 insertions(+), 3 deletions(-) diff --git a/server/csharp/MetaObjects/Loader/MetaDataLoader.cs b/server/csharp/MetaObjects/Loader/MetaDataLoader.cs index 04351ba49..0bc0a90a3 100644 --- a/server/csharp/MetaObjects/Loader/MetaDataLoader.cs +++ b/server/csharp/MetaObjects/Loader/MetaDataLoader.cs @@ -503,9 +503,12 @@ public LoadResult Load(IReadOnlyList sources) // Pass 4: filterable-without-index drift warning warnings.AddRange(ValidationPasses.ValidateFilterableHasIndex(root)); - // Pass 4b: @filterable on a subtype with no operator band → error (SP-H Unit9) + // Pass 4b: @filterable on a subtype with no operator band, or an array → error (SP-H Unit9 / #335) errors.AddRange(ValidationPasses.ValidateFilterableHasSupportedOps(root)); + // Pass 4c: @sortable on a subtype with no operator band, or an array → error (#335 Half B) + errors.AddRange(ValidationPasses.ValidateSortableHasSupportedSubtype(root)); + // Pass 5: origin path validation (FR-024 B5/B6 — @via inference, // cardinality, extends/origin agreement) errors.AddRange(ValidationPasses.ValidateOriginPaths(root)); diff --git a/server/csharp/MetaObjects/Loader/ValidationPasses.cs b/server/csharp/MetaObjects/Loader/ValidationPasses.cs index 9bd42d31a..11d6649cf 100644 --- a/server/csharp/MetaObjects/Loader/ValidationPasses.cs +++ b/server/csharp/MetaObjects/Loader/ValidationPasses.cs @@ -376,6 +376,23 @@ public static IReadOnlyList ValidateFilterableHasSupportedOps(MetaDat { // ADR-0039: resolving — a concrete field may inherit @filterable via extends (TS validation-passes.ts:332). if (field.Attr(FIELD_ATTR_FILTERABLE) is not true) continue; + + // #335 Half B — an ARRAY field has no operator band either. Every FR-009 + // operator (eq/ne/gt/gte/lt/lte/in/like/isNull) is a scalar comparison; + // none applies to a collection column. Same reason as the subtype check + // below, so same code. + // ADR-0039: ResolvedIsArray(), never the own `isArray` flag. + if (field.ResolvedIsArray()) + { + errors.Add(new MetaError( + $"Field \"{obj.Name}.{field.Name}\" has @filterable: true but is an array " + + "(isArray: true). No filter operator applies to a collection column. " + + "Remove @filterable from this field.", + ErrorCode.ERR_FILTERABLE_UNSUPPORTED_SUBTYPE, + Envelope: field.Source)); + continue; + } + if (OpsForSubType(field.SubType).Length > 0) continue; errors.Add(new MetaError( $"Field \"{obj.Name}.{field.Name}\" has @filterable: true but its subtype " + @@ -389,6 +406,48 @@ public static IReadOnlyList ValidateFilterableHasSupportedOps(MetaDat return errors.AsReadOnly(); } + // ========================================================================= + // Pass 4c: ValidateSortableHasSupportedSubtype (#335 Half B) + // - @sortable: true on an ARRAY field, or a field subtype with NO entry in + // OPS_BY_SUBTYPE → error ERR_SORTABLE_UNSUPPORTED_SUBTYPE. @sortable + // defaults FROM @filterable, so it is independently checked only when + // explicit — a @sortable JSON or array column emits a sort entry over a + // column no dialect can ORDER BY meaningfully. + // + // Ported from typescript/packages/metadata/src/loader/validation-passes.ts + // validateSortableHasSupportedSubtype. + // ========================================================================= + + public static IReadOnlyList ValidateSortableHasSupportedSubtype(MetaData root) + { + var errors = new List(); + + foreach (var obj in root.OwnChildren() + .Where(c => c.Type == TYPE_OBJECT)) + { + // Children() (effective) — inherited @sortable fields are visible. + foreach (var field in obj.Children().Where(c => c.Type == TYPE_FIELD)) + { + // ADR-0039: resolving — a concrete field may inherit @sortable via extends. + if (field.Attr(FIELD_ATTR_SORTABLE) is not true) continue; + // ADR-0039: ResolvedIsArray(), never the own `isArray` flag. + bool isArray = field.ResolvedIsArray(); + if (!isArray && OpsForSubType(field.SubType).Length > 0) continue; + + var reason = isArray + ? "is an array (isArray: true) — a collection column has no ordering." + : $"its subtype \"{field.SubType}\" cannot be ordered."; + errors.Add(new MetaError( + $"Field \"{obj.Name}.{field.Name}\" has @sortable: true but {reason} " + + "Remove @sortable from this field.", + ErrorCode.ERR_SORTABLE_UNSUPPORTED_SUBTYPE, + Envelope: field.Source)); + } + } + + return errors.AsReadOnly(); + } + // ========================================================================= // Pass 5: ValidateOriginPaths // - passthrough.@from / aggregate.@of must resolve to existing Entity.field diff --git a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java index c2bc56998..5d5a9aa88 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java +++ b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java @@ -230,6 +230,7 @@ public static void run(MetaRoot root, MetaDataLoader loader) { pass(collected, () -> validateTemplates(root)); pass(collected, () -> validateEntityHasPrimaryIdentity(root, loader)); pass(collected, () -> validateFilterableHasSupportedOps(root)); + pass(collected, () -> validateSortableHasSupportedSubtype(root)); pass(collected, () -> validateIndexLookupFields(root)); // The capability ledger's closed status enum (requirements-as-metadata // ruling, Amendment 3) — the loader owns what is UNCONDITIONAL. @@ -3017,8 +3018,24 @@ private static void validateFilterableHasSupportedOps(MetaRoot root) { : (v instanceof String) ? "true".equalsIgnoreCase((String) v) : false; if (!filterable) continue; - if (subtypeSupportsFiltering(field.getSubType())) continue; String objName = obj.getShortName() != null ? obj.getShortName() : obj.getName(); + + // #335 Half B — an ARRAY field has no operator band either. Every + // FR-009 operator (eq/ne/gt/gte/lt/lte/in/like/isNull) is a scalar + // comparison; none applies to a collection column. Same reason as + // the subtype check below, so same code. + // ADR-0039: isArrayType(), the resolving accessor, never isArray(). + if (field.isArrayType()) { + throw new MetaDataException( + ErrorMessageConstants.ERR_FILTERABLE_UNSUPPORTED_SUBTYPE + + ": field \"" + objName + "." + field.getShortName() + + "\" has @filterable: true but is an array (isArray: true)." + + " No filter operator applies to a collection column." + + " Remove @filterable from this field.", + ErrorCode.ERR_FILTERABLE_UNSUPPORTED_SUBTYPE, field.getSource()); + } + + if (subtypeSupportsFiltering(field.getSubType())) continue; throw new MetaDataException( ErrorMessageConstants.ERR_FILTERABLE_UNSUPPORTED_SUBTYPE + ": field \"" + objName + "." + field.getShortName() @@ -3031,6 +3048,47 @@ private static void validateFilterableHasSupportedOps(MetaRoot root) { } } + // ========================================================================= + // @sortable on an array field or unsupported subtype — error pass (#335 Half B) + // + // @sortable defaults FROM @filterable, so it is checked independently only + // when explicit — nothing validated it before, while @filterable has had a + // hard error since SP-H Unit9. A @sortable array/unsupported-subtype column + // emits a sort entry over a column no dialect can ORDER BY meaningfully. + // → ERR_SORTABLE_UNSUPPORTED_SUBTYPE. + // ========================================================================= + + private static void validateSortableHasSupportedSubtype(MetaRoot root) { + for (MetaData rootChild : root.getChildren(MetaData.class, false)) { + if (!(rootChild instanceof MetaObject)) continue; + MetaObject obj = (MetaObject) rootChild; + // Effective fields (includes inherited via extends:/super:). + for (MetaField field : obj.getChildren(MetaField.class, true)) { + if (!field.hasMetaAttr(MetaField.ATTR_SORTABLE, true)) continue; + Object v = field.getMetaAttr(MetaField.ATTR_SORTABLE, true).getValue(); + boolean sortable = + (v instanceof Boolean) ? (Boolean) v + : (v instanceof String) ? "true".equalsIgnoreCase((String) v) + : false; + if (!sortable) continue; + // ADR-0039: isArrayType(), the resolving accessor, never isArray(). + boolean isArray = field.isArrayType(); + if (!isArray && subtypeSupportsFiltering(field.getSubType())) continue; + + String objName = obj.getShortName() != null ? obj.getShortName() : obj.getName(); + String reason = isArray + ? "is an array (isArray: true) — a collection column has no ordering." + : "its subtype \"" + field.getSubType() + "\" cannot be ordered."; + throw new MetaDataException( + ErrorMessageConstants.ERR_SORTABLE_UNSUPPORTED_SUBTYPE + + ": field \"" + objName + "." + field.getShortName() + + "\" has @sortable: true but " + reason + + " Remove @sortable from this field.", + ErrorCode.ERR_SORTABLE_UNSUPPORTED_SUBTYPE, field.getSource()); + } + } + } + /** True iff {@code subType} has a canonical filter-operator band. */ private static boolean subtypeSupportsFiltering(String st) { return com.metaobjects.query.FilterOps.supportsFiltering(st); diff --git a/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java b/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java index 71065f859..1b0c8a3fa 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java +++ b/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java @@ -209,6 +209,15 @@ private ErrorMessageConstants() { */ public static final String ERR_FILTERABLE_UNSUPPORTED_SUBTYPE = "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE"; + /** + * #335 Half B — a field carries {@code @sortable: true} but is an array, or its + * subtype has no filter-operator band (the same band {@code @sortable} borrows + * its ordering test from). A sort entry over such a column has no dialect that + * can order it meaningfully. + * Cross-language contract: {@code ERR_SORTABLE_UNSUPPORTED_SUBTYPE}. + */ + public static final String ERR_SORTABLE_UNSUPPORTED_SUBTYPE = "ERR_SORTABLE_UNSUPPORTED_SUBTYPE"; + /** * Error code emitted when a {@code layout.dataGrid @defaultSortField} value * does not name a real field on the owning entity. diff --git a/server/python/src/metaobjects/loader/validation_passes.py b/server/python/src/metaobjects/loader/validation_passes.py index 4b49a5686..90addcc8a 100644 --- a/server/python/src/metaobjects/loader/validation_passes.py +++ b/server/python/src/metaobjects/loader/validation_passes.py @@ -230,8 +230,10 @@ def run_validations( _validate_identity_passthrough(root, errors) _validate_max_occurs(root, registry, errors) _validate_filterable_has_index(root, warnings) - # SP-H Unit9 — @filterable on a subtype with no operator band → error. + # SP-H Unit9 — @filterable on a subtype with no operator band, or an array → error. _validate_filterable_has_supported_ops(root, errors) + # #335 Half B — @sortable on an array field, or a subtype with no operator band → error. + _validate_sortable_has_supported_subtype(root, errors) _validate_index_lookup_fields(root, errors) @@ -3129,6 +3131,24 @@ def _validate_filterable_has_supported_ops( for field in node.fields(): if field.attrs().get("filterable") is not True: continue + + # #335 Half B — an ARRAY field has no operator band either. Every + # FR-009 operator (eq/ne/gt/gte/lt/lte/in/like/isNull) is a scalar + # comparison; none applies to a collection column. Same reason as + # the subtype check below, so same code. + # ADR-0039: resolved_is_array(), never the own `is_array` flag. + if field.resolved_is_array(): + errors.append( + MetaError( + f'Field "{node.name}.{field.name}" has @filterable: true but is an array ' + f"(isArray: true). No filter operator applies to a collection column. " + f"Remove @filterable from this field.", + ErrorCode.ERR_FILTERABLE_UNSUPPORTED_SUBTYPE, + envelope=field.source, + ) + ) + continue + if ops_for_subtype(field.sub_type): continue errors.append( @@ -3143,6 +3163,46 @@ def _validate_filterable_has_supported_ops( ) +# --------------------------------------------------------------------------- +# Pass: @sortable on an array field or unsupported subtype (#335 Half B) +# --------------------------------------------------------------------------- +# @sortable defaults FROM @filterable, so it is checked independently only +# when explicit — nothing validated it before, while @filterable has had a +# hard error since SP-H Unit9. A @sortable JSON or array column emits a sort +# entry over a column no dialect can ORDER BY meaningfully. +# → ERR_SORTABLE_UNSUPPORTED_SUBTYPE. + + +def _validate_sortable_has_supported_subtype( + root: MetaData, + errors: list[MetaError], +) -> None: + for node in _walk(root): + if node.type != TYPE_OBJECT or not isinstance(node, MetaObject): + continue + for field in node.fields(): + if field.attrs().get("sortable") is not True: + continue + # ADR-0039: resolved_is_array(), never the own `is_array` flag. + is_array = field.resolved_is_array() + if not is_array and ops_for_subtype(field.sub_type): + continue + + reason = ( + "is an array (isArray: true) — a collection column has no ordering." + if is_array + else f'its subtype "{field.sub_type}" cannot be ordered.' + ) + errors.append( + MetaError( + f'Field "{node.name}.{field.name}" has @sortable: true but {reason} ' + f"Remove @sortable from this field.", + ErrorCode.ERR_SORTABLE_UNSUPPORTED_SUBTYPE, + envelope=field.source, + ) + ) + + # --------------------------------------------------------------------------- # Pass: field.object @storage validation # --------------------------------------------------------------------------- From 53f9968f36f43cf7695078ffa5ab1e8ef4275656 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 23 Aug 2026 08:59:50 -0400 Subject: [PATCH 10/25] docs(plan): correct Task 5's test model and Task 9's file count (#335) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both corrections were established by execution, not review. Task 5's model was illegal in two ways unrelated to this feature: a projection cannot `extends` an entity, and a projection's identity must `extends` an entity identity rather than declare fresh @fields. Replaced with the shape the corpus already uses, verified to load clean and to produce exactly the one intended error for the whole-object form. Task 9's grep returns 9 files, not 7 — two are this plan and its spec quoting the sentence. Scoped the grep to product files. Co-Authored-By: Claude Opus 5 (1M context) --- ...026-08-22-issue-335-whole-object-rollup.md | 37 ++++++++++++++++++- 1 file changed, 35 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/plans/2026-08-22-issue-335-whole-object-rollup.md b/docs/superpowers/plans/2026-08-22-issue-335-whole-object-rollup.md index 9e5d9613b..c33ef4ee5 100644 --- a/docs/superpowers/plans/2026-08-22-issue-335-whole-object-rollup.md +++ b/docs/superpowers/plans/2026-08-22-issue-335-whole-object-rollup.md @@ -515,6 +515,35 @@ git commit -m "fix(metamodel): array filter/sort rules in the remaining three po **Why:** Today `@of` is required for `collect`, so only a single scalar column can be rolled up. `@of` is already `"required": false` in the registry — the constraint is validation-only. +> ### CORRECTION — use this model, not the one written below +> +> The model in Step 1 is **wrong** and was verified so by execution. It produces two structural +> errors that have nothing to do with #335: `ERR_SUBTYPE_RULE_VIOLATION` (*"a projection may +> only extend another projection"* — it has `object.projection extends "Product"`, an entity) +> and `ERR_PROJECTION_IDENTITY_NOT_EXTENDED` (a projection identity must `extends` an entity +> identity, not declare fresh `@fields`). +> +> Use this shape instead — copied from the corpus's own `error-origin-aggregate-no-to-many` +> and **verified**: the scalar baseline loads with **0 errors**, and the whole-object form +> produces **exactly one** error, `ERR_INVALID_ORIGIN … missing @of`. +> +> ```jsonc +> { "object.projection": { "name": "ProductWithSuppliers", "children": [ +> { "field.uuid": { "name": "productId", "extends": "acme::Product.id" } }, +> , +> { "identity.primary": { "name": "id", "extends": "acme::Product.id" } } +> ]}} +> ``` +> +> The projection carries **no** object-level `extends`; each field carries its own, and the +> identity uses `extends` rather than `@fields`. Entities are plain: `source.rdb @table`, +> `field.uuid id`, `identity.primary { name: "id", @fields: ["id"] }`, and `Product` holds +> `relationship.association { name: "suppliers", @objectRef: "acme::Supplier", @cardinality: "many" }`. +> +> Also note: `MetaDataLoader.load()` is **async** and returns errors on the result — it does +> not throw. `await` it. Copy the exact harness from +> `server/typescript/packages/metadata/test/validation-filterable-array.test.ts`. + - [ ] **Step 1: Write the failing test** Create `server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts`. Include a **positive** case and each **negative** arm: @@ -933,10 +962,14 @@ git commit -m "feat(metamodel): whole-object rollup in the remaining three ports - [ ] **Step 1: Find every copy** ```bash -grep -rln "Required for count/sum/avg/min/max/collect" . +grep -rln "Required for count/sum/avg/min/max/collect" . --exclude-dir=docs ``` -Expected: exactly 7 files. +Expected: exactly **7 product files** — the seven listed under **File Structure**. + +**Exclude `docs/superpowers/`.** Without that flag the grep returns **9**, because this plan and +its spec both quote the sentence. Verified. Editing those two is harmless but pointless; they +are prose about the change, not copies of the contract. - [ ] **Step 2: Rewrite the sentence identically in all seven** From 451a99f8827b936d0af060cb0ec787c14da87fa6 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 23 Aug 2026 09:10:45 -0400 Subject: [PATCH 11/25] =?UTF-8?q?feat(metamodel):=20@of=20is=20optional=20?= =?UTF-8?q?on=20@agg:collect=20=E2=80=94=20whole-object=20rollup=20(#335)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An @of-absent origin.aggregate @agg:collect is now a WHOLE-OBJECT rollup: the carrying field must be a field.object (isArray, @objectRef) and the collect assembles the related rows as an array of that value object rather than an array of one scalar column. @via is required (no @of entity to infer the relationship from), @distinct is refused (a guaranteed no-op on the object form), and @orderBy resolves against the @via terminal entity via a new _viaTerminalEntityNode helper. The @objectRef must resolve to an object.value — #210's payload-scoped value-only rule never reaches a projection-hosted field, so this branch enforces it itself (ERR_SUBTYPE_RULE_VIOLATION), closing the same hole 0.20.16 (#270) closed for payload-typed fields. The scalar collect-with-@of path is untouched. --- .../metadata/src/loader/validation-passes.ts | 116 ++++++++++++++- .../validation-collect-whole-object.test.ts | 133 ++++++++++++++++++ 2 files changed, 245 insertions(+), 4 deletions(-) create mode 100644 server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts diff --git a/server/typescript/packages/metadata/src/loader/validation-passes.ts b/server/typescript/packages/metadata/src/loader/validation-passes.ts index 3d54b335d..d6c8ab6e9 100644 --- a/server/typescript/packages/metadata/src/loader/validation-passes.ts +++ b/server/typescript/packages/metadata/src/loader/validation-passes.ts @@ -766,6 +766,50 @@ function _validateViaPath( return hops; } +/** + * #335 — walk an explicit `@via` "Entity.rel[.rel...]" path and return its + * TERMINAL entity node (the entity reached after the last hop), or undefined + * if any segment fails to resolve. Mirrors `_validateViaPath`'s hop-walking + * exactly but returns the terminal object instead of the walked hop nodes, + * and pushes no errors — callers use it only once `_validateViaPath` has + * already validated the same path and returned successfully. + * + * A whole-object `@agg:collect` has no `@of` entity to resolve `@orderBy` + * keys against (the #195 scalar path resolves them against the `@of` column's + * entity); this is the terminal-entity equivalent for the whole-object path. + * Codegen-side equivalent: `codegen-ts/src/projection/extract-view-spec.ts`'s + * `viaTerminalEntity`. + */ +function _viaTerminalEntityNode( + viaAttr: string, + root: MetaData, + projection: MetaData, +): MetaData | undefined { + const segments = viaAttr.split("."); + const entityName = segments[0]; + if (entityName === undefined) return undefined; + // ADR-0042 — a bare @via HEAD resolves in the projection's package. + const referrerPkg = projection.package ?? projection.fileDefaultPackage ?? ""; + let currentObj = _findObject(root, entityName, referrerPkg); + if (!currentObj) return undefined; + for (const relName of segments.slice(1)) { + const rel = _findRelationship(currentObj, relName) ?? _findReference(currentObj, relName); + if (!rel) return undefined; + const refTarget = _hopTargetName(rel); + if (typeof refTarget !== "string" || refTarget === "") return undefined; + // ADR-0042 — the hop target resolves in the package of the entity that + // DECLARES the relationship/reference, i.e. currentObj. + const nextObj = _findObject( + root, + refTarget, + currentObj.package ?? currentObj.fileDefaultPackage ?? "", + ); + if (!nextObj) return undefined; + currentObj = nextObj; + } + return currentObj; +} + // --------------------------------------------------------------------------- // FR-024 B5 — base-entity derivation, single-hop-unique @via inference, and // origin cardinality checks (spec §5–§6; ADR-0029 decisions 5–6). @@ -1306,11 +1350,75 @@ export function validateOriginPaths(root: MetaData): ParseError[] { continue; } - // --- count/sum/avg/min/max/collect: @of REQUIRED --- + // --- @of: REQUIRED for count/sum/avg/min/max; OPTIONAL for collect --- + // #335 — an @of-absent collect is a WHOLE-OBJECT rollup: collect the + // related rows as an array of the field's declared @objectRef value + // object rather than an array of one scalar column. if (!ofPresent) { - errors.push(new ParseError( - `origin.aggregate on ${obj.name}.${field.name}: missing @of.`, - { code: "ERR_INVALID_ORIGIN", source: src })); + if (!isCollect) { + errors.push(new ParseError( + `origin.aggregate on ${obj.name}.${field.name}: missing @of.`, + { code: "ERR_INVALID_ORIGIN", source: src })); + continue; + } + // Whole-object rollup. The carrying field must be a field.object + // naming a value object, and @via must be explicit (there is no @of + // entity to infer the single-hop relation from). + // ADR-0039: resolving — @objectRef may be inherited via extends. + const objectRef = field.attr(FIELD_ATTR_OBJECT_REF); + if (field.subType !== FIELD_SUBTYPE_OBJECT || typeof objectRef !== "string" || objectRef === "") { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: @of is omitted, so this is a ` + + `whole-object rollup — the carrying field must be a field.object declaring @objectRef ` + + `(add @of to collect a single column instead).`, + { code: "ERR_INVALID_ORIGIN", source: src })); + continue; + } + // #210's value-only rule is PAYLOAD-scoped and never reaches a + // projection-hosted field, so this branch enforces it itself. + // Without it an @objectRef to an entity silently rolls up the FULL + // entity — the #270 shape, this time baked into DDL. + // ADR-0042 — a bare @objectRef resolves in the DECLARING owner's + // package (an inherited field resolves in the package that + // declared it) — same rule _checkNestedPayloadRefsValueOnly uses. + const refOwner = field.parent ?? obj; + const refPkg = refOwner.package ?? refOwner.fileDefaultPackage ?? ""; + const refTarget = resolveObjectRef(root, objectRef, refPkg).node; + if (refTarget !== undefined && refTarget.subType !== OBJECT_SUBTYPE_VALUE) { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: @objectRef '${objectRef}' ` + + `resolves to ${TYPE_OBJECT}.${refTarget.subType} — a whole-object rollup must target an ` + + `object.value (#210, ADR-0028).`, + { code: "ERR_SUBTYPE_RULE_VIOLATION", source: src })); + continue; + } + // ADR-0039: own — origin.* never inherits (ADR-0029). + const viaAttr = origin.ownAttr(ORIGIN_AGGREGATE_ATTR_VIA); + if (typeof viaAttr !== "string" || viaAttr === "") { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: @via is required on a ` + + `whole-object rollup — there is no @of entity to infer the relationship from.`, + { code: "ERR_INVALID_ORIGIN", source: src })); + continue; + } + // @distinct is refused on the object form. It is NOT an engine limit + // (both engines dedupe JSON objects); it is a guaranteed no-op + // whenever the value object carries the entity's primary key, which + // is the common case, and a silent no-op is worse than a refusal. + if (hasDistinct) { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: @distinct is not supported on a ` + + `whole-object rollup (it is a no-op whenever the value object carries the primary key).`, + { code: "ERR_INVALID_ORIGIN", source: src })); + continue; + } + const hops = _validateViaPath(viaAttr, root, obj, field.name, src, errors); + if (hops !== undefined) _checkAggregateCardinality(hops, obj, field.name, src, errors); + // @orderBy keys resolve against the @via TERMINAL entity, not @of. + if (hasOrderBy) { + const terminal = _viaTerminalEntityNode(viaAttr, root, obj); + _validateOrderByKeys(orderBy, terminal, obj, field.name, "origin.aggregate @agg:collect", src, errors); + } continue; } // NOTE (FR-024 B6): NO extends/origin agreement check on aggregates — diff --git a/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts b/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts new file mode 100644 index 000000000..f9773de4b --- /dev/null +++ b/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts @@ -0,0 +1,133 @@ +// #335 Half A — @of is optional on origin.aggregate @agg:collect. +// +// Today collect always requires @of (an "Entity.field" naming ONE scalar +// column). When @of is absent, this is instead a WHOLE-OBJECT rollup: the +// carrying field must be a field.object (isArray, @objectRef) and the +// collect assembles the related rows as an array of that value object +// rather than an array of one scalar column. +// +// Model shape verified by execution against the corrected model in the +// task-5 brief (the model originally drafted in the brief's Step 1 does +// NOT load — it declares an object-level `object.projection extends +// "Product"`, an entity, which is ERR_SUBTYPE_RULE_VIOLATION, and an +// identity.primary with fresh @fields instead of extends, which is +// ERR_PROJECTION_IDENTITY_NOT_EXTENDED). This model instead mirrors the +// corpus's own proven-loading shape (fixtures/conformance/ +// error-origin-aggregate-no-to-many/input/meta.demo.json, CustomerSummary): +// no object-level extends on the projection; a plain field mirroring the +// base entity's PK via field-level extends; identity.primary named and +// extending the base entity's (also-named) identity, never declaring its +// own @fields. +// +// MetaDataLoader.load() is async and returns errors on the LoadResult — it +// does not throw. Harness copied from the sibling +// validation-filterable-array.test.ts. + +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader } from "../src/loader/meta-data-loader.js"; +import { InMemoryStringSource } from "../src/loader/meta-data-source.js"; + +/** Product 1:N Supplier, plus a projection rolling suppliers up as objects. */ +const model = (collectField: string) => `{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { "name": "Supplier", "children": [ + { "source.rdb": { "@kind": "table", "@table": "suppliers" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "name" } }, + { "field.long": { "name": "productId" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "identity.reference": { "name": "product", "@references": "Product", "@fields": ["productId"] } } + ]}}, + { "object.entity": { "name": "Product", "children": [ + { "source.rdb": { "@kind": "table", "@table": "products" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "relationship.association": { "name": "suppliers", "@cardinality": "many", "@objectRef": "Supplier" } } + ]}}, + { "object.value": { "name": "SupplierBrief", "children": [ + { "field.long": { "name": "id" } }, + { "field.string": { "name": "name" } } + ]}}, + { "object.projection": { "name": "ProductWithSuppliers", "children": [ + { "source.rdb": { "@kind": "view", "@view": "v_product_suppliers" } }, + { "field.long": { "name": "productId", "extends": "Product.id" } }, + { "identity.primary": { "name": "id", "extends": "Product.id" } }, + ${collectField} + ]}} + ] + } +}`; + +const WHOLE_OBJECT = `{ "field.object": { + "name": "supplierBriefs", "isArray": true, "@objectRef": "SupplierBrief", + "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers" } } ] +}}`; + +async function loadErrors(src: string): Promise<{ code: string; message: string }[]> { + const result = await new MetaDataLoader().load([ + new InMemoryStringSource(src, { id: "meta.demo.json" }), + ]); + return result.errors as unknown as { code: string; message: string }[]; +} + +describe("@of-absent collect (whole-object rollup)", () => { + test("loads on a field.object @objectRef isArray with @via", async () => { + const errors = await loadErrors(model(WHOLE_OBJECT)); + expect(errors).toEqual([]); + }); + + test("fails without @objectRef", async () => { + const src = model(`{ "field.object": { + "name": "supplierBriefs", "isArray": true, + "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers" } } ] + }}`); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_INVALID_ORIGIN"); + expect(hit).toBeDefined(); + expect(hit?.message).toContain("supplierBriefs"); + }); + + test("fails when @objectRef targets an entity, not a value", async () => { + const src = model(`{ "field.object": { + "name": "supplierBriefs", "isArray": true, "@objectRef": "Supplier", + "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers" } } ] + }}`); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_SUBTYPE_RULE_VIOLATION"); + expect(hit).toBeDefined(); + expect(hit?.message).toContain("supplierBriefs"); + }); + + test("fails without @via (nothing to infer the relation from)", async () => { + const src = model(`{ "field.object": { + "name": "supplierBriefs", "isArray": true, "@objectRef": "SupplierBrief", + "children": [ { "origin.aggregate": { "@agg": "collect" } } ] + }}`); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_INVALID_ORIGIN"); + expect(hit).toBeDefined(); + expect(hit?.message).toContain("supplierBriefs"); + }); + + test("fails when @distinct is declared", async () => { + const src = model(`{ "field.object": { + "name": "supplierBriefs", "isArray": true, "@objectRef": "SupplierBrief", + "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers", "@distinct": true } } ] + }}`); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_INVALID_ORIGIN"); + expect(hit).toBeDefined(); + expect(hit?.message).toContain("supplierBriefs"); + }); + + test("a collect WITH @of is unaffected", async () => { + const src = model(`{ "field.string": { + "name": "supplierNames", "isArray": true, + "children": [ { "origin.aggregate": { "@agg": "collect", "@of": "Supplier.name", "@via": "Product.suppliers" } } ] + }}`); + const errors = await loadErrors(src); + expect(errors).toEqual([]); + }); +}); From 4b998a7275836eca39023eadfb47f7e2be845946 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 23 Aug 2026 09:23:45 -0400 Subject: [PATCH 12/25] fix(metamodel): pin cardinality + @orderBy-terminal in whole-object collect (#335) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fix round 1 on the whole-object collect review: two of the six must-enforce rules had no test arm, so deleting the _validateViaPath/_checkAggregateCardinality call and the entire _viaTerminalEntityNode call site still passed all six original tests. - Add "@via provably to-one" arm (ERR_ORIGIN_CARDINALITY via a single reference hop, inherently to-one). - Add a 3-entity A.b.c chain arm proving @orderBy resolves against the @via TERMINAL entity (C) rather than the head (A) or a middle hop (B). - Gate the @orderBy call on `hops !== undefined` — _viaTerminalEntityNode does not reproduce _validateViaPath's malformed-@via guard, so an invalid @via was emitting a second, misleadingly-scoped @orderBy error alongside the real one. - The three arms that previously discriminated only by the shared ERR_INVALID_ORIGIN code now also assert each rule's distinctive message fragment. Both new arms verified by commenting out their corresponding source lines (cardinality call; the whole hasOrderBy block) and confirming the matching arm — and only that arm — fails, then restoring. --- .../metadata/src/loader/validation-passes.ts | 6 +- .../validation-collect-whole-object.test.ts | 88 +++++++++++++++++++ 2 files changed, 93 insertions(+), 1 deletion(-) diff --git a/server/typescript/packages/metadata/src/loader/validation-passes.ts b/server/typescript/packages/metadata/src/loader/validation-passes.ts index d6c8ab6e9..90b2e86fb 100644 --- a/server/typescript/packages/metadata/src/loader/validation-passes.ts +++ b/server/typescript/packages/metadata/src/loader/validation-passes.ts @@ -1415,7 +1415,11 @@ export function validateOriginPaths(root: MetaData): ParseError[] { const hops = _validateViaPath(viaAttr, root, obj, field.name, src, errors); if (hops !== undefined) _checkAggregateCardinality(hops, obj, field.name, src, errors); // @orderBy keys resolve against the @via TERMINAL entity, not @of. - if (hasOrderBy) { + // Gated on hops !== undefined — _viaTerminalEntityNode does not + // reproduce _validateViaPath's malformed-shape guard, so an + // invalid @via (e.g. single-segment "A") must not also emit a + // second, misleadingly-scoped @orderBy error. + if (hasOrderBy && hops !== undefined) { const terminal = _viaTerminalEntityNode(viaAttr, root, obj); _validateOrderByKeys(orderBy, terminal, obj, field.name, "origin.aggregate @agg:collect", src, errors); } diff --git a/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts b/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts index f9773de4b..83c5bcf46 100644 --- a/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts +++ b/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts @@ -22,6 +22,16 @@ // MetaDataLoader.load() is async and returns errors on the LoadResult — it // does not throw. Harness copied from the sibling // validation-filterable-array.test.ts. +// +// Fix round 1: the brief's six must-enforce rules and its own six test +// arms did not correspond 1:1 — cardinality and @orderBy-resolves-against- +// the-terminal-entity had no arm at all, so a review that deleted the +// _validateViaPath/_checkAggregateCardinality call AND the entire +// `_viaTerminalEntityNode` call site (every caller of the helper) still +// passed all six original tests. Two arms added below close that gap, and +// the three arms that previously discriminated only by the shared +// ERR_INVALID_ORIGIN code now also assert each rule's distinctive message +// fragment. import { describe, test, expect } from "bun:test"; import { MetaDataLoader } from "../src/loader/meta-data-loader.js"; @@ -60,6 +70,51 @@ const model = (collectField: string) => `{ } }`; +/** + * A.b.c three-entity chain: A -> (relationship b, many) -> B -> (relationship + * c, many) -> C. B declares field "name"; C does not. A whole-object collect + * on a projection of A rolls up C (via "A.b.c", 2 hops) — @orderBy must + * resolve against C (the TERMINAL entity reached after BOTH hops), not A + * (the @via head) or B (the middle hop). Both relationships are @cardinality + * "many" so no ERR_ORIGIN_CARDINALITY noise competes with the assertion. + */ +const CHAIN_MODEL = `{ + "metadata.root": { + "package": "acme::chain", + "children": [ + { "object.entity": { "name": "C", "children": [ + { "source.rdb": { "@kind": "table", "@table": "cs" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } } + ]}}, + { "object.entity": { "name": "B", "children": [ + { "source.rdb": { "@kind": "table", "@table": "bs" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "name" } }, + { "relationship.association": { "name": "c", "@cardinality": "many", "@objectRef": "C" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } } + ]}}, + { "object.entity": { "name": "A", "children": [ + { "source.rdb": { "@kind": "table", "@table": "as" } }, + { "field.long": { "name": "id" } }, + { "relationship.association": { "name": "b", "@cardinality": "many", "@objectRef": "B" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } } + ]}}, + { "object.value": { "name": "CBrief", "children": [ + { "field.long": { "name": "id" } } + ]}}, + { "object.projection": { "name": "AWithCs", "children": [ + { "source.rdb": { "@kind": "view", "@view": "v_a_cs" } }, + { "field.long": { "name": "aId", "extends": "A.id" } }, + { "identity.primary": { "name": "id", "extends": "A.id" } }, + { "field.object": { "name": "items", "isArray": true, "@objectRef": "CBrief", "children": [ + { "origin.aggregate": { "@agg": "collect", "@via": "A.b.c", "@orderBy": ["name"] } } + ]}} + ]}} + ] + } +}`; + const WHOLE_OBJECT = `{ "field.object": { "name": "supplierBriefs", "isArray": true, "@objectRef": "SupplierBrief", "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers" } } ] @@ -87,6 +142,7 @@ describe("@of-absent collect (whole-object rollup)", () => { const hit = errors.find((e) => e.code === "ERR_INVALID_ORIGIN"); expect(hit).toBeDefined(); expect(hit?.message).toContain("supplierBriefs"); + expect(hit?.message).toContain("must be a field.object"); }); test("fails when @objectRef targets an entity, not a value", async () => { @@ -109,6 +165,7 @@ describe("@of-absent collect (whole-object rollup)", () => { const hit = errors.find((e) => e.code === "ERR_INVALID_ORIGIN"); expect(hit).toBeDefined(); expect(hit?.message).toContain("supplierBriefs"); + expect(hit?.message).toContain("@via is required"); }); test("fails when @distinct is declared", async () => { @@ -120,6 +177,37 @@ describe("@of-absent collect (whole-object rollup)", () => { const hit = errors.find((e) => e.code === "ERR_INVALID_ORIGIN"); expect(hit).toBeDefined(); expect(hit?.message).toContain("supplierBriefs"); + expect(hit?.message).toContain("@distinct is not supported"); + }); + + test("fails when @via is provably to-one (cardinality)", async () => { + // "Supplier.product" is a single reference hop — inherently to-one + // (_hopCardinality treats every identity.reference hop as CARDINALITY_ONE) + // — so aggregating over it is the passthrough-not-aggregate mistake + // ADR-0029 decision 6 rejects. This is the arm that pins the + // `_validateViaPath` + `_checkAggregateCardinality` call in the + // whole-object branch: deleting that call leaves this model loading + // clean. + const src = model(`{ "field.object": { + "name": "supplierBriefs", "isArray": true, "@objectRef": "SupplierBrief", + "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Supplier.product" } } ] + }}`); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_ORIGIN_CARDINALITY"); + expect(hit).toBeDefined(); + }); + + test("@orderBy resolves against the @via TERMINAL entity, not the head or a middle hop", async () => { + // CHAIN_MODEL's "name" field exists on B (the middle hop) but not on C + // (the terminal entity two hops from A). This is the arm that pins + // `_viaTerminalEntityNode` actually walking to the END of the path: a + // regression to head-only resolution would name "A" in the error, a + // regression that stops at the first hop would name "B", and deleting + // the `if (hasOrderBy...)` call site entirely emits no such error at all. + const errors = await loadErrors(CHAIN_MODEL); + const hit = errors.find((e) => e.code === "ERR_INVALID_ORIGIN" && e.message.includes("@orderBy")); + expect(hit).toBeDefined(); + expect(hit?.message).toContain('no such field "name" on C'); }); test("a collect WITH @of is unaffected", async () => { From 104605602c90be68ca25af082d5170be2c44cd64 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 23 Aug 2026 09:34:09 -0400 Subject: [PATCH 13/25] feat(metamodel): whole-object rollup resolves VO members against the @via terminal (#335) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The lowering for an @of-absent origin.aggregate @agg:collect (the whole- object rollup added in the prior commit) now projects EXACTLY the declared value object's members, matched by name against the @via terminal entity: - An unmatched member fails with a new code, ERR_COLLECT_MEMBER_UNRESOLVED. Failing open here is how #270 turned a curated value object into the full entity, invisible in a diff because the metadata still read as curated. - A member whose field. differs from the matched entity field's fails with ERR_INVALID_ORIGIN — the object-form analogue of the scalar @of path's element-type check, same #185 type-preserving doctrine. Both sides are read with the resolving accessor (children()), never own*(), per ADR-0039 — a value object or entity may bring members via extends, and an own-only read would silently skip them, which is exactly the bug class this guard exists to prevent. ERR_COLLECT_MEMBER_UNRESOLVED is registered across all six error ledgers (ERROR-CODES.json, TS/Python/C#/Java ErrorCode enums, and Java's separate ErrorMessageConstants message ledger — discovered on an earlier #335 task to be a sixth site, not five). ERR_INVALID_ORIGIN is reused, not re-added. Two new arms added to the existing validation-collect-whole-object.test.ts (the model helper gained an optional voFields parameter so SupplierBrief's member list can vary per test without touching any existing call site). Each arm's pin was verified by commenting out its corresponding check, confirming that arm alone failed, then restoring to a clean diff. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq --- fixtures/conformance/ERROR-CODES.json | 1 + server/csharp/MetaObjects/Errors.cs | 5 ++ .../main/java/com/metaobjects/ErrorCode.java | 8 +++ .../util/ErrorMessageConstants.java | 9 ++++ server/python/src/metaobjects/errors.py | 5 ++ .../packages/metadata/src/errors.ts | 5 ++ .../metadata/src/loader/validation-passes.ts | 40 +++++++++++++++ .../validation-collect-whole-object.test.ts | 49 ++++++++++++++++--- 8 files changed, 116 insertions(+), 6 deletions(-) diff --git a/fixtures/conformance/ERROR-CODES.json b/fixtures/conformance/ERROR-CODES.json index a259282a8..25688fe99 100644 --- a/fixtures/conformance/ERROR-CODES.json +++ b/fixtures/conformance/ERROR-CODES.json @@ -71,6 +71,7 @@ "ERR_MERGE_CONFLICT": "FR5c: multi-file overlay merge produced an attribute conflict \u2014 two contributing files declared the same @attr with different non-empty values on the same logical node. Envelope is `format: \"merged\"` with `contributors[]` naming both files.", "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE": "SP-H Unit9: a field carries @filterable: true but its subtype has no filter-operator band (e.g. field.object). Generating a filter for it would emit an empty operator set \u2014 a filter route that rejects every request. Use a filterable subtype (string/enum/uuid/number/currency/date/boolean) or remove @filterable.", "ERR_SORTABLE_UNSUPPORTED_SUBTYPE": "#335 Half B: a field carries @sortable: true but cannot be ordered \u2014 either it is an array (isArray: true; a collection column has no ordering) or its subtype has no filter-operator band (e.g. field.object), the same subtype-support signal @filterable uses (SP-H Unit9). Generating a sort entry for it would emit an ORDER BY no dialect can execute. Use a sortable subtype (string/enum/uuid/number/currency/date/boolean) or remove @sortable.", + "ERR_COLLECT_MEMBER_UNRESOLVED": "#335 Half A: a whole-object origin.aggregate @agg:collect (no @of \u2014 the carrying field.object collects related rows as its declared @objectRef value object) has a value-object member with no matching field (by name) on the @via terminal entity. The lowering projects exactly the declared members; failing open here is how #270 turned a curated value object into the full entity, invisible in a diff because the metadata still read as curated.", "ERR_REGISTRY_SEALED": "ADR-0023: a registration (register / registerType / extend / registerCommonAttrs / setDefaultSubType / addConstraint / registerProviders) was attempted against a registry sealed after its agreed metamodel-provider bootstrap. Made-up metamodel attributes/types are structurally disallowed in the library: codegen cannot register post-bootstrap. A new metamodel attribute requires a registered provider + human agreement. Downstream apps that need extra vocabulary compose their own (unsealed) registry.", "ERR_UNKNOWN": "An internal loader error with no stable error code.", "ERR_ENTITY_PRIMARY_SOURCE_READONLY": "FR-024 (ADR-0028) hard cutover: an object.entity's PRIMARY source has a read-only @kind (view/materializedView/storedProc/tableFunction). Read-only kinds are legal only in non-primary roles; a derived read model is an object.projection.", diff --git a/server/csharp/MetaObjects/Errors.cs b/server/csharp/MetaObjects/Errors.cs index 1567d33bf..8bcc36200 100644 --- a/server/csharp/MetaObjects/Errors.cs +++ b/server/csharp/MetaObjects/Errors.cs @@ -177,6 +177,11 @@ public enum ErrorCode // filter-operator band (e.g. field.object). Would silently emit a sort // entry over a column no dialect can ORDER BY. ERR_SORTABLE_UNSUPPORTED_SUBTYPE, + // #335 Half A — a whole-object @agg:collect's value-object member has no + // matching field (by name) on the @via terminal entity. The lowering + // projects exactly the declared members; failing open here is how #270 + // turned a curated value object into the full entity. + ERR_COLLECT_MEMBER_UNRESOLVED, // Index-key resolution for index.lookup AND identity.secondary (#342) — the key is // @fields XOR @expr: neither declared, BOTH declared (@expr is used INSTEAD of // @fields), whichever is declared supplies no key, or a named field does not exist diff --git a/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java b/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java index 4b687dcfd..5b527b6ed 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java +++ b/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java @@ -374,6 +374,14 @@ public enum ErrorCode { */ ERR_SORTABLE_UNSUPPORTED_SUBTYPE, + /** + * #335 Half A — a whole-object {@code @agg:collect}'s value-object member has no + * matching field (by name) on the {@code @via} terminal entity. The lowering + * projects exactly the declared members; failing open here is how #270 turned a + * curated value object into the full entity. + */ + ERR_COLLECT_MEMBER_UNRESOLVED, + /** * ADR-0023 Decision 2: a registration ({@code register}/{@code extendType}/ * {@code registerCommonAttribute}/{@code addConstraint}/{@code registerType}/ diff --git a/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java b/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java index 1b0c8a3fa..ff715ea0b 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java +++ b/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java @@ -218,6 +218,15 @@ private ErrorMessageConstants() { */ public static final String ERR_SORTABLE_UNSUPPORTED_SUBTYPE = "ERR_SORTABLE_UNSUPPORTED_SUBTYPE"; + /** + * #335 Half A — a whole-object {@code @agg:collect}'s value-object member has no + * matching field (by name) on the {@code @via} terminal entity. The lowering + * projects exactly the declared members; failing open here is how #270 turned a + * curated value object into the full entity. + * Cross-language contract: {@code ERR_COLLECT_MEMBER_UNRESOLVED}. + */ + public static final String ERR_COLLECT_MEMBER_UNRESOLVED = "ERR_COLLECT_MEMBER_UNRESOLVED"; + /** * Error code emitted when a {@code layout.dataGrid @defaultSortField} value * does not name a real field on the owning entity. diff --git a/server/python/src/metaobjects/errors.py b/server/python/src/metaobjects/errors.py index 61700192b..e5d6724d7 100644 --- a/server/python/src/metaobjects/errors.py +++ b/server/python/src/metaobjects/errors.py @@ -160,6 +160,11 @@ class ErrorCode(str, Enum): # filter-operator band (e.g. field.object). Would silently emit a sort # entry over a column no dialect can ORDER BY. ERR_SORTABLE_UNSUPPORTED_SUBTYPE = "ERR_SORTABLE_UNSUPPORTED_SUBTYPE" + # #335 Half A — a whole-object @agg:collect's value-object member has no + # matching field (by name) on the @via terminal entity. The lowering + # projects exactly the declared members; failing open here is how #270 + # turned a curated value object into the full entity. + ERR_COLLECT_MEMBER_UNRESOLVED = "ERR_COLLECT_MEMBER_UNRESOLVED" # ADR-0023 — a registration was attempted against a registry sealed after its # agreed metamodel-provider bootstrap. Codegen cannot invent metamodel attrs. ERR_REGISTRY_SEALED = "ERR_REGISTRY_SEALED" diff --git a/server/typescript/packages/metadata/src/errors.ts b/server/typescript/packages/metadata/src/errors.ts index 78858091f..e9f39db3b 100644 --- a/server/typescript/packages/metadata/src/errors.ts +++ b/server/typescript/packages/metadata/src/errors.ts @@ -163,6 +163,11 @@ export const ERROR_CODES = [ // filter-operator band (e.g. field.object). Would silently emit a sort // entry over a column no dialect can ORDER BY. "ERR_SORTABLE_UNSUPPORTED_SUBTYPE", + // #335 Half A — a whole-object @agg:collect's value-object member has no + // matching field (by name) on the @via terminal entity. The lowering + // projects exactly the declared members; failing open here is how #270 + // turned a curated value object into the full entity. + "ERR_COLLECT_MEMBER_UNRESOLVED", // ADR-0023 — a registration was attempted against a registry sealed after its // agreed metamodel-provider bootstrap. Codegen cannot invent metamodel attrs. "ERR_REGISTRY_SEALED", diff --git a/server/typescript/packages/metadata/src/loader/validation-passes.ts b/server/typescript/packages/metadata/src/loader/validation-passes.ts index 90b2e86fb..fed3bb164 100644 --- a/server/typescript/packages/metadata/src/loader/validation-passes.ts +++ b/server/typescript/packages/metadata/src/loader/validation-passes.ts @@ -1423,6 +1423,46 @@ export function validateOriginPaths(root: MetaData): ParseError[] { const terminal = _viaTerminalEntityNode(viaAttr, root, obj); _validateOrderByKeys(orderBy, terminal, obj, field.name, "origin.aggregate @agg:collect", src, errors); } + // Member resolution — the lowering projects EXACTLY the declared + // value object's members, matched by NAME against the @via + // TERMINAL entity. An unmatched member must error: failing open + // here is how #270 turned a curated value object into the full + // entity, invisible in a diff because the metadata still read as + // curated. Gated on hops !== undefined for the same reason the + // @orderBy check above is: _viaTerminalEntityNode does not + // reproduce _validateViaPath's malformed-shape guard. + if (hops !== undefined) { + const terminal = _viaTerminalEntityNode(viaAttr, root, obj); + if (terminal !== undefined && refTarget !== undefined) { + // ADR-0039: resolving — a value object may inherit members via + // extends, and the terminal entity may inherit fields; + // own-only would silently skip inherited members, which is + // exactly the #270 bug class this guard exists to prevent. + const terminalFields = terminal.children().filter((c) => c.type === TYPE_FIELD); + for (const member of refTarget.children().filter((c) => c.type === TYPE_FIELD)) { + const match = terminalFields.find((f) => f.name === member.name); + if (match === undefined) { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: value-object member ` + + `'${member.name}' has no matching field on '${terminal.name}' — a whole-object ` + + `rollup projects exactly the declared members.`, + { code: "ERR_COLLECT_MEMBER_UNRESOLVED", source: src })); + continue; + } + // Per-member type agreement — the object-form analogue of + // the scalar element-type check below (#185 type-preserving + // doctrine), reusing ERR_INVALID_ORIGIN for the same "types + // disagree" shape rather than minting a second code. + if (member.subType !== match.subType) { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: value-object member ` + + `'${member.name}' is field.${member.subType} but '${terminal.name}.${match.name}' ` + + `is field.${match.subType} — a whole-object rollup preserves each member's type.`, + { code: "ERR_INVALID_ORIGIN", source: src })); + } + } + } + } continue; } // NOTE (FR-024 B6): NO extends/origin agreement check on aggregates — diff --git a/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts b/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts index 83c5bcf46..97817509b 100644 --- a/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts +++ b/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts @@ -37,8 +37,17 @@ import { describe, test, expect } from "bun:test"; import { MetaDataLoader } from "../src/loader/meta-data-loader.js"; import { InMemoryStringSource } from "../src/loader/meta-data-source.js"; -/** Product 1:N Supplier, plus a projection rolling suppliers up as objects. */ -const model = (collectField: string) => `{ +/** SupplierBrief's default member set — mirrors Supplier.id + Supplier.name exactly. */ +const DEFAULT_SUPPLIER_BRIEF_FIELDS = + `{ "field.long": { "name": "id" } }, { "field.string": { "name": "name" } }`; + +/** + * Product 1:N Supplier, plus a projection rolling suppliers up as objects. + * `voFields` parameterises SupplierBrief's member list (#335 Half A member + * resolution) — defaults to the field set every pre-existing arm in this file + * assumes, so no existing call site needs updating. + */ +const model = (collectField: string, voFields: string = DEFAULT_SUPPLIER_BRIEF_FIELDS) => `{ "metadata.root": { "package": "acme::shop", "children": [ @@ -56,10 +65,7 @@ const model = (collectField: string) => `{ { "identity.primary": { "name": "id", "@fields": ["id"] } }, { "relationship.association": { "name": "suppliers", "@cardinality": "many", "@objectRef": "Supplier" } } ]}}, - { "object.value": { "name": "SupplierBrief", "children": [ - { "field.long": { "name": "id" } }, - { "field.string": { "name": "name" } } - ]}}, + { "object.value": { "name": "SupplierBrief", "children": [ ${voFields} ] }}, { "object.projection": { "name": "ProductWithSuppliers", "children": [ { "source.rdb": { "@kind": "view", "@view": "v_product_suppliers" } }, { "field.long": { "name": "productId", "extends": "Product.id" } }, @@ -218,4 +224,35 @@ describe("@of-absent collect (whole-object rollup)", () => { const errors = await loadErrors(src); expect(errors).toEqual([]); }); + + // #335 Half A — member resolution. The lowering projects EXACTLY the + // declared value object's members, matched by NAME against the @via + // TERMINAL entity (Supplier, here). Failing open on an unresolved member is + // how #270 turned a curated value object into the full entity. + + test("a VO member with no matching field on the @via terminal fails", async () => { + // SupplierBrief declares "nickname"; Supplier has no such field. + const src = model( + WHOLE_OBJECT, + `{ "field.long": { "name": "id" } }, { "field.string": { "name": "nickname" } }`, + ); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_COLLECT_MEMBER_UNRESOLVED"); + expect(hit).toBeDefined(); + expect(hit?.message).toContain("nickname"); + expect(hit?.message).toContain("Supplier"); + }); + + test("a VO member whose type differs from the matched field fails", async () => { + // SupplierBrief declares "name" as a field.long; Supplier.name is a field.string. + const src = model( + WHOLE_OBJECT, + `{ "field.long": { "name": "id" } }, { "field.long": { "name": "name" } }`, + ); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_INVALID_ORIGIN" && e.message.includes("value-object member")); + expect(hit).toBeDefined(); + expect(hit?.message).toContain("'name' is field.long"); + expect(hit?.message).toContain("Supplier.name' is field.string"); + }); }); From bbd1fab40b4695da66c102b65a63f1ea2992ec1d Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 23 Aug 2026 09:47:10 -0400 Subject: [PATCH 14/25] test(conformance): gate the whole-object rollup and its eight error arms (#335) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Half A conformance fixtures for the @of-absent origin.aggregate @agg:collect whole-object rollup: one positive fixture plus eight negative arms covering every rule the loader enforces on the @of-absent branch (the scope grew from the brief's original six to eight — @distinct and @orderBy-terminal landed after the brief was written). Models are lifted from the verified validation-collect-whole-object.test.ts harness; jsonPaths and the positive expected.json are copied verbatim from the runner's actual output, never hand-derived. error-collect-no-object-ref intentionally asserts two errors (ERR_INVALID_ORIGIN from the new rule plus the pre-existing ERR_OBJECT_FIELD_WITHOUT_OBJECT_REF, which fires on any field.object missing @objectRef regardless of collect) — both are inherent to the shape. error-collect-via-to-one's value object is narrowed to a single id member (matching the mis-cardinality path's actual terminal) so member-resolution can't raise a second, unrelated ERR_COLLECT_MEMBER_UNRESOLVED alongside the intended ERR_ORIGIN_CARDINALITY. Conformance suite: 588 -> 606 pass (+18 = 9 fixtures x lint + conformance). Full metadata package: 2507 -> 2525 pass, 0 fail. --- fixtures/conformance/README.md | 45 +++++ .../collect-whole-object/expected.json | 142 ++++++++++++++++ .../collect-whole-object/input/meta.demo.json | 142 ++++++++++++++++ .../collect-whole-object/providers.json | 1 + .../expected-errors.json | 15 ++ .../input/meta.demo.json | 143 ++++++++++++++++ .../error-collect-distinct/providers.json | 1 + .../expected-errors.json | 15 ++ .../input/meta.demo.json | 142 ++++++++++++++++ .../providers.json | 1 + .../expected-errors.json | 15 ++ .../input/meta.demo.json | 142 ++++++++++++++++ .../providers.json | 1 + .../expected-errors.json | 25 +++ .../input/meta.demo.json | 141 ++++++++++++++++ .../providers.json | 1 + .../error-collect-no-via/expected-errors.json | 15 ++ .../error-collect-no-via/input/meta.demo.json | 141 ++++++++++++++++ .../error-collect-no-via/providers.json | 1 + .../expected-errors.json | 15 ++ .../input/meta.demo.json | 159 ++++++++++++++++++ .../providers.json | 1 + .../expected-errors.json | 15 ++ .../input/meta.demo.json | 142 ++++++++++++++++ .../providers.json | 1 + .../expected-errors.json | 15 ++ .../input/meta.demo.json | 137 +++++++++++++++ .../error-collect-via-to-one/providers.json | 1 + 28 files changed, 1615 insertions(+) create mode 100644 fixtures/conformance/collect-whole-object/expected.json create mode 100644 fixtures/conformance/collect-whole-object/input/meta.demo.json create mode 100644 fixtures/conformance/collect-whole-object/providers.json create mode 100644 fixtures/conformance/error-collect-distinct/expected-errors.json create mode 100644 fixtures/conformance/error-collect-distinct/input/meta.demo.json create mode 100644 fixtures/conformance/error-collect-distinct/providers.json create mode 100644 fixtures/conformance/error-collect-member-type-mismatch/expected-errors.json create mode 100644 fixtures/conformance/error-collect-member-type-mismatch/input/meta.demo.json create mode 100644 fixtures/conformance/error-collect-member-type-mismatch/providers.json create mode 100644 fixtures/conformance/error-collect-member-unresolved/expected-errors.json create mode 100644 fixtures/conformance/error-collect-member-unresolved/input/meta.demo.json create mode 100644 fixtures/conformance/error-collect-member-unresolved/providers.json create mode 100644 fixtures/conformance/error-collect-no-object-ref/expected-errors.json create mode 100644 fixtures/conformance/error-collect-no-object-ref/input/meta.demo.json create mode 100644 fixtures/conformance/error-collect-no-object-ref/providers.json create mode 100644 fixtures/conformance/error-collect-no-via/expected-errors.json create mode 100644 fixtures/conformance/error-collect-no-via/input/meta.demo.json create mode 100644 fixtures/conformance/error-collect-no-via/providers.json create mode 100644 fixtures/conformance/error-collect-orderby-not-terminal/expected-errors.json create mode 100644 fixtures/conformance/error-collect-orderby-not-terminal/input/meta.demo.json create mode 100644 fixtures/conformance/error-collect-orderby-not-terminal/providers.json create mode 100644 fixtures/conformance/error-collect-ref-not-value/expected-errors.json create mode 100644 fixtures/conformance/error-collect-ref-not-value/input/meta.demo.json create mode 100644 fixtures/conformance/error-collect-ref-not-value/providers.json create mode 100644 fixtures/conformance/error-collect-via-to-one/expected-errors.json create mode 100644 fixtures/conformance/error-collect-via-to-one/input/meta.demo.json create mode 100644 fixtures/conformance/error-collect-via-to-one/providers.json diff --git a/fixtures/conformance/README.md b/fixtures/conformance/README.md index d27e12532..5b658dc08 100644 --- a/fixtures/conformance/README.md +++ b/fixtures/conformance/README.md @@ -96,6 +96,47 @@ exercise an array field through the filter/sort tier, so they are what makes the rule (added to the TypeScript loader for #335) a cross-port contract rather than a TS-only unit test. +## Whole-object rollup (#335) + +`origin.aggregate @agg:collect` with `@of` omitted is a whole-object rollup: the +carrying field must be a `field.object @isArray @objectRef`, and the origin +assembles the related rows as an array of that declared value object — each +member matched by name against the `@via` path's TERMINAL entity — instead of +an array of one scalar column. + +- `collect-whole-object` — the positive case: a `field.object @isArray + @objectRef` collects the related entity's rows as an array of the declared + value object. **This is the corpus's only whole-object rollup** — every other + `@agg:collect` fixture (`origin-agg-collect`, `error-agg-collect-*`) collects a + single scalar column via `@of`. +- `error-collect-no-object-ref` — the carrying field declares no `@objectRef`. + Gates `ERR_INVALID_ORIGIN` (the whole-object-rollup rule itself) **and** + `ERR_OBJECT_FIELD_WITHOUT_OBJECT_REF` (a `field.object` always requires + `@objectRef`, independent of collect) — both fire on the same shape, so both + are asserted. +- `error-collect-ref-not-value` — `@objectRef` resolves to an `object.entity` + instead of an `object.value`. Gates `ERR_SUBTYPE_RULE_VIOLATION` (#210, + ADR-0028: a whole-object rollup must target a value). +- `error-collect-no-via` — `@via` is absent, and with no `@of` there is no + entity to infer the relationship from. Gates `ERR_INVALID_ORIGIN`. +- `error-collect-via-to-one` — every hop on `@via` is to-one (a passthrough + shape, not a rollup). Gates `ERR_ORIGIN_CARDINALITY`. +- `error-collect-distinct` — `@distinct` is declared on a whole-object rollup (a + no-op once the value object carries the primary key). Gates + `ERR_INVALID_ORIGIN`. +- `error-collect-member-unresolved` — a declared value-object member has no + matching field, by name, on the `@via` terminal entity. Gates + `ERR_COLLECT_MEMBER_UNRESOLVED` — failing open here is how #270 turned a + curated value object into the full entity, invisible in a diff. +- `error-collect-member-type-mismatch` — a declared value-object member's + `field.` differs from the matched terminal-entity field's. Gates + `ERR_INVALID_ORIGIN` (a whole-object rollup preserves each member's declared + type). +- `error-collect-orderby-not-terminal` — `@orderBy` names a field that exists on + a middle hop of a multi-hop `@via` path but not on the path's TERMINAL entity. + Gates `ERR_INVALID_ORIGIN` — `@orderBy` keys always resolve against the + terminal, never the head or an intermediate hop. + ## Adding a fixture Create a directory; add `input/` and expectation files. No runner code changes — @@ -120,6 +161,10 @@ carrying an origin. No surviving origin expresses a whole-object rollup along a relationship — `origin.aggregate @agg:collect` reduces a *column* via `@of`. That shape becomes expressible again when [#335](https://github.com/metaobjectsdev/metaobjects/issues/335) makes `@of` optional on `collect`; the fixture is the place to restore it. +**Restored** — see "Whole-object rollup (#335)" below: `collect-whole-object` is +a dedicated fixture, not a restoration of `flattened-kitchen-sink` itself (that +fixture stays scoped to its own concern), but it is the corpus's positive case +for exactly this shape. ## Generated fixtures (differential testing) diff --git a/fixtures/conformance/collect-whole-object/expected.json b/fixtures/conformance/collect-whole-object/expected.json new file mode 100644 index 000000000..76cedec37 --- /dev/null +++ b/fixtures/conformance/collect-whole-object/expected.json @@ -0,0 +1,142 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Supplier", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "suppliers" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + }, + { + "field.long": { + "name": "productId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "product", + "@fields": [ + "productId" + ], + "@references": "Product" + } + } + ] + } + }, + { + "object.entity": { + "name": "Product", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "products" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "relationship.association": { + "name": "suppliers", + "@cardinality": "many", + "@objectRef": "Supplier" + } + } + ] + } + }, + { + "object.value": { + "name": "SupplierBrief", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + } + ] + } + }, + { + "object.projection": { + "name": "ProductWithSuppliers", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_product_suppliers" + } + }, + { + "field.long": { + "name": "productId", + "extends": "Product.id" + } + }, + { + "identity.primary": { + "name": "id", + "extends": "Product.id" + } + }, + { + "field.object": { + "name": "supplierBriefs", + "isArray": true, + "@objectRef": "SupplierBrief", + "children": [ + { + "origin.aggregate": { + "@agg": "collect", + "@via": "Product.suppliers" + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/collect-whole-object/input/meta.demo.json b/fixtures/conformance/collect-whole-object/input/meta.demo.json new file mode 100644 index 000000000..c3ae29579 --- /dev/null +++ b/fixtures/conformance/collect-whole-object/input/meta.demo.json @@ -0,0 +1,142 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Supplier", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "suppliers" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + }, + { + "field.long": { + "name": "productId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "product", + "@references": "Product", + "@fields": [ + "productId" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Product", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "products" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "relationship.association": { + "name": "suppliers", + "@cardinality": "many", + "@objectRef": "Supplier" + } + } + ] + } + }, + { + "object.value": { + "name": "SupplierBrief", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + } + ] + } + }, + { + "object.projection": { + "name": "ProductWithSuppliers", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_product_suppliers" + } + }, + { + "field.long": { + "name": "productId", + "extends": "Product.id" + } + }, + { + "identity.primary": { + "name": "id", + "extends": "Product.id" + } + }, + { + "field.object": { + "name": "supplierBriefs", + "isArray": true, + "@objectRef": "SupplierBrief", + "children": [ + { + "origin.aggregate": { + "@agg": "collect", + "@via": "Product.suppliers" + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/collect-whole-object/providers.json b/fixtures/conformance/collect-whole-object/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/collect-whole-object/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-collect-distinct/expected-errors.json b/fixtures/conformance/error-collect-distinct/expected-errors.json new file mode 100644 index 000000000..3242747ca --- /dev/null +++ b/fixtures/conformance/error-collect-distinct/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_INVALID_ORIGIN", + "source": { + "format": "json", + "files": [ + "meta.demo.json" + ], + "jsonPath": "$['metadata.root'].children[3]['object.projection'].children[3]['field.object'].children[0]['origin.aggregate']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-collect-distinct/input/meta.demo.json b/fixtures/conformance/error-collect-distinct/input/meta.demo.json new file mode 100644 index 000000000..2bb30a4e6 --- /dev/null +++ b/fixtures/conformance/error-collect-distinct/input/meta.demo.json @@ -0,0 +1,143 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Supplier", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "suppliers" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + }, + { + "field.long": { + "name": "productId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "product", + "@references": "Product", + "@fields": [ + "productId" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Product", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "products" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "relationship.association": { + "name": "suppliers", + "@cardinality": "many", + "@objectRef": "Supplier" + } + } + ] + } + }, + { + "object.value": { + "name": "SupplierBrief", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + } + ] + } + }, + { + "object.projection": { + "name": "ProductWithSuppliers", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_product_suppliers" + } + }, + { + "field.long": { + "name": "productId", + "extends": "Product.id" + } + }, + { + "identity.primary": { + "name": "id", + "extends": "Product.id" + } + }, + { + "field.object": { + "name": "supplierBriefs", + "isArray": true, + "@objectRef": "SupplierBrief", + "children": [ + { + "origin.aggregate": { + "@agg": "collect", + "@via": "Product.suppliers", + "@distinct": true + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-collect-distinct/providers.json b/fixtures/conformance/error-collect-distinct/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-collect-distinct/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-collect-member-type-mismatch/expected-errors.json b/fixtures/conformance/error-collect-member-type-mismatch/expected-errors.json new file mode 100644 index 000000000..3242747ca --- /dev/null +++ b/fixtures/conformance/error-collect-member-type-mismatch/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_INVALID_ORIGIN", + "source": { + "format": "json", + "files": [ + "meta.demo.json" + ], + "jsonPath": "$['metadata.root'].children[3]['object.projection'].children[3]['field.object'].children[0]['origin.aggregate']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-collect-member-type-mismatch/input/meta.demo.json b/fixtures/conformance/error-collect-member-type-mismatch/input/meta.demo.json new file mode 100644 index 000000000..606ce3aad --- /dev/null +++ b/fixtures/conformance/error-collect-member-type-mismatch/input/meta.demo.json @@ -0,0 +1,142 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Supplier", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "suppliers" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + }, + { + "field.long": { + "name": "productId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "product", + "@references": "Product", + "@fields": [ + "productId" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Product", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "products" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "relationship.association": { + "name": "suppliers", + "@cardinality": "many", + "@objectRef": "Supplier" + } + } + ] + } + }, + { + "object.value": { + "name": "SupplierBrief", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.long": { + "name": "name" + } + } + ] + } + }, + { + "object.projection": { + "name": "ProductWithSuppliers", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_product_suppliers" + } + }, + { + "field.long": { + "name": "productId", + "extends": "Product.id" + } + }, + { + "identity.primary": { + "name": "id", + "extends": "Product.id" + } + }, + { + "field.object": { + "name": "supplierBriefs", + "isArray": true, + "@objectRef": "SupplierBrief", + "children": [ + { + "origin.aggregate": { + "@agg": "collect", + "@via": "Product.suppliers" + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-collect-member-type-mismatch/providers.json b/fixtures/conformance/error-collect-member-type-mismatch/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-collect-member-type-mismatch/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-collect-member-unresolved/expected-errors.json b/fixtures/conformance/error-collect-member-unresolved/expected-errors.json new file mode 100644 index 000000000..937d5ad1a --- /dev/null +++ b/fixtures/conformance/error-collect-member-unresolved/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_COLLECT_MEMBER_UNRESOLVED", + "source": { + "format": "json", + "files": [ + "meta.demo.json" + ], + "jsonPath": "$['metadata.root'].children[3]['object.projection'].children[3]['field.object'].children[0]['origin.aggregate']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-collect-member-unresolved/input/meta.demo.json b/fixtures/conformance/error-collect-member-unresolved/input/meta.demo.json new file mode 100644 index 000000000..11044defd --- /dev/null +++ b/fixtures/conformance/error-collect-member-unresolved/input/meta.demo.json @@ -0,0 +1,142 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Supplier", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "suppliers" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + }, + { + "field.long": { + "name": "productId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "product", + "@references": "Product", + "@fields": [ + "productId" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Product", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "products" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "relationship.association": { + "name": "suppliers", + "@cardinality": "many", + "@objectRef": "Supplier" + } + } + ] + } + }, + { + "object.value": { + "name": "SupplierBrief", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "nickname" + } + } + ] + } + }, + { + "object.projection": { + "name": "ProductWithSuppliers", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_product_suppliers" + } + }, + { + "field.long": { + "name": "productId", + "extends": "Product.id" + } + }, + { + "identity.primary": { + "name": "id", + "extends": "Product.id" + } + }, + { + "field.object": { + "name": "supplierBriefs", + "isArray": true, + "@objectRef": "SupplierBrief", + "children": [ + { + "origin.aggregate": { + "@agg": "collect", + "@via": "Product.suppliers" + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-collect-member-unresolved/providers.json b/fixtures/conformance/error-collect-member-unresolved/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-collect-member-unresolved/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-collect-no-object-ref/expected-errors.json b/fixtures/conformance/error-collect-no-object-ref/expected-errors.json new file mode 100644 index 000000000..22fabc0ce --- /dev/null +++ b/fixtures/conformance/error-collect-no-object-ref/expected-errors.json @@ -0,0 +1,25 @@ +{ + "errors": [ + { + "code": "ERR_INVALID_ORIGIN", + "source": { + "format": "json", + "files": [ + "meta.demo.json" + ], + "jsonPath": "$['metadata.root'].children[3]['object.projection'].children[3]['field.object'].children[0]['origin.aggregate']" + } + }, + { + "code": "ERR_OBJECT_FIELD_WITHOUT_OBJECT_REF", + "source": { + "format": "json", + "files": [ + "meta.demo.json" + ], + "jsonPath": "$['metadata.root'].children[3]['object.projection'].children[3]['field.object']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-collect-no-object-ref/input/meta.demo.json b/fixtures/conformance/error-collect-no-object-ref/input/meta.demo.json new file mode 100644 index 000000000..150ee27c2 --- /dev/null +++ b/fixtures/conformance/error-collect-no-object-ref/input/meta.demo.json @@ -0,0 +1,141 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Supplier", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "suppliers" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + }, + { + "field.long": { + "name": "productId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "product", + "@references": "Product", + "@fields": [ + "productId" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Product", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "products" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "relationship.association": { + "name": "suppliers", + "@cardinality": "many", + "@objectRef": "Supplier" + } + } + ] + } + }, + { + "object.value": { + "name": "SupplierBrief", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + } + ] + } + }, + { + "object.projection": { + "name": "ProductWithSuppliers", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_product_suppliers" + } + }, + { + "field.long": { + "name": "productId", + "extends": "Product.id" + } + }, + { + "identity.primary": { + "name": "id", + "extends": "Product.id" + } + }, + { + "field.object": { + "name": "supplierBriefs", + "isArray": true, + "children": [ + { + "origin.aggregate": { + "@agg": "collect", + "@via": "Product.suppliers" + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-collect-no-object-ref/providers.json b/fixtures/conformance/error-collect-no-object-ref/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-collect-no-object-ref/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-collect-no-via/expected-errors.json b/fixtures/conformance/error-collect-no-via/expected-errors.json new file mode 100644 index 000000000..3242747ca --- /dev/null +++ b/fixtures/conformance/error-collect-no-via/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_INVALID_ORIGIN", + "source": { + "format": "json", + "files": [ + "meta.demo.json" + ], + "jsonPath": "$['metadata.root'].children[3]['object.projection'].children[3]['field.object'].children[0]['origin.aggregate']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-collect-no-via/input/meta.demo.json b/fixtures/conformance/error-collect-no-via/input/meta.demo.json new file mode 100644 index 000000000..c33702111 --- /dev/null +++ b/fixtures/conformance/error-collect-no-via/input/meta.demo.json @@ -0,0 +1,141 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Supplier", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "suppliers" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + }, + { + "field.long": { + "name": "productId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "product", + "@references": "Product", + "@fields": [ + "productId" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Product", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "products" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "relationship.association": { + "name": "suppliers", + "@cardinality": "many", + "@objectRef": "Supplier" + } + } + ] + } + }, + { + "object.value": { + "name": "SupplierBrief", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + } + ] + } + }, + { + "object.projection": { + "name": "ProductWithSuppliers", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_product_suppliers" + } + }, + { + "field.long": { + "name": "productId", + "extends": "Product.id" + } + }, + { + "identity.primary": { + "name": "id", + "extends": "Product.id" + } + }, + { + "field.object": { + "name": "supplierBriefs", + "isArray": true, + "@objectRef": "SupplierBrief", + "children": [ + { + "origin.aggregate": { + "@agg": "collect" + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-collect-no-via/providers.json b/fixtures/conformance/error-collect-no-via/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-collect-no-via/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-collect-orderby-not-terminal/expected-errors.json b/fixtures/conformance/error-collect-orderby-not-terminal/expected-errors.json new file mode 100644 index 000000000..1c118fe75 --- /dev/null +++ b/fixtures/conformance/error-collect-orderby-not-terminal/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_INVALID_ORIGIN", + "source": { + "format": "json", + "files": [ + "meta.demo.json" + ], + "jsonPath": "$['metadata.root'].children[4]['object.projection'].children[3]['field.object'].children[0]['origin.aggregate']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-collect-orderby-not-terminal/input/meta.demo.json b/fixtures/conformance/error-collect-orderby-not-terminal/input/meta.demo.json new file mode 100644 index 000000000..aec49a446 --- /dev/null +++ b/fixtures/conformance/error-collect-orderby-not-terminal/input/meta.demo.json @@ -0,0 +1,159 @@ +{ + "metadata.root": { + "package": "acme::chain", + "children": [ + { + "object.entity": { + "name": "C", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "cs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "B", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "bs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + }, + { + "relationship.association": { + "name": "c", + "@cardinality": "many", + "@objectRef": "C" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "A", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "as" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "relationship.association": { + "name": "b", + "@cardinality": "many", + "@objectRef": "B" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.value": { + "name": "CBrief", + "children": [ + { + "field.long": { + "name": "id" + } + } + ] + } + }, + { + "object.projection": { + "name": "AWithCs", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_a_cs" + } + }, + { + "field.long": { + "name": "aId", + "extends": "A.id" + } + }, + { + "identity.primary": { + "name": "id", + "extends": "A.id" + } + }, + { + "field.object": { + "name": "items", + "isArray": true, + "@objectRef": "CBrief", + "children": [ + { + "origin.aggregate": { + "@agg": "collect", + "@via": "A.b.c", + "@orderBy": [ + "name" + ] + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-collect-orderby-not-terminal/providers.json b/fixtures/conformance/error-collect-orderby-not-terminal/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-collect-orderby-not-terminal/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-collect-ref-not-value/expected-errors.json b/fixtures/conformance/error-collect-ref-not-value/expected-errors.json new file mode 100644 index 000000000..833e2e33c --- /dev/null +++ b/fixtures/conformance/error-collect-ref-not-value/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_SUBTYPE_RULE_VIOLATION", + "source": { + "format": "json", + "files": [ + "meta.demo.json" + ], + "jsonPath": "$['metadata.root'].children[3]['object.projection'].children[3]['field.object'].children[0]['origin.aggregate']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-collect-ref-not-value/input/meta.demo.json b/fixtures/conformance/error-collect-ref-not-value/input/meta.demo.json new file mode 100644 index 000000000..5cbf9c32c --- /dev/null +++ b/fixtures/conformance/error-collect-ref-not-value/input/meta.demo.json @@ -0,0 +1,142 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Supplier", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "suppliers" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + }, + { + "field.long": { + "name": "productId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "product", + "@references": "Product", + "@fields": [ + "productId" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Product", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "products" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "relationship.association": { + "name": "suppliers", + "@cardinality": "many", + "@objectRef": "Supplier" + } + } + ] + } + }, + { + "object.value": { + "name": "SupplierBrief", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + } + ] + } + }, + { + "object.projection": { + "name": "ProductWithSuppliers", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_product_suppliers" + } + }, + { + "field.long": { + "name": "productId", + "extends": "Product.id" + } + }, + { + "identity.primary": { + "name": "id", + "extends": "Product.id" + } + }, + { + "field.object": { + "name": "supplierBriefs", + "isArray": true, + "@objectRef": "Supplier", + "children": [ + { + "origin.aggregate": { + "@agg": "collect", + "@via": "Product.suppliers" + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-collect-ref-not-value/providers.json b/fixtures/conformance/error-collect-ref-not-value/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-collect-ref-not-value/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-collect-via-to-one/expected-errors.json b/fixtures/conformance/error-collect-via-to-one/expected-errors.json new file mode 100644 index 000000000..cc0ee5ac1 --- /dev/null +++ b/fixtures/conformance/error-collect-via-to-one/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_ORIGIN_CARDINALITY", + "source": { + "format": "json", + "files": [ + "meta.demo.json" + ], + "jsonPath": "$['metadata.root'].children[3]['object.projection'].children[3]['field.object'].children[0]['origin.aggregate']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-collect-via-to-one/input/meta.demo.json b/fixtures/conformance/error-collect-via-to-one/input/meta.demo.json new file mode 100644 index 000000000..0e0df1320 --- /dev/null +++ b/fixtures/conformance/error-collect-via-to-one/input/meta.demo.json @@ -0,0 +1,137 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Supplier", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "suppliers" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + }, + { + "field.long": { + "name": "productId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "product", + "@references": "Product", + "@fields": [ + "productId" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Product", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "products" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "relationship.association": { + "name": "suppliers", + "@cardinality": "many", + "@objectRef": "Supplier" + } + } + ] + } + }, + { + "object.value": { + "name": "SupplierBrief", + "children": [ + { + "field.long": { + "name": "id" + } + } + ] + } + }, + { + "object.projection": { + "name": "ProductWithSuppliers", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_product_suppliers" + } + }, + { + "field.long": { + "name": "productId", + "extends": "Product.id" + } + }, + { + "identity.primary": { + "name": "id", + "extends": "Product.id" + } + }, + { + "field.object": { + "name": "supplierBriefs", + "isArray": true, + "@objectRef": "SupplierBrief", + "children": [ + { + "origin.aggregate": { + "@agg": "collect", + "@via": "Supplier.product" + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-collect-via-to-one/providers.json b/fixtures/conformance/error-collect-via-to-one/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-collect-via-to-one/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] From e4a80e2c7d12f875bf3f666e6fe89768080e2fad Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 23 Aug 2026 11:19:34 -0400 Subject: [PATCH 15/25] fix(metamodel): whole-object member agreement judges array-ness too (#335) The per-member type-agreement check added in Task 6 cites the #185 type-preserving doctrine but implemented only half of it: it compared `member.subType !== match.subType` and ignored array-ness entirely. A value-object member declared `field.string` matching a terminal-entity `field.string isArray: true` (or the reverse) passed validation, and the lowering would project an array into a scalar member. The sibling this rule names, `_checkPassthroughType`, already compares BOTH axes via the `field.[]` label and carries a comment explaining why subtype names never contain "[]", so equal labels mean same subType AND same array-ness. This check now does the same thing, so the two halves of one doctrine are enforced the same way. Found by the whole-branch review. Caught before Task 8 deliberately: porting a known-incomplete rule into three more ports and then fixing four is strictly worse than fixing the reference implementation first. Also corrects the ERR_FILTERABLE_UNSUPPORTED_SUBTYPE ledger description, which still described only the subtype cause after this branch gave the code a second trigger (array field) in all four ports. Its sibling ERR_SORTABLE_UNSUPPORTED_SUBTYPE already documented both. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq --- fixtures/conformance/ERROR-CODES.json | 2 +- .../metadata/src/loader/validation-passes.ts | 15 ++++++++++++--- .../validation-collect-whole-object.test.ts | 18 ++++++++++++++++++ 3 files changed, 31 insertions(+), 4 deletions(-) diff --git a/fixtures/conformance/ERROR-CODES.json b/fixtures/conformance/ERROR-CODES.json index 25688fe99..3a602dd79 100644 --- a/fixtures/conformance/ERROR-CODES.json +++ b/fixtures/conformance/ERROR-CODES.json @@ -69,7 +69,7 @@ "ERR_DISCRIMINATOR_VALUE_TYPE_MISMATCH": "FR-014: @discriminatorValue is not a member of the discriminator field's @values (when the discriminator field is field.enum) or cannot be coerced to its subtype.", "ERR_YAML_COERCION": "A YAML 1.2 silent type coercion produced a value whose runtime type differs from the attribute's declared valueType (e.g. an unquoted `column: TRUE` parsed as boolean for a string-typed attr). Emitted by every port's YAML loader; canonical JSON is unaffected.", "ERR_MERGE_CONFLICT": "FR5c: multi-file overlay merge produced an attribute conflict \u2014 two contributing files declared the same @attr with different non-empty values on the same logical node. Envelope is `format: \"merged\"` with `contributors[]` naming both files.", - "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE": "SP-H Unit9: a field carries @filterable: true but its subtype has no filter-operator band (e.g. field.object). Generating a filter for it would emit an empty operator set \u2014 a filter route that rejects every request. Use a filterable subtype (string/enum/uuid/number/currency/date/boolean) or remove @filterable.", + "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE": "#335 Half B / SP-H Unit9: a field carries @filterable: true but cannot be filtered — either it is an array (isArray: true; a collection column has no filter-operator band) or its subtype has no filter-operator band (e.g. field.object), the same subtype-support signal @sortable uses. Generating a filter for it would emit an empty operator set — a filter route that rejects every request. Use a filterable subtype (string/enum/uuid/number/currency/date/boolean), drop isArray, or remove @filterable.", "ERR_SORTABLE_UNSUPPORTED_SUBTYPE": "#335 Half B: a field carries @sortable: true but cannot be ordered \u2014 either it is an array (isArray: true; a collection column has no ordering) or its subtype has no filter-operator band (e.g. field.object), the same subtype-support signal @filterable uses (SP-H Unit9). Generating a sort entry for it would emit an ORDER BY no dialect can execute. Use a sortable subtype (string/enum/uuid/number/currency/date/boolean) or remove @sortable.", "ERR_COLLECT_MEMBER_UNRESOLVED": "#335 Half A: a whole-object origin.aggregate @agg:collect (no @of \u2014 the carrying field.object collects related rows as its declared @objectRef value object) has a value-object member with no matching field (by name) on the @via terminal entity. The lowering projects exactly the declared members; failing open here is how #270 turned a curated value object into the full entity, invisible in a diff because the metadata still read as curated.", "ERR_REGISTRY_SEALED": "ADR-0023: a registration (register / registerType / extend / registerCommonAttrs / setDefaultSubType / addConstraint / registerProviders) was attempted against a registry sealed after its agreed metamodel-provider bootstrap. Made-up metamodel attributes/types are structurally disallowed in the library: codegen cannot register post-bootstrap. A new metamodel attribute requires a registered provider + human agreement. Downstream apps that need extra vocabulary compose their own (unsealed) registry.", diff --git a/server/typescript/packages/metadata/src/loader/validation-passes.ts b/server/typescript/packages/metadata/src/loader/validation-passes.ts index fed3bb164..16eb4d741 100644 --- a/server/typescript/packages/metadata/src/loader/validation-passes.ts +++ b/server/typescript/packages/metadata/src/loader/validation-passes.ts @@ -1453,11 +1453,20 @@ export function validateOriginPaths(root: MetaData): ParseError[] { // the scalar element-type check below (#185 type-preserving // doctrine), reusing ERR_INVALID_ORIGIN for the same "types // disagree" shape rather than minting a second code. - if (member.subType !== match.subType) { + // Compare BOTH axes via the type-label, exactly as the + // sibling _checkPassthroughType does: subtype names never + // contain "[]", so equal labels ⇔ same subType AND same + // array-ness. Judging subType alone would let a scalar + // member match an array field on the terminal (and vice + // versa) and project an array into a scalar member. + // ADR-0039: resolvedIsArray(), never the own `isArray` flag. + const memberLabel = `field.${member.subType}${member.resolvedIsArray() ? "[]" : ""}`; + const matchLabel = `field.${match.subType}${match.resolvedIsArray() ? "[]" : ""}`; + if (memberLabel !== matchLabel) { errors.push(new ParseError( `origin.aggregate @agg:collect on ${obj.name}.${field.name}: value-object member ` + - `'${member.name}' is field.${member.subType} but '${terminal.name}.${match.name}' ` + - `is field.${match.subType} — a whole-object rollup preserves each member's type.`, + `'${member.name}' is ${memberLabel} but '${terminal.name}.${match.name}' ` + + `is ${matchLabel} — a whole-object rollup preserves each member's type.`, { code: "ERR_INVALID_ORIGIN", source: src })); } } diff --git a/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts b/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts index 97817509b..31f06d467 100644 --- a/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts +++ b/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts @@ -255,4 +255,22 @@ describe("@of-absent collect (whole-object rollup)", () => { expect(hit?.message).toContain("'name' is field.long"); expect(hit?.message).toContain("Supplier.name' is field.string"); }); + + test("a VO member whose ARRAY-NESS differs from the matched field fails", async () => { + // SupplierBrief declares "name" as field.string isArray; Supplier.name is a + // scalar field.string. Same subType, different array-ness — the #185 + // type-preserving doctrine this rule cites judges BOTH axes (the sibling + // _checkPassthroughType compares the `field.[]` label for exactly + // this reason). Comparing subType alone would project an array into a + // scalar member. + const src = model( + WHOLE_OBJECT, + `{ "field.long": { "name": "id" } }, { "field.string": { "name": "name", "isArray": true } }`, + ); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_INVALID_ORIGIN" && e.message.includes("value-object member")); + expect(hit).toBeDefined(); + expect(hit?.message).toContain("'name' is field.string[]"); + expect(hit?.message).toContain("Supplier.name' is field.string"); + }); }); From cfa5126240a9f0b19ca8b5f3fa81de205bb7a85a Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 23 Aug 2026 11:22:04 -0400 Subject: [PATCH 16/25] docs(plan): Task 8 ports EIGHT rules against NINE fixtures, not six/seven (#335) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Task 7 shipped eight rules and nine fixtures after a scope correction the ledger recorded but this plan never absorbed: member TYPE agreement and @orderBy-resolves- against-the-@via-terminal had no place in the original six. Following the stale text literally would have under-ported two rules into three ports — precisely the silent cross-port omission the shared corpus exists to prevent. Also records the trap that makes Task 8's own signal untrustworthy: five of the eight negative fixtures assert ERR_INVALID_ORIGIN, which is ALSO what an unported loader emits for the old "missing @of" error. Those five pass vacuously before any porting work begins, so "conformance green" does not mean the rules were implemented. The step now names the guard-neutralisation technique the Task 7 reviewer used, and repeats the no-git-stash prohibition (worktrees share one stash list). Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq --- ...026-08-22-issue-335-whole-object-rollup.md | 20 +++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/docs/superpowers/plans/2026-08-22-issue-335-whole-object-rollup.md b/docs/superpowers/plans/2026-08-22-issue-335-whole-object-rollup.md index c33ef4ee5..7316e998f 100644 --- a/docs/superpowers/plans/2026-08-22-issue-335-whole-object-rollup.md +++ b/docs/superpowers/plans/2026-08-22-issue-335-whole-object-rollup.md @@ -913,7 +913,9 @@ git commit -m "test(conformance): gate the whole-object rollup and its six error - Modify: `server/python/src/metaobjects/loader/validation_passes.py` **Interfaces:** -- Consumes: Tasks 5–7. The seven fixtures are the contract. +- Consumes: Tasks 5–7. The NINE fixtures are the contract (8 negative + 1 positive). + Task 7 shipped eight rules, not the six this plan was written against — the ledger + recorded the scope correction; this text was stale. Port all EIGHT. - Produces: four ports green on the whole corpus. - [ ] **Step 1: Locate each port's gate** @@ -924,9 +926,12 @@ grep -rn "missing @of" server/csharp server/java server/python Python phrases it differently — search for its `AGG_COLLECT` handling near the `collect produces a list` message instead. -- [ ] **Step 2: Implement the same six rules per port** +- [ ] **Step 2: Implement the same EIGHT rules per port** -Object-ref required; ref must be `object.value`; `@via` required; cardinality; member resolution; `@distinct` refused. Use each port's **resolving** accessors throughout (see Task 4, Step 2). +Object-ref required; ref must be `object.value`; `@via` required; cardinality; member +resolution; member TYPE agreement (subtype AND array-ness — compare the +`field.[]` label, as TS does); `@orderBy` keys resolve against the `@via` +TERMINAL entity (not the head or a middle hop); `@distinct` refused. Use each port's **resolving** accessors throughout (see Task 4, Step 2). - [ ] **Step 3: Run each port's conformance suite** @@ -936,7 +941,14 @@ cd server/csharp && dotnet build && dotnet test --filter "FullyQualifiedName~Con cd server/java && mvn -q -pl metadata test ``` -Expected: green on all seven new fixtures. Same two traps as Task 4, Step 4. +Expected: green on all nine new fixtures. Same two traps as Task 4, Step 4. + +**Green is NOT proof here.** Five of the eight negatives (no-via, no-object-ref, +distinct, member-type-mismatch, orderby-not-terminal) assert `ERR_INVALID_ORIGIN`, +which is ALSO what an unported loader emits for the old "missing @of" — so they pass +vacuously before you write a line. Verify each rule fires for its OWN reason: neutralise +one guard at a time (`if (false && )`) in a COPY under /tmp and re-run that +fixture. Never `git stash` — worktrees share one stash list. - [ ] **Step 4: Commit** From 2d79f5333bb74c82bafbaebd035595752b97a9ed Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 23 Aug 2026 11:32:43 -0400 Subject: [PATCH 17/25] refactor(metamodel): one @via walk, not three; extract the whole-object rules (#335) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit All four cleanup reviewers independently flagged the same thing, and the efficiency pass measured it: `_validateViaPath` already holds the terminal entity in `currentObj` when it returns, and throws it away — so `_viaTerminalEntityNode` existed only to recover it by walking the identical path a second time, and the whole-object branch called that helper TWICE. Three walks where one would do. On a 1103-object estate the two redundant walks measured ~56ms of a 778ms load — 7.2%. Worse than the cost: the duplicate re-derived the ADR-0042 package-resolution rule by hand, so the loader carried two independent @via path-walkers that had to be kept in agreement. The branch's own comments admitted the divergence ("_viaTerminalEntityNode does not reproduce _validateViaPath's malformed-shape guard") and both call sites needed a redundant `hops !== undefined` gate to paper over it. Those comments were the smell, not the invariant. `_validateViaPath` now returns `{ hops, terminal }` — the shape `_validateFromPath` already uses for the same reason. `terminal` is defined exactly when `hops` is (all five early exits return undefined; the sole success path has walked to the end), so the gate is structural rather than commented. `_viaTerminalEntityNode` is deleted: 44 lines, and three ports that no longer have to reimplement it in Task 8. Also extracted, because the helper set IS the cross-port porting contract — every port mirrors this file's `_check*`/`_validate*` decomposition 1:1, and Java's validateOriginNode is already ~330 lines: - `_checkCollectMembers` — the member-resolution + type-agreement rules. Measured nesting in the @of-absent arm drops from 10 levels to 4. - `_typeLabel` — the `field.[]` idiom, previously inline in four places with its invariant re-explained each time. `_checkPassthroughType` now shares it. Fixture fix (found by the altitude pass): error-collect-orderby-not-terminal declared `name` only on B, the MIDDLE hop. Task 7's review proved it discriminates middle-resolution — but a port resolving @orderBy against the HEAD (the likelier mistake, since the walk starts there) also errored and so PASSED. Entity A now declares `name` too. Verified by mutation: resolving against the head fails the fixture now and did not before. Python's `@sortable` check inlined the literal "sortable" where the other three ports used their constant; now imports FIELD_ATTR_SORTABLE. Verified: typecheck clean, 2526 pass / 0 fail, conformance 606/0. The extracted helper was mutation-checked (disabling it fails 3 unit tests + 2 fixtures, so it is reachable and pinned, not dead code). Python conformance is unchanged at 4 failed / 451 passed — the known interim red for Half A, which Task 8 ports. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq --- .../input/meta.demo.json | 5 + .../metaobjects/loader/validation_passes.py | 3 +- .../metadata/src/loader/validation-passes.ts | 226 ++++++++---------- 3 files changed, 111 insertions(+), 123 deletions(-) diff --git a/fixtures/conformance/error-collect-orderby-not-terminal/input/meta.demo.json b/fixtures/conformance/error-collect-orderby-not-terminal/input/meta.demo.json index aec49a446..209353eb3 100644 --- a/fixtures/conformance/error-collect-orderby-not-terminal/input/meta.demo.json +++ b/fixtures/conformance/error-collect-orderby-not-terminal/input/meta.demo.json @@ -81,6 +81,11 @@ "name": "id" } }, + { + "field.string": { + "name": "name" + } + }, { "relationship.association": { "name": "b", diff --git a/server/python/src/metaobjects/loader/validation_passes.py b/server/python/src/metaobjects/loader/validation_passes.py index 90addcc8a..9d0c5eea1 100644 --- a/server/python/src/metaobjects/loader/validation_passes.py +++ b/server/python/src/metaobjects/loader/validation_passes.py @@ -26,6 +26,7 @@ FIELD_ATTR_INT_VALUE_MAP, FIELD_ATTR_OBJECT_REF, FIELD_ATTR_REQUIRED, + FIELD_ATTR_SORTABLE, FIELD_ATTR_STORAGE, FIELD_ATTR_VALUE_TYPE, FIELD_ATTR_VALUES, @@ -3181,7 +3182,7 @@ def _validate_sortable_has_supported_subtype( if node.type != TYPE_OBJECT or not isinstance(node, MetaObject): continue for field in node.fields(): - if field.attrs().get("sortable") is not True: + if field.attrs().get(FIELD_ATTR_SORTABLE) is not True: continue # ADR-0039: resolved_is_array(), never the own `is_array` flag. is_array = field.resolved_is_array() diff --git a/server/typescript/packages/metadata/src/loader/validation-passes.ts b/server/typescript/packages/metadata/src/loader/validation-passes.ts index 16eb4d741..79b7a003c 100644 --- a/server/typescript/packages/metadata/src/loader/validation-passes.ts +++ b/server/typescript/packages/metadata/src/loader/validation-passes.ts @@ -649,10 +649,25 @@ function _validateFromPath( return { entity: sourceObj, field: sourceField }; } +/** A fully-walked `@via` path: the relationship hop nodes in path order, and + * the TERMINAL entity reached after the last hop. Both fall out of one walk, + * so they are returned together — recovering the terminal with a second walk + * means maintaining a second copy of the ADR-0042 package-resolution rule. + * Shape mirrors `_validateFromPath`'s `ResolvedFromTarget`, which returns a + * pair for the same reason. */ +interface WalkedViaPath { + hops: MetaData[]; + terminal: MetaData; +} + /** - * Validate an explicit `@via` "Entity.rel[.rel...]" path. Returns the walked - * relationship hop nodes (in path order) on full success — FR-024 B5 runs the - * cardinality checks over them — or undefined when any error was pushed. + * Validate an explicit `@via` "Entity.rel[.rel...]" path. On full success + * returns the walked relationship hop nodes (in path order) — FR-024 B5 runs + * the cardinality checks over them — together with the terminal entity node + * (#335: a whole-object `@agg:collect` has no `@of` entity, so `@orderBy` keys + * and value-object members resolve against the terminal instead). Returns + * undefined when any error was pushed, so `terminal` is defined exactly when + * `hops` is. */ function _validateViaPath( viaAttr: string, @@ -661,7 +676,7 @@ function _validateViaPath( fieldName: string, originSource: ErrorSource, errors: ParseError[], -): MetaData[] | undefined { +): WalkedViaPath | undefined { const projectionName = projection.name; // FR5d — referrer is `::`. const referrer = `${projection.fqn()}::${fieldName}`; @@ -763,51 +778,8 @@ function _validateViaPath( hops.push(rel); currentObj = nextObj; } - return hops; -} - -/** - * #335 — walk an explicit `@via` "Entity.rel[.rel...]" path and return its - * TERMINAL entity node (the entity reached after the last hop), or undefined - * if any segment fails to resolve. Mirrors `_validateViaPath`'s hop-walking - * exactly but returns the terminal object instead of the walked hop nodes, - * and pushes no errors — callers use it only once `_validateViaPath` has - * already validated the same path and returned successfully. - * - * A whole-object `@agg:collect` has no `@of` entity to resolve `@orderBy` - * keys against (the #195 scalar path resolves them against the `@of` column's - * entity); this is the terminal-entity equivalent for the whole-object path. - * Codegen-side equivalent: `codegen-ts/src/projection/extract-view-spec.ts`'s - * `viaTerminalEntity`. - */ -function _viaTerminalEntityNode( - viaAttr: string, - root: MetaData, - projection: MetaData, -): MetaData | undefined { - const segments = viaAttr.split("."); - const entityName = segments[0]; - if (entityName === undefined) return undefined; - // ADR-0042 — a bare @via HEAD resolves in the projection's package. - const referrerPkg = projection.package ?? projection.fileDefaultPackage ?? ""; - let currentObj = _findObject(root, entityName, referrerPkg); - if (!currentObj) return undefined; - for (const relName of segments.slice(1)) { - const rel = _findRelationship(currentObj, relName) ?? _findReference(currentObj, relName); - if (!rel) return undefined; - const refTarget = _hopTargetName(rel); - if (typeof refTarget !== "string" || refTarget === "") return undefined; - // ADR-0042 — the hop target resolves in the package of the entity that - // DECLARES the relationship/reference, i.e. currentObj. - const nextObj = _findObject( - root, - refTarget, - currentObj.package ?? currentObj.fileDefaultPackage ?? "", - ); - if (!nextObj) return undefined; - currentObj = nextObj; - } - return currentObj; + // currentObj is the terminal: every earlier exit returned undefined. + return { hops, terminal: currentObj }; } // --------------------------------------------------------------------------- @@ -1057,6 +1029,54 @@ function _checkAggregateCardinality( } } +/** + * #335 — a whole-object `@agg:collect` projects EXACTLY the declared value + * object's members, each matched by NAME against the `@via` terminal entity. + * + * Two rules, both fail-closed: + * - a member with no matching field on the terminal is unresolvable. Failing + * OPEN here is how #270 turned a curated value object into the full entity, + * invisible in a diff because the metadata still read as curated. + * - a matched member must agree on BOTH type axes (#185 type-preserving + * doctrine), so a scalar member cannot bind an array field or vice versa. + * + * Reuses ERR_INVALID_ORIGIN for the type disagreement rather than minting a + * second code — it is the same "types disagree" shape the scalar arm reports. + */ +function _checkCollectMembers( + refTarget: MetaData, + terminal: MetaData, + obj: MetaData, + field: MetaData, + src: ErrorSource, + errors: ParseError[], +): void { + // ADR-0039: resolving — a value object may inherit members via extends, and + // the terminal entity may inherit fields; own-only would silently skip + // inherited members, which is exactly the #270 bug class this guards. + const terminalFields = terminal.children().filter((c) => c.type === TYPE_FIELD); + for (const member of refTarget.children().filter((c) => c.type === TYPE_FIELD)) { + const match = terminalFields.find((f) => f.name === member.name); + if (match === undefined) { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: value-object member ` + + `'${member.name}' has no matching field on '${terminal.name}' — a whole-object ` + + `rollup projects exactly the declared members.`, + { code: "ERR_COLLECT_MEMBER_UNRESOLVED", source: src })); + continue; + } + const memberLabel = _typeLabel(member); + const matchLabel = _typeLabel(match); + if (memberLabel !== matchLabel) { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: value-object member ` + + `'${member.name}' is ${memberLabel} but '${terminal.name}.${match.name}' ` + + `is ${matchLabel} — a whole-object rollup preserves each member's type.`, + { code: "ERR_INVALID_ORIGIN", source: src })); + } + } +} + /** * FR-024 B6 (spec §4; ADR-0029 decision 7) — extends/origin agreement. * @@ -1122,6 +1142,17 @@ function _checkExtendsOriginAgreement( * FR-015 stored-proc parameter refs the retired ERR_PARAMETER_REF_PASSTHROUGH_ * TYPE_MISMATCH used to cover). */ +/** + * Both type axes in one comparable token. Subtype names never contain "[]", so + * equal labels ⇔ same subType AND same array-ness. Nullability is deliberately + * NOT judged — an outer-join view legitimately widens NOT NULL. + * ADR-0039: resolvedIsArray(), never the own `isArray` flag — a field may + * inherit its array-ness via extends. + */ +function _typeLabel(field: MetaData): string { + return `field.${field.subType}${field.resolvedIsArray() ? "[]" : ""}`; +} + function _checkPassthroughType( field: MetaData, fromField: MetaData, @@ -1132,11 +1163,8 @@ function _checkPassthroughType( errors: ParseError[], ): void { if (convert) return; // deliberate type change acknowledged - // Compare both axes at once via the type-label: subtype names never contain - // "[]", so equal labels ⇔ same subType AND same array-ness (nullability is - // deliberately not judged — an outer-join view legitimately widens NOT NULL). - const declared = `field.${field.subType}${field.resolvedIsArray() ? "[]" : ""}`; - const source = `field.${fromField.subType}${fromField.resolvedIsArray() ? "[]" : ""}`; + const declared = _typeLabel(field); + const source = _typeLabel(fromField); if (declared === source) return; errors.push( new ParseError( @@ -1257,9 +1285,9 @@ export function validateOriginPaths(root: MetaData): ParseError[] { // ADR-0039: own — origin.* never inherits (ADR-0029). const via = origin.ownAttr(ORIGIN_PASSTHROUGH_ATTR_VIA); if (typeof via === "string" && via !== "") { - const hops = _validateViaPath(via, root, obj, field.name, origin.source, errors); - if (hops !== undefined) { - _checkPassthroughCardinality(hops, obj, field.name, origin.source, errors); + const walked = _validateViaPath(via, root, obj, field.name, origin.source, errors); + if (walked !== undefined) { + _checkPassthroughCardinality(walked.hops, obj, field.name, origin.source, errors); } } else if (fromTarget !== undefined && !isValueHost) { // FR-024 §6 — no @via: derive the base entity; a @from targeting @@ -1344,8 +1372,8 @@ export function validateOriginPaths(root: MetaData): ParseError[] { `origin.aggregate @agg:${String(agg)} on ${obj.name}.${field.name}: requires an explicit @via (a quantifier has no @of to infer the path from).`, { code: "ERR_INVALID_ORIGIN", source: src })); } else { - const hops = _validateViaPath(via, root, obj, field.name, src, errors); - if (hops !== undefined) _checkAggregateCardinality(hops, obj, field.name, src, errors); + const walked = _validateViaPath(via, root, obj, field.name, src, errors); + if (walked !== undefined) _checkAggregateCardinality(walked.hops, obj, field.name, src, errors); } continue; } @@ -1412,64 +1440,18 @@ export function validateOriginPaths(root: MetaData): ParseError[] { { code: "ERR_INVALID_ORIGIN", source: src })); continue; } - const hops = _validateViaPath(viaAttr, root, obj, field.name, src, errors); - if (hops !== undefined) _checkAggregateCardinality(hops, obj, field.name, src, errors); - // @orderBy keys resolve against the @via TERMINAL entity, not @of. - // Gated on hops !== undefined — _viaTerminalEntityNode does not - // reproduce _validateViaPath's malformed-shape guard, so an - // invalid @via (e.g. single-segment "A") must not also emit a - // second, misleadingly-scoped @orderBy error. - if (hasOrderBy && hops !== undefined) { - const terminal = _viaTerminalEntityNode(viaAttr, root, obj); - _validateOrderByKeys(orderBy, terminal, obj, field.name, "origin.aggregate @agg:collect", src, errors); - } - // Member resolution — the lowering projects EXACTLY the declared - // value object's members, matched by NAME against the @via - // TERMINAL entity. An unmatched member must error: failing open - // here is how #270 turned a curated value object into the full - // entity, invisible in a diff because the metadata still read as - // curated. Gated on hops !== undefined for the same reason the - // @orderBy check above is: _viaTerminalEntityNode does not - // reproduce _validateViaPath's malformed-shape guard. - if (hops !== undefined) { - const terminal = _viaTerminalEntityNode(viaAttr, root, obj); - if (terminal !== undefined && refTarget !== undefined) { - // ADR-0039: resolving — a value object may inherit members via - // extends, and the terminal entity may inherit fields; - // own-only would silently skip inherited members, which is - // exactly the #270 bug class this guard exists to prevent. - const terminalFields = terminal.children().filter((c) => c.type === TYPE_FIELD); - for (const member of refTarget.children().filter((c) => c.type === TYPE_FIELD)) { - const match = terminalFields.find((f) => f.name === member.name); - if (match === undefined) { - errors.push(new ParseError( - `origin.aggregate @agg:collect on ${obj.name}.${field.name}: value-object member ` + - `'${member.name}' has no matching field on '${terminal.name}' — a whole-object ` + - `rollup projects exactly the declared members.`, - { code: "ERR_COLLECT_MEMBER_UNRESOLVED", source: src })); - continue; - } - // Per-member type agreement — the object-form analogue of - // the scalar element-type check below (#185 type-preserving - // doctrine), reusing ERR_INVALID_ORIGIN for the same "types - // disagree" shape rather than minting a second code. - // Compare BOTH axes via the type-label, exactly as the - // sibling _checkPassthroughType does: subtype names never - // contain "[]", so equal labels ⇔ same subType AND same - // array-ness. Judging subType alone would let a scalar - // member match an array field on the terminal (and vice - // versa) and project an array into a scalar member. - // ADR-0039: resolvedIsArray(), never the own `isArray` flag. - const memberLabel = `field.${member.subType}${member.resolvedIsArray() ? "[]" : ""}`; - const matchLabel = `field.${match.subType}${match.resolvedIsArray() ? "[]" : ""}`; - if (memberLabel !== matchLabel) { - errors.push(new ParseError( - `origin.aggregate @agg:collect on ${obj.name}.${field.name}: value-object member ` + - `'${member.name}' is ${memberLabel} but '${terminal.name}.${match.name}' ` + - `is ${matchLabel} — a whole-object rollup preserves each member's type.`, - { code: "ERR_INVALID_ORIGIN", source: src })); - } - } + // One walk yields both the hops (cardinality) and the terminal + // entity (@orderBy keys, member resolution). An invalid @via + // (e.g. single-segment "A") returns undefined having already + // pushed its own error, so everything downstream is skipped and + // no second, misleadingly-scoped error is emitted. + const via = _validateViaPath(viaAttr, root, obj, field.name, src, errors); + if (via !== undefined) { + _checkAggregateCardinality(via.hops, obj, field.name, src, errors); + // @orderBy keys resolve against the @via TERMINAL entity, not @of. + _validateOrderByKeys(orderBy, via.terminal, obj, field.name, "origin.aggregate @agg:collect", src, errors); + if (refTarget !== undefined) { + _checkCollectMembers(refTarget, via.terminal, obj, field, src, errors); } } continue; @@ -1492,9 +1474,9 @@ export function validateOriginPaths(root: MetaData): ParseError[] { // ADR-0039: own — origin.* never inherits (ADR-0029). const via = origin.ownAttr(ORIGIN_AGGREGATE_ATTR_VIA); if (typeof via === "string" && via !== "") { - const hops = _validateViaPath(via, root, obj, field.name, src, errors); - if (hops !== undefined) { - _checkAggregateCardinality(hops, obj, field.name, src, errors); + const walked = _validateViaPath(via, root, obj, field.name, src, errors); + if (walked !== undefined) { + _checkAggregateCardinality(walked.hops, obj, field.name, src, errors); } continue; } @@ -1587,8 +1569,8 @@ export function validateOriginPaths(root: MetaData): ParseError[] { // @via — explicit (validated + cardinality) or single-hop-unique inferred. const via = origin.ownAttr(ORIGIN_FIRST_ATTR_VIA); if (typeof via === "string" && via !== "") { - const hops = _validateViaPath(via, root, obj, field.name, src, errors); - if (hops !== undefined) _checkAggregateCardinality(hops, obj, field.name, src, errors); + const walked = _validateViaPath(via, root, obj, field.name, src, errors); + if (walked !== undefined) _checkAggregateCardinality(walked.hops, obj, field.name, src, errors); } else if (ofTarget !== undefined) { // (A value host never reaches here — the #210 assembly-origin // check above already rejected origin.first on a value.) From 3c900abbbc4c11374088ecfa04068a56d24dd542 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 23 Aug 2026 14:52:16 -0400 Subject: [PATCH 18/25] chore(metamodel)!: metamodelVersion 0.11 -> 0.12 for #335 (#335) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit #335 changes the registered vocabulary: `@of` becomes OPTIONAL on `origin.aggregate @agg: collect`, which is a rule relaxation an adopter can observe, plus eight new load errors. Under ADR-0035 Amendment 2 that moves the metamodel contract's number, and pre-1.0 a breaking metamodel change moves the MINOR. It is 0.12, NOT the 0.11 the design spec's §A7 claimed. 0.11 is CLAIMED — by #342's XOR relaxation, shipping in 0.24.1; origin/main's 2af77e7cd names it the sole claimant precisely so a parallel branch would not reason "v0.24.0 shipped 0.10, so 0.11 is free" and collide. This branch already carries #342 in its ancestry, which is what its 0.11 was recording; #335's own change needs the next number. No gate would have caught this. #335's only registry footprint is `@of`'s byte-gated DESCRIPTION PROSE, and check-metamodel-version.mjs WARNs on prose rather than classifying it — a rule can change with no machine-readable footprint (#210's only manifest edit was a `rules` string). Confirmed empirically: the gate reported "declared 0.11 — ok" before this commit and "declared 0.12 — ok" after. Human decision, made by Doug. Written by `check-metamodel-version.mjs --set 0.12` so all five sites move together — the manifest plus all four port constants. A partial edit only fails in the forgotten port's registry-conformance lane, which is release-tag/dispatch-only for three of them. Verified: all five sites read 0.12 with no residual 0.11, gate green, typecheck clean, metadata 2526/0, Python registry-conformance 10/10. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq --- fixtures/registry-conformance/expected-registry.json | 2 +- server/csharp/MetaObjects/RegistryManifest.cs | 2 +- .../main/java/com/metaobjects/registry/RegistryManifest.java | 2 +- server/python/src/metaobjects/registry_manifest.py | 2 +- server/typescript/packages/metadata/src/registry-manifest.ts | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) diff --git a/fixtures/registry-conformance/expected-registry.json b/fixtures/registry-conformance/expected-registry.json index 6a8b468b0..41a088f38 100644 --- a/fixtures/registry-conformance/expected-registry.json +++ b/fixtures/registry-conformance/expected-registry.json @@ -1,5 +1,5 @@ { - "metamodelVersion": "0.11", + "metamodelVersion": "0.12", "types": [ { "type": "attr", diff --git a/server/csharp/MetaObjects/RegistryManifest.cs b/server/csharp/MetaObjects/RegistryManifest.cs index 2728360dd..36e93443a 100644 --- a/server/csharp/MetaObjects/RegistryManifest.cs +++ b/server/csharp/MetaObjects/RegistryManifest.cs @@ -374,7 +374,7 @@ private static ManifestType ToManifestType(TypeDefinition def) => /// unstable (semver major-0). The 1.0 cut flips this to "1.0". /// Emitted as the FIRST top-level key in the registry manifest (C4). /// - public const string MetamodelVersion = "0.11"; + public const string MetamodelVersion = "0.12"; /// /// Emit the canonical registry manifest as a byte-stable JSON string. diff --git a/server/java/metadata/src/main/java/com/metaobjects/registry/RegistryManifest.java b/server/java/metadata/src/main/java/com/metaobjects/registry/RegistryManifest.java index 6488804d7..48ca69dc1 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/registry/RegistryManifest.java +++ b/server/java/metadata/src/main/java/com/metaobjects/registry/RegistryManifest.java @@ -274,7 +274,7 @@ public static List metamodelProviders() { * the 1.0 cut will flip it to {@code "1.0"}. Mirrors the TS reference's * {@code METAMODEL_VERSION} constant. */ - public static final String METAMODEL_VERSION = "0.11"; + public static final String METAMODEL_VERSION = "0.12"; // ------------------------------------------------------------------ // SP-G Phase1 Units2-3 — manifest emitter exclusions (documented, uniform diff --git a/server/python/src/metaobjects/registry_manifest.py b/server/python/src/metaobjects/registry_manifest.py index 741efafd0..905b2a3fb 100644 --- a/server/python/src/metaobjects/registry_manifest.py +++ b/server/python/src/metaobjects/registry_manifest.py @@ -255,7 +255,7 @@ def _sorted_children(definition: TypeDefinition) -> list[dict[str, object]]: return children -METAMODEL_VERSION = "0.11" +METAMODEL_VERSION = "0.12" """Rolled-up spec-version for the cross-port registry manifest. ``"0"`` = pre-1.0 / unstable (semver major-0). Flips to ``"1.0"`` at the diff --git a/server/typescript/packages/metadata/src/registry-manifest.ts b/server/typescript/packages/metadata/src/registry-manifest.ts index c346cb734..dbbf6fade 100644 --- a/server/typescript/packages/metadata/src/registry-manifest.ts +++ b/server/typescript/packages/metadata/src/registry-manifest.ts @@ -112,7 +112,7 @@ interface ManifestType { * constant read `"0.10"`). Bump with that script — never by hand — so the manifest and * all four port constants move together. */ -export const METAMODEL_VERSION = "0.11"; +export const METAMODEL_VERSION = "0.12"; /** The full canonical manifest. All collections are sorted for byte-stability. */ interface RegistryManifest { From 34aa5f0e3b34362abda81c0f5f21fbdef7ad709d Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 23 Aug 2026 15:21:28 -0400 Subject: [PATCH 19/25] feat(metamodel): the whole-object arm gets its own error code (#335) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Five of the eight negative conformance fixtures for the @of-absent @agg:collect asserted ERR_INVALID_ORIGIN — which is ALSO what a loader that still REQUIRES @of emits for that exact metadata. The corpus compares only `code` + `source` and never message text (spec/conformance-tests.md), so those five passed vacuously against three ports that implement none of the rules: Python conformance failed 4 of the 9, not 9 of the 9. ERR_COLLECT_WHOLE_OBJECT now carries the five refusals raised by the whole-object arm itself — carrier is not a field.object declaring @objectRef; @via absent; @distinct declared; an @orderBy key that does not resolve against the @via TERMINAL entity; a value-object member whose declared field.[] disagrees with the matched terminal field's. The three arms that already had distinct codes keep them (ERR_SUBTYPE_RULE_VIOLATION for a non-value @objectRef, ERR_ORIGIN_CARDINALITY for a to-one @via, ERR_COLLECT_MEMBER_UNRESOLVED for an unmatched member). The @orderBy arm runs through _validateOrderByKeys, shared with the scalar @of arm and with origin.first, so the helper gained a trailing `code: ErrorCode = "ERR_INVALID_ORIGIN"` parameter. Only the whole-object call site passes the new code; the other two keep their envelope byte-for-byte, so no existing fixture, port or adopter-visible behaviour changes. Registered in all SIX ledgers (ERROR-CODES.json, the TS/Python/C#/Java enums, and Java's separate ErrorMessageConstants). C#'s FixtureLint rejects an unregistered code in a fixture, which is independent proof the registration took in the shared ledger. Signal verified by execution on every unported port, not by reading: Python 9 failed / 568 passed (was 4 failed / 568 passed), C# 9 failed / 979 passed, Java 606 run / 9 failures — in each case exactly the nine #335 fixtures, so Task 8 now has a real red-before/green-after gate per rule. TS is unchanged green: metadata 2526/0, conformance 606/0, typecheck clean, dotnet build with no `error CS`, `mvn -pl metadata compile` clean. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq --- ...026-08-22-issue-335-whole-object-rollup.md | 81 +++++++++++++++++-- fixtures/conformance/ERROR-CODES.json | 1 + .../expected-errors.json | 2 +- .../expected-errors.json | 2 +- .../expected-errors.json | 2 +- .../error-collect-no-via/expected-errors.json | 2 +- .../expected-errors.json | 2 +- server/csharp/MetaObjects/Errors.cs | 8 ++ .../main/java/com/metaobjects/ErrorCode.java | 12 +++ .../util/ErrorMessageConstants.java | 13 +++ server/python/src/metaobjects/errors.py | 8 ++ .../packages/metadata/src/errors.ts | 8 ++ .../metadata/src/loader/validation-passes.ts | 30 ++++--- .../validation-collect-whole-object.test.ts | 24 +++--- 14 files changed, 165 insertions(+), 30 deletions(-) diff --git a/docs/superpowers/plans/2026-08-22-issue-335-whole-object-rollup.md b/docs/superpowers/plans/2026-08-22-issue-335-whole-object-rollup.md index 7316e998f..fcc861a73 100644 --- a/docs/superpowers/plans/2026-08-22-issue-335-whole-object-rollup.md +++ b/docs/superpowers/plans/2026-08-22-issue-335-whole-object-rollup.md @@ -905,6 +905,71 @@ git commit -m "test(conformance): gate the whole-object rollup and its six error --- +## Task 7b: Half A — mint `ERR_COLLECT_WHOLE_OBJECT` so the fixtures discriminate + +**Added after Task 7, ruled by Doug 2026-08-23.** Not in the original plan. + +**Files:** +- Modify: the six error ledgers — `fixtures/conformance/ERROR-CODES.json`, + `server/typescript/packages/metadata/src/errors.ts`, + `server/python/src/metaobjects/errors.py`, `server/csharp/MetaObjects/Errors.cs`, + `server/java/.../com/metaobjects/ErrorCode.java`, + `server/java/.../com/metaobjects/util/ErrorMessageConstants.java` +- Modify: `server/typescript/packages/metadata/src/loader/validation-passes.ts` +- Modify: the five fixtures' `expected-errors.json` +- Modify: `server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts` + +**Interfaces:** +- Consumes: Tasks 5–7 (the eight rules and the nine fixtures). +- Produces: a corpus that goes RED on an unported port, which is Task 8's signal. + +**Why:** five of the eight negative fixtures assert `ERR_INVALID_ORIGIN` — the same code +a loader that still REQUIRES `@of` emits for that same metadata. The corpus compares only +`code` + `source` (`spec/conformance-tests.md` §"Errors and warnings"; *"Error message text +is never compared"*), so those five pass **vacuously** on a port that implements nothing. +Verified: Python conformance failed 4 of the 9 before this task. + +**Scope — the five arms that move**, all inside the `@of`-absent branch: +carrier-not-`field.object`/no-`@objectRef`; `@via` required; `@distinct` refused; +`@orderBy` key not on the `@via` TERMINAL entity; value-object member type disagreement. +Rules 2/5/7 keep their existing distinct codes (`ERR_SUBTYPE_RULE_VIOLATION`, +`ERR_ORIGIN_CARDINALITY`, `ERR_COLLECT_MEMBER_UNRESOLVED`) — untouched. + +- [x] **Step 1: Six ledgers.** Mirror the `ERR_COLLECT_MEMBER_UNRESOLVED` entries added in + `e67cd9340`. Java needs BOTH `ErrorCode.java` and `ErrorMessageConstants.java`. + The TS ledger test compares SORTED sets, so placement is free — keep it beside its sibling. + +- [x] **Step 2: The `@orderBy` arm goes through a SHARED helper.** `_validateOrderByKeys` is + also called by the scalar `@of` arm and by `origin.first`. Give it a trailing + `code: ErrorCode = "ERR_INVALID_ORIGIN"` parameter and pass the new code from the + whole-object call site only — the other two call sites keep their envelope byte-for-byte. + +- [x] **Step 3: Retarget the five fixtures.** In `error-collect-no-object-ref` change only the + FIRST entry; `ERR_OBJECT_FIELD_WITHOUT_OBJECT_REF` stays second (the envelope assertion is + positional, and all four ports run origin validation before field-object-storage validation). + +- [x] **Step 4: Prove the signal is real.** Run an UNPORTED port's corpus and confirm the + failure count rises from 4 to 9. + +```bash +cd server/python && uv run --extra integration pytest -q -k conformance +``` + +- [x] **Step 5: Commit** + +```bash +git add fixtures/conformance/ERROR-CODES.json fixtures/conformance/error-collect-* \ + server/typescript/packages/metadata/src/errors.ts \ + server/typescript/packages/metadata/src/loader/validation-passes.ts \ + server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts \ + server/python/src/metaobjects/errors.py server/csharp/MetaObjects/Errors.cs \ + server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java \ + server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java +git commit -m "feat(metamodel): the whole-object arm gets its own error code (#335)" +``` + +--- + ## Task 8: Half A — port the loader rules to C#, Java, Python **Files:** @@ -943,12 +1008,16 @@ cd server/java && mvn -q -pl metadata test Expected: green on all nine new fixtures. Same two traps as Task 4, Step 4. -**Green is NOT proof here.** Five of the eight negatives (no-via, no-object-ref, -distinct, member-type-mismatch, orderby-not-terminal) assert `ERR_INVALID_ORIGIN`, -which is ALSO what an unported loader emits for the old "missing @of" — so they pass -vacuously before you write a line. Verify each rule fires for its OWN reason: neutralise -one guard at a time (`if (false && )`) in a COPY under /tmp and re-run that -fixture. Never `git stash` — worktrees share one stash list. +**Task 7b closed the vacuous-pass hole.** Those five negatives used to assert +`ERR_INVALID_ORIGIN` — what an unported loader emits for the old "missing @of" — so they +passed before a line was written. They now assert `ERR_COLLECT_WHOLE_OBJECT`, and all NINE +fail on an unported port (Python: 9 failed / 568 passed, measured). So the corpus IS a +red-before/green-after signal per rule. + +It still is not proof that each fixture fails for its OWN reason rather than a neighbour's +guard. Verify that the way Task 7's review did: neutralise one guard at a time +(`if (false && )`) in a COPY under /tmp and re-run that fixture. Never `git stash` — +worktrees share one stash list, so a stash op reaches other sessions' work. - [ ] **Step 4: Commit** diff --git a/fixtures/conformance/ERROR-CODES.json b/fixtures/conformance/ERROR-CODES.json index 3a602dd79..410b0e94e 100644 --- a/fixtures/conformance/ERROR-CODES.json +++ b/fixtures/conformance/ERROR-CODES.json @@ -71,6 +71,7 @@ "ERR_MERGE_CONFLICT": "FR5c: multi-file overlay merge produced an attribute conflict \u2014 two contributing files declared the same @attr with different non-empty values on the same logical node. Envelope is `format: \"merged\"` with `contributors[]` naming both files.", "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE": "#335 Half B / SP-H Unit9: a field carries @filterable: true but cannot be filtered — either it is an array (isArray: true; a collection column has no filter-operator band) or its subtype has no filter-operator band (e.g. field.object), the same subtype-support signal @sortable uses. Generating a filter for it would emit an empty operator set — a filter route that rejects every request. Use a filterable subtype (string/enum/uuid/number/currency/date/boolean), drop isArray, or remove @filterable.", "ERR_SORTABLE_UNSUPPORTED_SUBTYPE": "#335 Half B: a field carries @sortable: true but cannot be ordered \u2014 either it is an array (isArray: true; a collection column has no ordering) or its subtype has no filter-operator band (e.g. field.object), the same subtype-support signal @filterable uses (SP-H Unit9). Generating a sort entry for it would emit an ORDER BY no dialect can execute. Use a sortable subtype (string/enum/uuid/number/currency/date/boolean) or remove @sortable.", + "ERR_COLLECT_WHOLE_OBJECT": "#335 Half A: a whole-object origin.aggregate @agg:collect \u2014 one with no @of, which rolls the related rows up as the carrying field's declared @objectRef value object rather than as one scalar column \u2014 is malformed. Raised when the carrying field is not a field.object declaring @objectRef; when @via is absent (there is no @of entity to infer the relationship from); when @distinct is declared (refused \u2014 a guaranteed no-op whenever the value object carries the primary key, and a silent no-op is worse than a refusal); when an @orderBy key does not resolve against the @via TERMINAL entity; or when a value-object member's declared field.[] disagrees with the matched terminal field's. Distinct from ERR_INVALID_ORIGIN so a conformance fixture can tell \"the whole-object arm rejected this\" from \"a loader that still requires @of rejected this\" \u2014 indistinguishable otherwise, since the corpus compares only code + source envelope.", "ERR_COLLECT_MEMBER_UNRESOLVED": "#335 Half A: a whole-object origin.aggregate @agg:collect (no @of \u2014 the carrying field.object collects related rows as its declared @objectRef value object) has a value-object member with no matching field (by name) on the @via terminal entity. The lowering projects exactly the declared members; failing open here is how #270 turned a curated value object into the full entity, invisible in a diff because the metadata still read as curated.", "ERR_REGISTRY_SEALED": "ADR-0023: a registration (register / registerType / extend / registerCommonAttrs / setDefaultSubType / addConstraint / registerProviders) was attempted against a registry sealed after its agreed metamodel-provider bootstrap. Made-up metamodel attributes/types are structurally disallowed in the library: codegen cannot register post-bootstrap. A new metamodel attribute requires a registered provider + human agreement. Downstream apps that need extra vocabulary compose their own (unsealed) registry.", "ERR_UNKNOWN": "An internal loader error with no stable error code.", diff --git a/fixtures/conformance/error-collect-distinct/expected-errors.json b/fixtures/conformance/error-collect-distinct/expected-errors.json index 3242747ca..f87765e1a 100644 --- a/fixtures/conformance/error-collect-distinct/expected-errors.json +++ b/fixtures/conformance/error-collect-distinct/expected-errors.json @@ -1,7 +1,7 @@ { "errors": [ { - "code": "ERR_INVALID_ORIGIN", + "code": "ERR_COLLECT_WHOLE_OBJECT", "source": { "format": "json", "files": [ diff --git a/fixtures/conformance/error-collect-member-type-mismatch/expected-errors.json b/fixtures/conformance/error-collect-member-type-mismatch/expected-errors.json index 3242747ca..f87765e1a 100644 --- a/fixtures/conformance/error-collect-member-type-mismatch/expected-errors.json +++ b/fixtures/conformance/error-collect-member-type-mismatch/expected-errors.json @@ -1,7 +1,7 @@ { "errors": [ { - "code": "ERR_INVALID_ORIGIN", + "code": "ERR_COLLECT_WHOLE_OBJECT", "source": { "format": "json", "files": [ diff --git a/fixtures/conformance/error-collect-no-object-ref/expected-errors.json b/fixtures/conformance/error-collect-no-object-ref/expected-errors.json index 22fabc0ce..f07822cbc 100644 --- a/fixtures/conformance/error-collect-no-object-ref/expected-errors.json +++ b/fixtures/conformance/error-collect-no-object-ref/expected-errors.json @@ -1,7 +1,7 @@ { "errors": [ { - "code": "ERR_INVALID_ORIGIN", + "code": "ERR_COLLECT_WHOLE_OBJECT", "source": { "format": "json", "files": [ diff --git a/fixtures/conformance/error-collect-no-via/expected-errors.json b/fixtures/conformance/error-collect-no-via/expected-errors.json index 3242747ca..f87765e1a 100644 --- a/fixtures/conformance/error-collect-no-via/expected-errors.json +++ b/fixtures/conformance/error-collect-no-via/expected-errors.json @@ -1,7 +1,7 @@ { "errors": [ { - "code": "ERR_INVALID_ORIGIN", + "code": "ERR_COLLECT_WHOLE_OBJECT", "source": { "format": "json", "files": [ diff --git a/fixtures/conformance/error-collect-orderby-not-terminal/expected-errors.json b/fixtures/conformance/error-collect-orderby-not-terminal/expected-errors.json index 1c118fe75..ebe223470 100644 --- a/fixtures/conformance/error-collect-orderby-not-terminal/expected-errors.json +++ b/fixtures/conformance/error-collect-orderby-not-terminal/expected-errors.json @@ -1,7 +1,7 @@ { "errors": [ { - "code": "ERR_INVALID_ORIGIN", + "code": "ERR_COLLECT_WHOLE_OBJECT", "source": { "format": "json", "files": [ diff --git a/server/csharp/MetaObjects/Errors.cs b/server/csharp/MetaObjects/Errors.cs index 8bcc36200..1f92c4527 100644 --- a/server/csharp/MetaObjects/Errors.cs +++ b/server/csharp/MetaObjects/Errors.cs @@ -177,6 +177,14 @@ public enum ErrorCode // filter-operator band (e.g. field.object). Would silently emit a sort // entry over a column no dialect can ORDER BY. ERR_SORTABLE_UNSUPPORTED_SUBTYPE, + // #335 Half A — a whole-object @agg:collect (no @of; the carrying field.object + // rolls related rows up as its declared @objectRef value object) is malformed: + // carrier is not a field.object with @objectRef, @via absent, @distinct declared + // (refused — a no-op whenever the value object carries the primary key), an + // @orderBy key not on the @via TERMINAL entity, or a member's declared type + // disagreeing with the matched terminal field's. Distinct from ERR_INVALID_ORIGIN + // so a fixture can tell this arm from a loader that still requires @of. + ERR_COLLECT_WHOLE_OBJECT, // #335 Half A — a whole-object @agg:collect's value-object member has no // matching field (by name) on the @via terminal entity. The lowering // projects exactly the declared members; failing open here is how #270 diff --git a/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java b/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java index 5b527b6ed..3bdb034a1 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java +++ b/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java @@ -374,6 +374,18 @@ public enum ErrorCode { */ ERR_SORTABLE_UNSUPPORTED_SUBTYPE, + /** + * #335 Half A — a whole-object {@code @agg:collect} (no {@code @of}; the carrying + * {@code field.object} rolls related rows up as its declared {@code @objectRef} value + * object) is malformed: the carrier is not a {@code field.object} with {@code @objectRef}, + * {@code @via} is absent, {@code @distinct} is declared (refused — a no-op whenever the + * value object carries the primary key), an {@code @orderBy} key does not resolve against + * the {@code @via} TERMINAL entity, or a member's declared type disagrees with the matched + * terminal field's. Distinct from {@code ERR_INVALID_ORIGIN} so a conformance fixture can + * tell this arm from a loader that still requires {@code @of}. + */ + ERR_COLLECT_WHOLE_OBJECT, + /** * #335 Half A — a whole-object {@code @agg:collect}'s value-object member has no * matching field (by name) on the {@code @via} terminal entity. The lowering diff --git a/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java b/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java index ff715ea0b..20aa9f624 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java +++ b/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java @@ -218,6 +218,19 @@ private ErrorMessageConstants() { */ public static final String ERR_SORTABLE_UNSUPPORTED_SUBTYPE = "ERR_SORTABLE_UNSUPPORTED_SUBTYPE"; + /** + * #335 Half A — a whole-object {@code @agg:collect} (no {@code @of}; the carrying + * {@code field.object} rolls related rows up as its declared {@code @objectRef} value + * object) is malformed: the carrier is not a {@code field.object} with {@code @objectRef}, + * {@code @via} is absent, {@code @distinct} is declared (refused — a no-op whenever the + * value object carries the primary key), an {@code @orderBy} key does not resolve against + * the {@code @via} TERMINAL entity, or a member's declared type disagrees with the matched + * terminal field's. Distinct from {@code ERR_INVALID_ORIGIN} so a conformance fixture can + * tell this arm from a loader that still requires {@code @of}. + * Cross-language contract: {@code ERR_COLLECT_WHOLE_OBJECT}. + */ + public static final String ERR_COLLECT_WHOLE_OBJECT = "ERR_COLLECT_WHOLE_OBJECT"; + /** * #335 Half A — a whole-object {@code @agg:collect}'s value-object member has no * matching field (by name) on the {@code @via} terminal entity. The lowering diff --git a/server/python/src/metaobjects/errors.py b/server/python/src/metaobjects/errors.py index e5d6724d7..ed6b4459f 100644 --- a/server/python/src/metaobjects/errors.py +++ b/server/python/src/metaobjects/errors.py @@ -160,6 +160,14 @@ class ErrorCode(str, Enum): # filter-operator band (e.g. field.object). Would silently emit a sort # entry over a column no dialect can ORDER BY. ERR_SORTABLE_UNSUPPORTED_SUBTYPE = "ERR_SORTABLE_UNSUPPORTED_SUBTYPE" + # #335 Half A — a whole-object @agg:collect (no @of; the carrying field.object + # rolls related rows up as its declared @objectRef value object) is malformed: + # carrier is not a field.object with @objectRef, @via absent, @distinct declared + # (refused — a no-op whenever the value object carries the primary key), an + # @orderBy key not on the @via TERMINAL entity, or a member's declared type + # disagreeing with the matched terminal field's. Distinct from ERR_INVALID_ORIGIN + # so a fixture can tell this arm from a loader that still requires @of. + ERR_COLLECT_WHOLE_OBJECT = "ERR_COLLECT_WHOLE_OBJECT" # #335 Half A — a whole-object @agg:collect's value-object member has no # matching field (by name) on the @via terminal entity. The lowering # projects exactly the declared members; failing open here is how #270 diff --git a/server/typescript/packages/metadata/src/errors.ts b/server/typescript/packages/metadata/src/errors.ts index e9f39db3b..ac3d934fb 100644 --- a/server/typescript/packages/metadata/src/errors.ts +++ b/server/typescript/packages/metadata/src/errors.ts @@ -163,6 +163,14 @@ export const ERROR_CODES = [ // filter-operator band (e.g. field.object). Would silently emit a sort // entry over a column no dialect can ORDER BY. "ERR_SORTABLE_UNSUPPORTED_SUBTYPE", + // #335 Half A — a whole-object @agg:collect (no @of; the carrying field.object + // rolls related rows up as its declared @objectRef value object) is malformed: + // carrier is not a field.object with @objectRef, @via absent, @distinct declared + // (refused — a no-op whenever the value object carries the primary key), an + // @orderBy key not on the @via TERMINAL entity, or a member's declared type + // disagreeing with the matched terminal field's. Distinct from ERR_INVALID_ORIGIN + // so a fixture can tell this arm from a loader that still requires @of. + "ERR_COLLECT_WHOLE_OBJECT", // #335 Half A — a whole-object @agg:collect's value-object member has no // matching field (by name) on the @via terminal entity. The lowering // projects exactly the declared members; failing open here is how #270 diff --git a/server/typescript/packages/metadata/src/loader/validation-passes.ts b/server/typescript/packages/metadata/src/loader/validation-passes.ts index 79b7a003c..3f4c57777 100644 --- a/server/typescript/packages/metadata/src/loader/validation-passes.ts +++ b/server/typescript/packages/metadata/src/loader/validation-passes.ts @@ -13,7 +13,7 @@ import type { MetaData } from "../shared/meta-data.js"; import type { MetaObject } from "../core/object/meta-object.js"; import type { MetaReferenceIdentity } from "../core/identity/meta-identity.js"; -import { ParseError } from "../errors.js"; +import { ParseError, type ErrorCode } from "../errors.js"; import { resolveObjectRef, didYouMeanHint } from "../naming-refs.js"; import { PACKAGE_SEPARATOR, CHILD_REF_SEPARATOR } from "../shared/structural.js"; import { resolvedSource, type ErrorSource } from "../source.js"; @@ -1040,8 +1040,12 @@ function _checkAggregateCardinality( * - a matched member must agree on BOTH type axes (#185 type-preserving * doctrine), so a scalar member cannot bind an array field or vice versa. * - * Reuses ERR_INVALID_ORIGIN for the type disagreement rather than minting a - * second code — it is the same "types disagree" shape the scalar arm reports. + * Both refusals carry a whole-object-specific code — ERR_COLLECT_MEMBER_UNRESOLVED + * for the unmatched member, ERR_COLLECT_WHOLE_OBJECT for the type disagreement. + * The latter is deliberately NOT the scalar arm's ERR_INVALID_ORIGIN: a loader + * that still requires @of rejects this metadata with ERR_INVALID_ORIGIN too, so + * sharing the code would make a corpus fixture pass on a port that implements + * nothing (the corpus compares only code + source, never message text). */ function _checkCollectMembers( refTarget: MetaData, @@ -1072,7 +1076,7 @@ function _checkCollectMembers( `origin.aggregate @agg:collect on ${obj.name}.${field.name}: value-object member ` + `'${member.name}' is ${memberLabel} but '${terminal.name}.${match.name}' ` + `is ${matchLabel} — a whole-object rollup preserves each member's type.`, - { code: "ERR_INVALID_ORIGIN", source: src })); + { code: "ERR_COLLECT_WHOLE_OBJECT", source: src })); } } } @@ -1186,6 +1190,10 @@ function _checkPassthroughType( * and carries no vocabulary. Shared by `@agg:collect` (element order) and * `origin.first` (row selection). A missing related entity means a prior error * already fired — skip silently. + * + * `code` lets the whole-object `@agg:collect` arm report ERR_COLLECT_WHOLE_OBJECT + * instead; it defaults to ERR_INVALID_ORIGIN so the scalar `@of` and `origin.first` + * call sites keep their existing envelope byte-for-byte. */ function _validateOrderByKeys( orderBy: unknown, @@ -1195,6 +1203,7 @@ function _validateOrderByKeys( label: string, originSource: ErrorSource, errors: ParseError[], + code: ErrorCode = "ERR_INVALID_ORIGIN", ): void { if (!Array.isArray(orderBy) || relatedEntity === undefined) return; for (const raw of orderBy) { @@ -1208,14 +1217,14 @@ function _validateOrderByKeys( errors.push( new ParseError( `${label} on ${obj.name}.${fieldName}: @orderBy key "${raw}" — no such field "${key}" on ${relatedEntity.name}.`, - { code: "ERR_INVALID_ORIGIN", source: originSource }, + { code, source: originSource }, ), ); } else if (dir !== undefined && !(SORT_ORDER_VALUES as readonly string[]).includes(dir)) { errors.push( new ParseError( `${label} on ${obj.name}.${fieldName}: @orderBy key "${raw}" — direction must be one of ${SORT_ORDER_VALUES.join("|")}.`, - { code: "ERR_INVALID_ORIGIN", source: originSource }, + { code, source: originSource }, ), ); } @@ -1399,7 +1408,7 @@ export function validateOriginPaths(root: MetaData): ParseError[] { `origin.aggregate @agg:collect on ${obj.name}.${field.name}: @of is omitted, so this is a ` + `whole-object rollup — the carrying field must be a field.object declaring @objectRef ` + `(add @of to collect a single column instead).`, - { code: "ERR_INVALID_ORIGIN", source: src })); + { code: "ERR_COLLECT_WHOLE_OBJECT", source: src })); continue; } // #210's value-only rule is PAYLOAD-scoped and never reaches a @@ -1426,7 +1435,7 @@ export function validateOriginPaths(root: MetaData): ParseError[] { errors.push(new ParseError( `origin.aggregate @agg:collect on ${obj.name}.${field.name}: @via is required on a ` + `whole-object rollup — there is no @of entity to infer the relationship from.`, - { code: "ERR_INVALID_ORIGIN", source: src })); + { code: "ERR_COLLECT_WHOLE_OBJECT", source: src })); continue; } // @distinct is refused on the object form. It is NOT an engine limit @@ -1437,7 +1446,7 @@ export function validateOriginPaths(root: MetaData): ParseError[] { errors.push(new ParseError( `origin.aggregate @agg:collect on ${obj.name}.${field.name}: @distinct is not supported on a ` + `whole-object rollup (it is a no-op whenever the value object carries the primary key).`, - { code: "ERR_INVALID_ORIGIN", source: src })); + { code: "ERR_COLLECT_WHOLE_OBJECT", source: src })); continue; } // One walk yields both the hops (cardinality) and the terminal @@ -1449,7 +1458,8 @@ export function validateOriginPaths(root: MetaData): ParseError[] { if (via !== undefined) { _checkAggregateCardinality(via.hops, obj, field.name, src, errors); // @orderBy keys resolve against the @via TERMINAL entity, not @of. - _validateOrderByKeys(orderBy, via.terminal, obj, field.name, "origin.aggregate @agg:collect", src, errors); + _validateOrderByKeys(orderBy, via.terminal, obj, field.name, "origin.aggregate @agg:collect", src, errors, + "ERR_COLLECT_WHOLE_OBJECT"); if (refTarget !== undefined) { _checkCollectMembers(refTarget, via.terminal, obj, field, src, errors); } diff --git a/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts b/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts index 31f06d467..835dec71b 100644 --- a/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts +++ b/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts @@ -29,9 +29,15 @@ // _validateViaPath/_checkAggregateCardinality call AND the entire // `_viaTerminalEntityNode` call site (every caller of the helper) still // passed all six original tests. Two arms added below close that gap, and -// the three arms that previously discriminated only by the shared -// ERR_INVALID_ORIGIN code now also assert each rule's distinctive message -// fragment. +// the three arms that previously discriminated only by a shared code now +// also assert each rule's distinctive message fragment. +// +// The five arms that once shared ERR_INVALID_ORIGIN now carry +// ERR_COLLECT_WHOLE_OBJECT. ERR_INVALID_ORIGIN is what a loader still +// REQUIRING @of emits for this same metadata, so the shared code could not +// tell "the whole-object arm rejected this" from "nothing is implemented" — +// which is precisely how five of the shared conformance fixtures passed +// against three unported ports. import { describe, test, expect } from "bun:test"; import { MetaDataLoader } from "../src/loader/meta-data-loader.js"; @@ -145,7 +151,7 @@ describe("@of-absent collect (whole-object rollup)", () => { "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers" } } ] }}`); const errors = await loadErrors(src); - const hit = errors.find((e) => e.code === "ERR_INVALID_ORIGIN"); + const hit = errors.find((e) => e.code === "ERR_COLLECT_WHOLE_OBJECT"); expect(hit).toBeDefined(); expect(hit?.message).toContain("supplierBriefs"); expect(hit?.message).toContain("must be a field.object"); @@ -168,7 +174,7 @@ describe("@of-absent collect (whole-object rollup)", () => { "children": [ { "origin.aggregate": { "@agg": "collect" } } ] }}`); const errors = await loadErrors(src); - const hit = errors.find((e) => e.code === "ERR_INVALID_ORIGIN"); + const hit = errors.find((e) => e.code === "ERR_COLLECT_WHOLE_OBJECT"); expect(hit).toBeDefined(); expect(hit?.message).toContain("supplierBriefs"); expect(hit?.message).toContain("@via is required"); @@ -180,7 +186,7 @@ describe("@of-absent collect (whole-object rollup)", () => { "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers", "@distinct": true } } ] }}`); const errors = await loadErrors(src); - const hit = errors.find((e) => e.code === "ERR_INVALID_ORIGIN"); + const hit = errors.find((e) => e.code === "ERR_COLLECT_WHOLE_OBJECT"); expect(hit).toBeDefined(); expect(hit?.message).toContain("supplierBriefs"); expect(hit?.message).toContain("@distinct is not supported"); @@ -211,7 +217,7 @@ describe("@of-absent collect (whole-object rollup)", () => { // regression that stops at the first hop would name "B", and deleting // the `if (hasOrderBy...)` call site entirely emits no such error at all. const errors = await loadErrors(CHAIN_MODEL); - const hit = errors.find((e) => e.code === "ERR_INVALID_ORIGIN" && e.message.includes("@orderBy")); + const hit = errors.find((e) => e.code === "ERR_COLLECT_WHOLE_OBJECT" && e.message.includes("@orderBy")); expect(hit).toBeDefined(); expect(hit?.message).toContain('no such field "name" on C'); }); @@ -250,7 +256,7 @@ describe("@of-absent collect (whole-object rollup)", () => { `{ "field.long": { "name": "id" } }, { "field.long": { "name": "name" } }`, ); const errors = await loadErrors(src); - const hit = errors.find((e) => e.code === "ERR_INVALID_ORIGIN" && e.message.includes("value-object member")); + const hit = errors.find((e) => e.code === "ERR_COLLECT_WHOLE_OBJECT" && e.message.includes("value-object member")); expect(hit).toBeDefined(); expect(hit?.message).toContain("'name' is field.long"); expect(hit?.message).toContain("Supplier.name' is field.string"); @@ -268,7 +274,7 @@ describe("@of-absent collect (whole-object rollup)", () => { `{ "field.long": { "name": "id" } }, { "field.string": { "name": "name", "isArray": true } }`, ); const errors = await loadErrors(src); - const hit = errors.find((e) => e.code === "ERR_INVALID_ORIGIN" && e.message.includes("value-object member")); + const hit = errors.find((e) => e.code === "ERR_COLLECT_WHOLE_OBJECT" && e.message.includes("value-object member")); expect(hit).toBeDefined(); expect(hit?.message).toContain("'name' is field.string[]"); expect(hit?.message).toContain("Supplier.name' is field.string"); From 03a0c2ecf1b0111eaae54a1d3e9a6af0dfbd0086 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 23 Aug 2026 15:36:43 -0400 Subject: [PATCH 20/25] feat(metamodel): whole-object rollup in the remaining three ports (#335) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Ports Half A's eight loader rules for an @of-absent origin.aggregate @agg:collect to C#, Java and Python. The nine shared conformance fixtures are the contract; all four ports now agree on every one. The eight rules, in the order they fire: the carrying field must be a field.object declaring @objectRef (ERR_COLLECT_WHOLE_OBJECT); that @objectRef must resolve to an object.value (ERR_SUBTYPE_RULE_VIOLATION — #210's rule is payload-scoped and never reaches a projection-hosted field, so this branch enforces it itself); @via is required (ERR_COLLECT_WHOLE_OBJECT); @distinct is refused (ERR_COLLECT_WHOLE_OBJECT); the @via path must be to-many (ERR_ORIGIN_CARDINALITY); every @orderBy key must resolve against the @via TERMINAL entity (ERR_COLLECT_WHOLE_OBJECT); every value-object member must match a terminal field by name (ERR_COLLECT_MEMBER_UNRESOLVED); and a matched member must agree on BOTH type axes (ERR_COLLECT_WHOLE_OBJECT). Each port mirrors the TS decomposition 1:1 rather than inlining, because the helper set IS the porting contract and Java's validateOriginNode was already ~330 lines: WalkedViaPath (validateViaPath now returns hops AND the terminal, so the terminal is not recovered by a second walk carrying a second copy of the ADR-0042 package-resolution rule), _checkCollectMembers, and _typeLabel. validateOrderByKeys gains an optional error code — Java as a 7-arg overload, C# and Python as a defaulted parameter — so only the whole-object call site reports ERR_COLLECT_WHOLE_OBJECT while the scalar @of and origin.first sites keep their envelope byte-for-byte. ADR-0039 throughout: resolving reads for @objectRef (C# Attr, Java hasMetaAttr/getMetaAttr, Python get_meta_attr), resolving children() on both the value object and the terminal entity, and the resolving array accessor inside the type label (C# ResolvedIsArray, Java isArrayType, Python resolved_is_array). @via stays an OWN read — origin.* never inherits (ADR-0029). Green is not the evidence here, and this cut does not rely on it. Every one of the 24 (8 rules x 3 ports) guards was neutralised individually against a backed-up copy and the corpus re-run: in all 24 cases EXACTLY the one fixture that gates that rule failed, and no other. Each file was restored and `diff -q` confirmed byte-identical to its backup afterwards; no `git stash` was used, since worktrees share one stash list. Two C# mutations initially tripped CS0162 under warnings-as-errors, so those were re-run with a runtime-opaque condition rather than a statically dead one — a detail worth keeping: a neutralisation that will not compile proves nothing. Suites: Python 1849 passed / 0 failed; C# 1696 passed / 0 failed across all four test projects with no `error CS`; Java 1494 run / 0 failures / BUILD SUCCESS; TypeScript unchanged at 2526 / 0 with typecheck clean. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq --- .../MetaObjects/Loader/ValidationPasses.cs | 181 ++++++++++++++-- .../metaobjects/loader/ValidationPhase.java | 195 ++++++++++++++++-- .../metaobjects/loader/validation_passes.py | 189 +++++++++++++++-- 3 files changed, 510 insertions(+), 55 deletions(-) diff --git a/server/csharp/MetaObjects/Loader/ValidationPasses.cs b/server/csharp/MetaObjects/Loader/ValidationPasses.cs index 11d6649cf..dc99fb002 100644 --- a/server/csharp/MetaObjects/Loader/ValidationPasses.cs +++ b/server/csharp/MetaObjects/Loader/ValidationPasses.cs @@ -523,9 +523,9 @@ public static IReadOnlyList ValidateOriginPaths(MetaData root) var viaObj = origin.OwnAttr(ORIGIN_PASSTHROUGH_ATTR_VIA); if (viaObj is string via && via != "") { - var hops = ValidateViaPath(via, root, obj, field.Name, errors, origin.Source); - if (hops is not null) - CheckPassthroughCardinality(hops, obj, field.Name, origin.Source, errors); + var walked = ValidateViaPath(via, root, obj, field.Name, errors, origin.Source); + if (walked is WalkedViaPath wp) + CheckPassthroughCardinality(wp.Hops, obj, field.Name, origin.Source, errors); } else if (fromTarget is ResolvedFromTarget ft2 && !isValueHost) { @@ -606,19 +606,95 @@ public static IReadOnlyList ValidateOriginPaths(MetaData root) ErrorCode.ERR_INVALID_ORIGIN, Envelope: src)); else { - var predHops = ValidateViaPath(predVia, root, obj, field.Name, errors, src); - if (predHops is not null) CheckAggregateCardinality(predHops, obj, field.Name, src, errors); + var predWalked = ValidateViaPath(predVia, root, obj, field.Name, errors, src); + if (predWalked is WalkedViaPath pw) CheckAggregateCardinality(pw.Hops, obj, field.Name, src, errors); } continue; } - // --- count/sum/avg/min/max/collect: @of REQUIRED --- + // --- @of: REQUIRED for count/sum/avg/min/max; OPTIONAL for collect --- + // #335 — an @of-absent collect is a WHOLE-OBJECT rollup: collect the + // related rows as an array of the field's declared @objectRef value + // object rather than an array of one scalar column. if (!ofPresent) { - // Missing-attr — keep origin's own source envelope. - errors.Add(new MetaError( - $"origin.aggregate on {obj.Name}.{field.Name}: missing @of.", - ErrorCode.ERR_INVALID_ORIGIN, Envelope: src)); + if (!isCollect) + { + // Missing-attr — keep origin's own source envelope. + errors.Add(new MetaError( + $"origin.aggregate on {obj.Name}.{field.Name}: missing @of.", + ErrorCode.ERR_INVALID_ORIGIN, Envelope: src)); + continue; + } + // Whole-object rollup. The carrying field must be a field.object + // naming a value object, and @via must be explicit (there is no @of + // entity to infer the single-hop relation from). + // ADR-0039: resolving — @objectRef may be inherited via extends. + var objectRefObj = field.Attr(FIELD_ATTR_OBJECT_REF); + if (field.SubType != FIELD_SUBTYPE_OBJECT + || objectRefObj is not string objectRef || objectRef == "") + { + errors.Add(new MetaError( + $"origin.aggregate @agg:collect on {obj.Name}.{field.Name}: @of is omitted, so this is a " + + "whole-object rollup — the carrying field must be a field.object declaring @objectRef " + + "(add @of to collect a single column instead).", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, Envelope: src)); + continue; + } + // #210's value-only rule is PAYLOAD-scoped and never reaches a + // projection-hosted field, so this branch enforces it itself. + // Without it an @objectRef to an entity silently rolls up the FULL + // entity — the #270 shape, this time baked into DDL. + // ADR-0042 — a bare @objectRef resolves in the DECLARING owner's + // package (an inherited field resolves in the package that declared it). + var refTarget = FindObject(root, objectRef, + NamingRefs.EffectivePackage(field.Parent ?? obj)); + if (refTarget is not null && refTarget.SubType != OBJECT_SUBTYPE_VALUE) + { + errors.Add(new MetaError( + $"origin.aggregate @agg:collect on {obj.Name}.{field.Name}: @objectRef '{objectRef}' " + + $"resolves to {TYPE_OBJECT}.{refTarget.SubType} — a whole-object rollup must target an " + + "object.value (#210, ADR-0028).", + ErrorCode.ERR_SUBTYPE_RULE_VIOLATION, Envelope: src)); + continue; + } + // ADR-0039: own — origin.* never inherits (ADR-0029). + if (origin.OwnAttr(ORIGIN_AGGREGATE_ATTR_VIA) is not string woVia || woVia == "") + { + errors.Add(new MetaError( + $"origin.aggregate @agg:collect on {obj.Name}.{field.Name}: @via is required on a " + + "whole-object rollup — there is no @of entity to infer the relationship from.", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, Envelope: src)); + continue; + } + // @distinct is refused on the object form. It is NOT an engine limit + // (both engines dedupe JSON objects); it is a guaranteed no-op + // whenever the value object carries the entity's primary key, which + // is the common case, and a silent no-op is worse than a refusal. + if (hasDistinct) + { + errors.Add(new MetaError( + $"origin.aggregate @agg:collect on {obj.Name}.{field.Name}: @distinct is not supported on a " + + "whole-object rollup (it is a no-op whenever the value object carries the primary key).", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, Envelope: src)); + continue; + } + // One walk yields both the hops (cardinality) and the terminal entity + // (@orderBy keys, member resolution). An invalid @via (e.g. a single- + // segment "A") returns null having already pushed its own error, so + // everything downstream is skipped and no second, misleadingly-scoped + // error is emitted. + var woWalked = ValidateViaPath(woVia, root, obj, field.Name, errors, src); + if (woWalked is WalkedViaPath ww) + { + CheckAggregateCardinality(ww.Hops, obj, field.Name, src, errors); + // @orderBy keys resolve against the @via TERMINAL entity, not @of. + ValidateOrderByKeys(orderBy, ww.Terminal, obj, field.Name, + "origin.aggregate @agg:collect", src, errors, + ErrorCode.ERR_COLLECT_WHOLE_OBJECT); + if (refTarget is not null) + CheckCollectMembers(refTarget, ww.Terminal, obj, field, src, errors); + } continue; } string of = (string)ofObj!; @@ -640,9 +716,9 @@ public static IReadOnlyList ValidateOriginPaths(MetaData root) var viaObj = origin.OwnAttr(ORIGIN_AGGREGATE_ATTR_VIA); if (viaObj is string via && via != "") { - var hops = ValidateViaPath(via, root, obj, field.Name, errors, src); - if (hops is not null) - CheckAggregateCardinality(hops, obj, field.Name, src, errors); + var walked = ValidateViaPath(via, root, obj, field.Name, errors, src); + if (walked is WalkedViaPath aw) + CheckAggregateCardinality(aw.Hops, obj, field.Name, src, errors); continue; } // FR-024 §6 — no @via on an aggregate: inference applies only @@ -736,8 +812,8 @@ public static IReadOnlyList ValidateOriginPaths(MetaData root) var viaObj = origin.OwnAttr(ORIGIN_FIRST_ATTR_VIA); if (viaObj is string via && via != "") { - var hops = ValidateViaPath(via, root, obj, field.Name, errors, src); - if (hops is not null) CheckAggregateCardinality(hops, obj, field.Name, src, errors); + var walked = ValidateViaPath(via, root, obj, field.Name, errors, src); + if (walked is WalkedViaPath fw) CheckAggregateCardinality(fw.Hops, obj, field.Name, src, errors); } else if (ofTarget is ResolvedFromTarget inferTarget) { @@ -775,9 +851,67 @@ public static IReadOnlyList ValidateOriginPaths(MetaData root) /// order) and origin.first (row selection). A null related entity means a /// prior error already fired — skip silently. Mirrors TS _validateOrderByKeys. /// + /// A field's declared type on BOTH axes — field.<subType> plus [] when it + /// is an array. ADR-0039: ResolvedIsArray(), so array-ness inherited via extends counts. + /// Mirrors the TS _typeLabel. + private static string TypeLabel(MetaData field) + => $"field.{field.SubType}{(field.ResolvedIsArray() ? "[]" : "")}"; + + /// #335 — a whole-object @agg:collect projects EXACTLY the declared value object's + /// members, matched BY NAME against the @via terminal entity's fields: + /// + /// an unmatched member is an error, never a silent drop. Failing open here is how + /// #270 turned a curated value object into the full entity, invisible in a diff because + /// the metadata still read as curated. + /// a matched member must agree on BOTH type axes (#185 type-preserving doctrine), + /// so a scalar member cannot bind an array field or vice versa. + /// + /// Both refusals carry a whole-object-specific code — ERR_COLLECT_MEMBER_UNRESOLVED for the + /// unmatched member, ERR_COLLECT_WHOLE_OBJECT for the type disagreement. The latter is + /// deliberately NOT the scalar arm's ERR_INVALID_ORIGIN: a loader that still requires @of + /// rejects this metadata with ERR_INVALID_ORIGIN too, so sharing the code would make a + /// corpus fixture pass on a port that implements nothing. + /// Mirrors the TS _checkCollectMembers. + private static void CheckCollectMembers( + MetaData refTarget, MetaData terminal, MetaData obj, MetaData field, + ErrorSource src, List errors) + { + // ADR-0039: resolving — a value object may inherit members via extends, and the + // terminal entity may inherit fields; own-only would silently skip inherited + // members, which is exactly the #270 bug class this guards. + var terminalFields = terminal.Children().Where(c => c.Type == TYPE_FIELD).ToList(); + foreach (var member in refTarget.Children().Where(c => c.Type == TYPE_FIELD)) + { + var match = terminalFields.FirstOrDefault(f => f.Name == member.Name); + if (match is null) + { + errors.Add(new MetaError( + $"origin.aggregate @agg:collect on {obj.Name}.{field.Name}: value-object member " + + $"'{member.Name}' has no matching field on '{terminal.Name}' — a whole-object " + + "rollup projects exactly the declared members.", + ErrorCode.ERR_COLLECT_MEMBER_UNRESOLVED, Envelope: src)); + continue; + } + string memberLabel = TypeLabel(member); + string matchLabel = TypeLabel(match); + if (memberLabel != matchLabel) + { + errors.Add(new MetaError( + $"origin.aggregate @agg:collect on {obj.Name}.{field.Name}: value-object member " + + $"'{member.Name}' is {memberLabel} but '{terminal.Name}.{match.Name}' is " + + $"{matchLabel} — a whole-object rollup preserves each member's type.", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, Envelope: src)); + } + } + } + + /// lets #335's whole-object @agg:collect arm report + /// ERR_COLLECT_WHOLE_OBJECT instead; it defaults to ERR_INVALID_ORIGIN so the scalar + /// @of and origin.first call sites keep their existing envelope byte-for-byte. private static void ValidateOrderByKeys( object? orderBy, MetaData? relatedEntity, MetaData obj, string fieldName, - string label, ErrorSource originSource, List errors) + string label, ErrorSource originSource, List errors, + ErrorCode code = ErrorCode.ERR_INVALID_ORIGIN) { // @orderBy is a declared string[] attr → stored as IReadOnlyList. if (orderBy is not IReadOnlyList keys || relatedEntity is null) return; @@ -793,13 +927,13 @@ private static void ValidateOrderByKeys( { errors.Add(new MetaError( $"{label} on {obj.Name}.{fieldName}: @orderBy key \"{raw}\" — no such field \"{key}\" on {relatedEntity.Name}.", - ErrorCode.ERR_INVALID_ORIGIN, Envelope: originSource)); + code, Envelope: originSource)); } else if (dir is not null && !SORT_ORDER_VALUES.Contains(dir)) { errors.Add(new MetaError( $"{label} on {obj.Name}.{fieldName}: @orderBy key \"{raw}\" — direction must be one of {string.Join("|", SORT_ORDER_VALUES)}.", - ErrorCode.ERR_INVALID_ORIGIN, Envelope: originSource)); + code, Envelope: originSource)); } } } @@ -951,7 +1085,13 @@ private static bool IsReferenceHop(MetaData hop) /// Validate an explicit `@via` "Entity.rel[.rel...]" path. Returns the walked /// relationship hop nodes (in path order) on full success — FR-024 B5 runs /// the cardinality checks over them — or null when any error was pushed. - private static List? ValidateViaPath( + /// A fully-walked @via path: the relationship hop nodes in path order, plus the + /// entity they terminate at. Returned as a pair (rather than re-walking for the terminal) + /// because a second walk means a second copy of the ADR-0042 package-resolution rule — + /// mirrors the TS WalkedViaPath. + private readonly record struct WalkedViaPath(List Hops, MetaData Terminal); + + private static WalkedViaPath? ValidateViaPath( string viaAttr, MetaData root, MetaData projection, @@ -1051,7 +1191,8 @@ private static bool IsReferenceHop(MetaData hop) currentObj = nextObj; } - return hops; + // currentObj is the terminal: every earlier exit returned null. + return new WalkedViaPath(hops, currentObj); } // ------------------------------------------------------------------------- diff --git a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java index 5d5a9aa88..0ea9f4759 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java +++ b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java @@ -2133,7 +2133,7 @@ private static void validateOriginNode(MetaRoot root, MetaObject obj, String via = origin.getVia(); if (via != null && !via.isEmpty()) { java.util.List hops = - validateViaPath(via, root, obj, field.getName(), origin.getSource()); + validateViaPath(via, root, obj, field.getName(), origin.getSource()).hops(); checkPassthroughCardinality(hops, obj, field.getName(), origin.getSource()); } else if (!isValueHost) { // FR-024 §6 — no @via: derive the base entity; a @from targeting the @@ -2249,18 +2249,94 @@ private static void validateOriginNode(MetaRoot root, MetaObject obj, ErrorCode.ERR_INVALID_ORIGIN, src); } java.util.List hops = - validateViaPath(via, root, obj, field.getName(), src); + validateViaPath(via, root, obj, field.getName(), src).hops(); checkAggregateCardinality(hops, obj, field.getName(), src); return; } - // --- count/sum/avg/min/max/collect: @of REQUIRED --- + // --- @of: REQUIRED for count/sum/avg/min/max; OPTIONAL for collect --- + // #335 — an @of-absent collect is a WHOLE-OBJECT rollup: collect the related + // rows as an array of the field's declared @objectRef value object rather than + // an array of one scalar column. if (!ofPresent) { - throw new MetaDataException( - ErrorMessageConstants.ERR_INVALID_ORIGIN - + ": origin.aggregate on " + obj.getName() + "." + field.getName() - + ": missing @of.", - ErrorCode.ERR_INVALID_ORIGIN, src); + if (!isCollect) { + throw new MetaDataException( + ErrorMessageConstants.ERR_INVALID_ORIGIN + + ": origin.aggregate on " + obj.getName() + "." + field.getName() + + ": missing @of.", + ErrorCode.ERR_INVALID_ORIGIN, src); + } + // Whole-object rollup. The carrying field must be a field.object naming a + // value object, and @via must be explicit (there is no @of entity to infer + // the single-hop relation from). + // ADR-0039: resolving (hasMetaAttr/getMetaAttr default to includeParentData) + // — @objectRef may be inherited via extends. + Object objectRefVal = field.hasMetaAttr(ObjectField.ATTR_OBJECTREF) + ? field.getMetaAttr(ObjectField.ATTR_OBJECTREF).getValue() + : null; + String objectRef = objectRefVal instanceof String ? (String) objectRefVal : null; + if (!ObjectField.SUBTYPE_OBJECT.equals(field.getSubType()) + || objectRef == null || objectRef.isEmpty()) { + throw new MetaDataException( + ErrorMessageConstants.ERR_COLLECT_WHOLE_OBJECT + + ": origin.aggregate @agg:collect on " + obj.getName() + "." + field.getName() + + ": @of is omitted, so this is a whole-object rollup — the carrying field" + + " must be a field.object declaring @objectRef (add @of to collect a" + + " single column instead).", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, src); + } + // #210's value-only rule is PAYLOAD-scoped and never reaches a projection-hosted + // field, so this branch enforces it itself. Without it an @objectRef to an entity + // silently rolls up the FULL entity — the #270 shape, this time baked into DDL. + // ADR-0042 — a bare @objectRef resolves in the DECLARING owner's package. + String refPkg = obj.getPackage() == null ? "" : obj.getPackage(); + MetaObject refTarget = resolveRootObject(root, objectRef, refPkg); + if (refTarget != null && !MetaObject.SUBTYPE_VALUE.equals(refTarget.getSubType())) { + throw new MetaDataException( + "ERR_SUBTYPE_RULE_VIOLATION" + + ": origin.aggregate @agg:collect on " + obj.getName() + "." + field.getName() + + ": @objectRef '" + objectRef + "' resolves to " + + MetaObject.TYPE_OBJECT + "." + refTarget.getSubType() + + " — a whole-object rollup must target an object.value (#210, ADR-0028).", + ErrorCode.ERR_SUBTYPE_RULE_VIOLATION, src); + } + // ADR-0039: own — origin.* never inherits (ADR-0029); getVia() reads the + // origin's own @via, as every other arm of this method does. + String woVia = origin.getVia(); + if (woVia == null || woVia.isEmpty()) { + throw new MetaDataException( + ErrorMessageConstants.ERR_COLLECT_WHOLE_OBJECT + + ": origin.aggregate @agg:collect on " + obj.getName() + "." + field.getName() + + ": @via is required on a whole-object rollup — there is no @of entity" + + " to infer the relationship from.", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, src); + } + // @distinct is refused on the object form. It is NOT an engine limit (both + // engines dedupe JSON objects); it is a guaranteed no-op whenever the value + // object carries the entity's primary key, which is the common case, and a + // silent no-op is worse than a refusal. + if (hasDistinct) { + throw new MetaDataException( + ErrorMessageConstants.ERR_COLLECT_WHOLE_OBJECT + + ": origin.aggregate @agg:collect on " + obj.getName() + "." + field.getName() + + ": @distinct is not supported on a whole-object rollup (it is a no-op" + + " whenever the value object carries the primary key).", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, src); + } + // One walk yields both the hops (cardinality) and the terminal entity + // (@orderBy keys, member resolution). An invalid @via throws out of + // validateViaPath with its own error, so nothing downstream runs and no + // second, misleadingly-scoped error is produced. + WalkedViaPath woWalked = validateViaPath(woVia, root, obj, field.getName(), src); + checkAggregateCardinality(woWalked.hops(), obj, field.getName(), src); + // @orderBy keys resolve against the @via TERMINAL entity, not @of. + validateOrderByKeys(originOrderBy(origin), woWalked.terminal(), obj, field.getName(), + "origin.aggregate @agg:collect", src, + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, ErrorMessageConstants.ERR_COLLECT_WHOLE_OBJECT); + if (refTarget != null) { + checkCollectMembers(refTarget, woWalked.terminal(), obj, field, src); + } + return; } OriginTarget ofTarget = validateFromOrOfPath(of, root, obj, field.getName(), "origin.aggregate.@of", src); @@ -2284,7 +2360,7 @@ private static void validateOriginNode(MetaRoot root, MetaObject obj, String via = origin.getVia(); if (via != null && !via.isEmpty()) { java.util.List hops = - validateViaPath(via, root, obj, field.getName(), src); + validateViaPath(via, root, obj, field.getName(), src).hops(); checkAggregateCardinality(hops, obj, field.getName(), src); return; } @@ -2393,7 +2469,7 @@ private static void validateOriginNode(MetaRoot root, MetaObject obj, String via = origin.getVia(); if (via != null && !via.isEmpty()) { java.util.List hops = - validateViaPath(via, root, obj, field.getName(), src); + validateViaPath(via, root, obj, field.getName(), src).hops(); checkAggregateCardinality(hops, obj, field.getName(), src); } else { // (A value host never reaches here — the #210 assembly-origin @@ -2475,6 +2551,72 @@ private static List originOrderBy(MetaOrigin origin) { return java.util.Collections.emptyList(); } + /** + * A field's declared type on BOTH axes — {@code field.} plus {@code []} when it is + * an array. ADR-0039: {@code isArrayType()} (resolving), so array-ness inherited via + * {@code extends} counts. Mirrors the TS {@code _typeLabel}. + */ + private static String typeLabel(MetaField field) { + return "field." + field.getSubType() + (field.isArrayType() ? "[]" : ""); + } + + /** + * #335 — a whole-object {@code @agg:collect} projects EXACTLY the declared value object's + * members, matched BY NAME against the {@code @via} terminal entity's fields: + *
    + *
  • an unmatched member is an error, never a silent drop. Failing open here is how #270 + * turned a curated value object into the full entity, invisible in a diff because the + * metadata still read as curated.
    • + *
  • a matched member must agree on BOTH type axes (#185 type-preserving doctrine), so a + * scalar member cannot bind an array field or vice versa.
    • + *
+ * Both refusals carry a whole-object-specific code — ERR_COLLECT_MEMBER_UNRESOLVED for the + * unmatched member, ERR_COLLECT_WHOLE_OBJECT for the type disagreement. The latter is + * deliberately NOT the scalar arm's ERR_INVALID_ORIGIN: a loader that still requires + * {@code @of} rejects this metadata with ERR_INVALID_ORIGIN too, so sharing the code would + * make a corpus fixture pass on a port that implements nothing. + * Mirrors the TS {@code _checkCollectMembers}. + */ + private static void checkCollectMembers(MetaObject refTarget, MetaObject terminal, + MetaObject obj, MetaField field, + com.metaobjects.source.ErrorSource src) { + // ADR-0039: resolving (includeParentData=true) — a value object may inherit members via + // extends, and the terminal entity may inherit fields; own-only would silently skip + // inherited members, which is exactly the #270 bug class this guards. + java.util.List> terminalFields = new java.util.ArrayList<>(); + for (MetaData c : terminal.getChildren(MetaData.class, true)) { + if (c instanceof MetaField) terminalFields.add((MetaField) c); + } + for (MetaData c : refTarget.getChildren(MetaData.class, true)) { + if (!(c instanceof MetaField)) continue; + MetaField member = (MetaField) c; + MetaField match = null; + for (MetaField f : terminalFields) { + if (nameMatches(f, member.getName())) { match = f; break; } + } + if (match == null) { + throw new MetaDataException( + ErrorMessageConstants.ERR_COLLECT_MEMBER_UNRESOLVED + + ": origin.aggregate @agg:collect on " + obj.getName() + "." + field.getName() + + ": value-object member '" + member.getName() + + "' has no matching field on '" + terminal.getName() + + "' — a whole-object rollup projects exactly the declared members.", + ErrorCode.ERR_COLLECT_MEMBER_UNRESOLVED, src); + } + String memberLabel = typeLabel(member); + String matchLabel = typeLabel(match); + if (!memberLabel.equals(matchLabel)) { + throw new MetaDataException( + ErrorMessageConstants.ERR_COLLECT_WHOLE_OBJECT + + ": origin.aggregate @agg:collect on " + obj.getName() + "." + field.getName() + + ": value-object member '" + member.getName() + "' is " + memberLabel + + " but '" + terminal.getName() + "." + match.getName() + "' is " + matchLabel + + " — a whole-object rollup preserves each member's type.", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, src); + } + } + } + /** * #195 — validate {@code @orderBy} keys ({@code 'field[:asc|desc]'}) resolve * against the RELATED entity's effective fields (the entity reached via @@ -2483,10 +2625,22 @@ private static List originOrderBy(MetaOrigin origin) { * {@code @agg:collect} (element order) and {@code origin.first} (row selection). * A {@code null} related entity means a prior error already fired — skip silently. * Mirrors the TS {@code _validateOrderByKeys}. + * + *

{@code code}/{@code codeName} let #335's whole-object {@code @agg:collect} arm report + * ERR_COLLECT_WHOLE_OBJECT instead. The 6-arg overload keeps ERR_INVALID_ORIGIN, so the + * scalar {@code @of} and {@code origin.first} call sites are byte-identical.

*/ private static void validateOrderByKeys(List orderBy, MetaObject relatedEntity, MetaObject obj, String fieldName, String label, com.metaobjects.source.ErrorSource originSource) { + validateOrderByKeys(orderBy, relatedEntity, obj, fieldName, label, originSource, + ErrorCode.ERR_INVALID_ORIGIN, ErrorMessageConstants.ERR_INVALID_ORIGIN); + } + + private static void validateOrderByKeys(List orderBy, MetaObject relatedEntity, + MetaObject obj, String fieldName, String label, + com.metaobjects.source.ErrorSource originSource, + ErrorCode code, String codeName) { if (orderBy == null || relatedEntity == null) return; for (String raw : orderBy) { if (raw == null) continue; @@ -2503,18 +2657,18 @@ private static void validateOrderByKeys(List orderBy, MetaObject related } if (target == null) { throw new MetaDataException( - ErrorMessageConstants.ERR_INVALID_ORIGIN + codeName + ": " + label + " on " + obj.getName() + "." + fieldName + ": @orderBy key \"" + raw + "\" — no such field \"" + key + "\" on " + relatedEntity.getName() + ".", - ErrorCode.ERR_INVALID_ORIGIN, originSource); + code, originSource); } if (dir != null && !SORT_ORDER_VALUES.contains(dir)) { throw new MetaDataException( - ErrorMessageConstants.ERR_INVALID_ORIGIN + codeName + ": " + label + " on " + obj.getName() + "." + fieldName + ": @orderBy key \"" + raw + "\" — direction must be one of asc|desc.", - ErrorCode.ERR_INVALID_ORIGIN, originSource); + code, originSource); } } } @@ -3506,7 +3660,15 @@ private static OriginTarget validateFromOrOfPath(String pathAttr, MetaRoot root, * {@code @references} (FR-024). The resolved target must be an entity at root, * which becomes the next hop's current entity. */ - private static java.util.List validateViaPath(String viaAttr, MetaRoot root, + /** + * A fully-walked {@code @via} path: the relationship hop nodes in path order, plus the + * entity they terminate at. Returned as a pair (rather than re-walking for the terminal) + * because a second walk means a second copy of the ADR-0042 package-resolution rule — + * mirrors the TS {@code WalkedViaPath}. + */ + private record WalkedViaPath(java.util.List hops, MetaObject terminal) { } + + private static WalkedViaPath validateViaPath(String viaAttr, MetaRoot root, MetaObject projection, String fieldName, com.metaobjects.source.ErrorSource envelope) { // FR5d — referrer is `::` (matches @@ -3599,7 +3761,8 @@ private static java.util.List validateViaPath(String viaAttr, MetaRoot hops.add(hop); currentObj = nextObj; } - return hops; + // currentObj is the terminal: every earlier exit threw. + return new WalkedViaPath(hops, currentObj); } // ========================================================================= diff --git a/server/python/src/metaobjects/loader/validation_passes.py b/server/python/src/metaobjects/loader/validation_passes.py index 9d0c5eea1..f3b99a28c 100644 --- a/server/python/src/metaobjects/loader/validation_passes.py +++ b/server/python/src/metaobjects/loader/validation_passes.py @@ -1584,6 +1584,16 @@ def _validate_entity_field_ref( return (entity, field_node) +class WalkedViaPath(NamedTuple): + """A fully-walked ``@via`` path: the relationship hop nodes in path order, + plus the entity they terminate at. Returned as a pair (rather than re-walking + for the terminal) because a second walk means a second copy of the ADR-0042 + package-resolution rule — mirrors the TS ``WalkedViaPath``.""" + + hops: list[MetaData] + terminal: MetaData + + def _validate_via_path( via: str, context: str, @@ -1592,12 +1602,15 @@ def _validate_via_path( errors: list[MetaError], origin_node: MetaData, referrer: str, -) -> list[MetaData] | None: +) -> WalkedViaPath | None: """Validate a dotted relationship path 'Entity.rel1[.rel2...]'. - Returns the walked relationship hop nodes (in path order) on full success - (FR-024 B5 runs the cardinality checks over them); appends ERR_INVALID_ORIGIN - and returns None if not. + Returns the walked relationship hop nodes (in path order) together with the + TERMINAL entity node on full success (FR-024 B5 runs the cardinality checks + over the hops; #335's whole-object @agg:collect has no @of entity, so its + @orderBy keys and value-object members resolve against the terminal); + appends ERR_INVALID_ORIGIN and returns None if not. ``terminal`` is defined + exactly when ``hops`` is — every early exit returns None. *origin_node* carries the parse-time envelope (files/json_path); *referrer* is the canonical referrer FQN (``::``) attached @@ -1695,7 +1708,8 @@ def _validate_via_path( hops.append(rel_node) current_entity = next_entity - return hops + # current_entity is the terminal: every earlier exit returned None. + return WalkedViaPath(hops, current_entity) # --------------------------------------------------------------------------- @@ -2004,6 +2018,61 @@ def _check_passthrough_type( ) +def _type_label(field: MetaData) -> str: + """A field's declared type on BOTH axes — ``field.`` plus ``[]`` when it is + an array. ADR-0039: resolved_is_array(), so array-ness inherited via extends counts. + Mirrors the TS ``_typeLabel``.""" + return f"field.{field.sub_type}{'[]' if field.resolved_is_array() else ''}" + + +def _check_collect_members( + ref_target: MetaData, + terminal: MetaData, + obj: MetaData, + field: MetaData, + src: object, + errors: list[MetaError], +) -> None: + """#335 — a whole-object @agg:collect projects EXACTLY the declared value object's + members, matched BY NAME against the @via terminal entity's fields: + + - an unmatched member is an error, never a silent drop. Failing open here is how + #270 turned a curated value object into the full entity, invisible in a diff + because the metadata still read as curated. + - a matched member must agree on BOTH type axes (#185 type-preserving doctrine), + so a scalar member cannot bind an array field or vice versa. + + Both refusals carry a whole-object-specific code — ERR_COLLECT_MEMBER_UNRESOLVED for + the unmatched member, ERR_COLLECT_WHOLE_OBJECT for the type disagreement. The latter + is deliberately NOT the scalar arm's ERR_INVALID_ORIGIN: a loader that still requires + @of rejects this metadata with ERR_INVALID_ORIGIN too, so sharing the code would make + a corpus fixture pass on a port that implements nothing. + + Mirrors the TS ``_checkCollectMembers``.""" + # ADR-0039: resolving — a value object may inherit members via extends, and the + # terminal entity may inherit fields; own-only would silently skip inherited + # members, which is exactly the #270 bug class this guards. + terminal_fields = [c for c in terminal.children() if c.type == TYPE_FIELD] + for member in (c for c in ref_target.children() if c.type == TYPE_FIELD): + match = next((f for f in terminal_fields if f.name == member.name), None) + if match is None: + errors.append(MetaError( + f"origin.aggregate @agg:collect on {obj.name}.{field.name}: value-object " + f"member '{member.name}' has no matching field on '{terminal.name}' — a " + f"whole-object rollup projects exactly the declared members.", + ErrorCode.ERR_COLLECT_MEMBER_UNRESOLVED, envelope=src)) + continue + member_label = _type_label(member) + match_label = _type_label(match) + if member_label != match_label: + errors.append(MetaError( + f"origin.aggregate @agg:collect on {obj.name}.{field.name}: value-object " + f"member '{member.name}' is {member_label} but " + f"'{terminal.name}.{match.name}' is {match_label} — a whole-object rollup " + f"preserves each member's type.", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, envelope=src)) + + def _validate_order_by_keys( order_by: object, related_entity: MetaData | None, @@ -2012,12 +2081,17 @@ def _validate_order_by_keys( label: str, origin_source: object, errors: list[MetaError], + code: ErrorCode = ErrorCode.ERR_INVALID_ORIGIN, ) -> None: """#195 — validate that ``@orderBy`` keys ('field[:asc|desc]') resolve against the RELATED entity's effective fields (the entity reached via @via/@of), and that any direction suffix is asc/desc. Shared by @agg:collect (element order) and origin.first (row selection). A missing related entity means a prior error already fired — skip. - Mirrors the TS _validateOrderByKeys.""" + Mirrors the TS _validateOrderByKeys. + + *code* lets #335's whole-object @agg:collect arm report ERR_COLLECT_WHOLE_OBJECT + instead; it defaults to ERR_INVALID_ORIGIN so the scalar @of and origin.first call + sites keep their existing envelope byte-for-byte.""" if not isinstance(order_by, (list, tuple)) or related_entity is None: return for raw in order_by: @@ -2036,7 +2110,7 @@ def _validate_order_by_keys( MetaError( f'{label} on {obj.name}.{field_name}: @orderBy key "{raw}" — no such ' f'field "{key}" on {related_entity.name}.', - ErrorCode.ERR_INVALID_ORIGIN, + code, envelope=origin_source, ) ) @@ -2045,7 +2119,7 @@ def _validate_order_by_keys( MetaError( f'{label} on {obj.name}.{field_name}: @orderBy key "{raw}" — direction ' f"must be one of {'|'.join(SORT_ORDER_VALUES)}.", - ErrorCode.ERR_INVALID_ORIGIN, + code, envelope=origin_source, ) ) @@ -2139,7 +2213,8 @@ def _validate_origin_paths( ) via = origin.attr(ORIGIN_ATTR_VIA) if isinstance(via, str) and via: - hops = _validate_via_path(via, ctx, root, host_pkg, errors, origin, referrer) + walked = _validate_via_path(via, ctx, root, host_pkg, errors, origin, referrer) + hops = walked.hops if walked is not None else None if hops is not None: _check_passthrough_cardinality(hops, node.name, origin.source, errors) elif from_target is not None and not is_value_host: @@ -2235,20 +2310,94 @@ def _validate_origin_paths( f"explicit @via (a quantifier has no @of to infer the path from).", ErrorCode.ERR_INVALID_ORIGIN, envelope=src)) else: - hops = _validate_via_path(via, ctx, root, host_pkg, errors, origin, referrer) + walked = _validate_via_path(via, ctx, root, host_pkg, errors, origin, referrer) + hops = walked.hops if walked is not None else None if hops is not None: _check_aggregate_cardinality(hops, node.name, src, errors) continue - # --- count/sum/avg/min/max/collect: @of REQUIRED --- + # --- @of: REQUIRED for count/sum/avg/min/max; OPTIONAL for collect --- + # #335 — an @of-absent collect is a WHOLE-OBJECT rollup: collect the + # related rows as an array of the field's declared @objectRef value + # object rather than an array of one scalar column. if not of_present: - errors.append( - MetaError( - f"{ctx} is missing required attribute '@{ORIGIN_ATTR_OF}'", - ErrorCode.ERR_INVALID_ORIGIN, - envelope=src, + if not is_collect: + errors.append( + MetaError( + f"{ctx} is missing required attribute '@{ORIGIN_ATTR_OF}'", + ErrorCode.ERR_INVALID_ORIGIN, + envelope=src, + ) ) - ) + continue + # Whole-object rollup. The carrying field must be a field.object + # naming a value object, and @via must be explicit (there is no @of + # entity to infer the single-hop relation from). + # ADR-0039: resolving — @objectRef may be inherited via extends. + object_ref = node.get_meta_attr(FIELD_ATTR_OBJECT_REF) + if node.sub_type != FIELD_SUBTYPE_OBJECT or not isinstance(object_ref, str) \ + or not object_ref: + errors.append(MetaError( + f"origin.aggregate @agg:collect on {obj.name}.{node.name}: @of is " + f"omitted, so this is a whole-object rollup — the carrying field " + f"must be a field.object declaring @objectRef (add @of to collect a " + f"single column instead).", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, envelope=src)) + continue + # #210's value-only rule is PAYLOAD-scoped and never reaches a + # projection-hosted field, so this branch enforces it itself. + # Without it an @objectRef to an entity silently rolls up the FULL + # entity — the #270 shape, this time baked into DDL. + # ADR-0042 — a bare @objectRef resolves in the DECLARING owner's + # package (an inherited field resolves in the package that declared it). + ref_owner = node.parent or obj + ref_pkg = ref_owner.package or ref_owner.file_default_package or "" + ref_target = resolve_object_ref(root, object_ref, ref_pkg) + if ref_target is not None and ref_target.sub_type != OBJECT_SUBTYPE_VALUE: + errors.append(MetaError( + f"origin.aggregate @agg:collect on {obj.name}.{node.name}: " + f"@objectRef '{object_ref}' resolves to " + f"{TYPE_OBJECT}.{ref_target.sub_type} — a whole-object rollup must " + f"target an object.value (#210, ADR-0028).", + ErrorCode.ERR_SUBTYPE_RULE_VIOLATION, envelope=src)) + continue + # ADR-0039: own — origin.* never inherits (ADR-0029). + via_attr = origin.attr(ORIGIN_ATTR_VIA) + if not isinstance(via_attr, str) or not via_attr: + errors.append(MetaError( + f"origin.aggregate @agg:collect on {obj.name}.{node.name}: @via is " + f"required on a whole-object rollup — there is no @of entity to " + f"infer the relationship from.", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, envelope=src)) + continue + # @distinct is refused on the object form. It is NOT an engine limit + # (both engines dedupe JSON objects); it is a guaranteed no-op + # whenever the value object carries the entity's primary key, which + # is the common case, and a silent no-op is worse than a refusal. + if has_distinct: + errors.append(MetaError( + f"origin.aggregate @agg:collect on {obj.name}.{node.name}: @distinct " + f"is not supported on a whole-object rollup (it is a no-op whenever " + f"the value object carries the primary key).", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, envelope=src)) + continue + # One walk yields both the hops (cardinality) and the terminal entity + # (@orderBy keys, member resolution). An invalid @via (e.g. a single- + # segment "A") returns None having already pushed its own error, so + # everything downstream is skipped and no second, misleadingly-scoped + # error is emitted. + walked = _validate_via_path( + via_attr, ctx, root, host_pkg, errors, origin, referrer) + if walked is not None: + _check_aggregate_cardinality(walked.hops, node.name, src, errors) + # @orderBy keys resolve against the @via TERMINAL entity, not @of. + _validate_order_by_keys( + order_by, walked.terminal, obj, node.name, + "origin.aggregate @agg:collect", src, errors, + ErrorCode.ERR_COLLECT_WHOLE_OBJECT) + if ref_target is not None: + _check_collect_members( + ref_target, walked.terminal, obj, node, src, errors) continue # NOTE (FR-024 B6): NO extends/origin agreement on aggregates — # an aggregate computes something new (spec §4 is passthrough-only). @@ -2270,7 +2419,8 @@ def _validate_origin_paths( obj, node.name, "origin.aggregate @agg:collect", src, errors) via = origin.attr(ORIGIN_ATTR_VIA) if isinstance(via, str) and via: - hops = _validate_via_path(via, ctx, root, host_pkg, errors, origin, referrer) + walked = _validate_via_path(via, ctx, root, host_pkg, errors, origin, referrer) + hops = walked.hops if walked is not None else None if hops is not None: _check_aggregate_cardinality(hops, node.name, src, errors) continue @@ -2381,7 +2531,8 @@ def _resolve_field(name: str, _base: MetaData = base) -> str | None: # @via — explicit (validated + cardinality) or single-hop-unique inferred. via = origin.attr(ORIGIN_ATTR_VIA) if isinstance(via, str) and via: - hops = _validate_via_path(via, ctx, root, host_pkg, errors, origin, referrer) + walked = _validate_via_path(via, ctx, root, host_pkg, errors, origin, referrer) + hops = walked.hops if walked is not None else None if hops is not None: _check_aggregate_cardinality(hops, node.name, src, errors) elif of_target is not None: From dfcbc0e5e66ead6a7deda7968adc514176702cc2 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 23 Aug 2026 15:42:08 -0400 Subject: [PATCH 21/25] docs(metamodel): @of is optional on collect, in all seven byte-gated copies (#335) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Until now the loader was LOOSER than the contract it ships: five ports accept an @of-absent @agg:collect while every registry copy still said @of was "Required for count/sum/avg/min/max/collect". Five prose strings change, byte-identically across the copies that carry them: - @of — required for count/sum/avg/min/max; OPTIONAL for collect, where absent means a whole-object rollup of the field's declared @objectRef value object. - @agg and the origin.aggregate type description — collect is an array rollup of the @of column OR of the carrying field.object's value object. - @distinct — not supported on a whole-object collect, with the reason (a guaranteed no-op whenever the value object carries the primary key). - the origin.aggregate `rules` string — the whole rule, including the one thing a port could not infer from the others. That last point is the reason this is not a one-line edit. The VO-member↔column binding is NAME MATCHING (member name == terminal field name), deliberately not `extends` — which keeps one value object collectable from two different entities, something `extends` would forbid. It is a NEW implicit convention in a metamodel whose stated doctrine is "extends is THE mechanism", so leaving it undocumented would have left four ports to each infer it from the TypeScript. It is now written into the byte-gated contract, along with the @via-terminal @orderBy scope and the both-axes type agreement. Six copies carry the long form and are regenerated or edited in lockstep; `server/csharp/.../Origin/OriginSchema.cs` carries its own shorter wording (it is a C# runtime schema, not part of the byte-gated manifest) and is edited to match in substance. `expected-registry.json` and the `metamodel-docs` fixtures are GENERATED — emitted from the TS registry and by `scripts/regen-metamodel-docs.ts` rather than hand-edited; the regenerated manifest differs from the committed one in exactly the five intended strings and nothing else. §A7 of the design spec is corrected in the same commit, since it documents this same contract. It claimed the change rode #342's unreleased `metamodelVersion` 0.11 and needed no edit. The re-check that section itself demanded then fired: origin/main's 2af77e7cd names 0.24.1 as the sole claimant of 0.11, so #335 needs 0.12 — already shipped in 7c99536a7. The superseded reasoning is recorded rather than deleted, because a reviewer independently re-derived it and called it correct: "v0.24.0 shipped 0.10 and main already carries the unreleased bump" is true and still wrong, because it never asks who else has claimed 0.11. Registry conformance green in all four ports (TS 8/0 — the only runner that compares description prose; Python 84/0; C# 25/0; Java RegistryManifest 3/0, which compares structure rather than prose). TS metadata 2526/0. `check-metamodel-version.mjs` reports the four prose changes as WARN and passes: "additive since v0.24.0; declared 0.12 — ok". Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq --- ...22-issue-335-whole-object-rollup-design.md | 28 ++++++++++++------- fixtures/metamodel-docs/expected/INDEX.md | 2 +- .../metamodel-docs/expected/types/origin.md | 10 +++---- .../expected-registry.json | 10 +++---- .../Persistence/Origin/OriginSchema.cs | 6 ++-- .../MetaObjects/SpecMetamodel/origin.json | 10 +++---- .../metaobjects/spec_metamodel/origin.json | 10 +++---- .../origin/origin-definition.embedded.ts | 10 +++---- spec/metamodel/origin.json | 10 +++---- 9 files changed, 52 insertions(+), 44 deletions(-) diff --git a/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md b/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md index 4fdf92f92..72d52efee 100644 --- a/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md +++ b/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md @@ -228,22 +228,30 @@ server/typescript/packages/metadata/src/persistence/origin/origin-definition.emb All seven change in lockstep or `registry-conformance` goes red in every port. -## A7. `metamodelVersion`: rides the existing `0.11` — no further bump +## A7. `metamodelVersion`: `0.12` — SETTLED and SHIPPED Classification: **additive**. Previously-invalid metadata becomes valid; nothing that loaded before stops loading. This is #210 in reverse — a rule change whose only machine-readable footprint is prose, which `check-metamodel-version.mjs` can only **WARN** about (`:36`), so no -gate forces the call and it is made here. +gate forces the call and it is made by a human. -**But no edit is required.** `#342` (`e96bbe329`) already moved `metamodelVersion` `0.10` → -`0.11`, and that bump is **unreleased** — `v0.24.0` shipped `0.10`. The gate baselines against -the last **release tag**, not `HEAD~1`, precisely so that a release cycle needs one bump rather -than one per PR. So `0.11` already covers this change. +**The number is `0.12`, set by `7c99536a7` via `check-metamodel-version.mjs --set 0.12`** (all +five sites at once: the manifest plus the four port constants). -Two conditions would change that, and the plan must re-check both at implementation time: -- If `0.11` ships in a release **before** this lands, this needs `0.12`. -- If Half B is judged breaking rather than additive (see B2's note), the classification — - though not necessarily the number, pre-1.0 — changes with it. +This section previously said the change rode `#342`'s unreleased `0.11` and needed no edit. That +reasoning held only while `0.11` was unclaimed, and it stopped holding during implementation: +`origin/main`'s `2af77e7cd` names **`0.24.1` as the sole claimant of `0.11`**. The re-check this +section itself demanded — *"if `0.11` ships in a release before this lands, this needs `0.12`"* — +therefore fired. Doug ruled `0.12` on 2026-08-23. + +Worth keeping, because a reviewer re-derived the superseded answer and called the `0.11` ride +correct: "`v0.24.0` shipped `0.10` and main already carries the unreleased bump" is true and +still gives the wrong answer, because it never asks *who else has already claimed `0.11`*. The +gate baselines against the last release TAG, so an unreleased bump looks free right up until +another change in the same cycle spends it. + +One condition remains open: if Half B is judged breaking rather than additive (see B2's note), +the classification — though not necessarily the number, pre-1.0 — changes with it. --- diff --git a/fixtures/metamodel-docs/expected/INDEX.md b/fixtures/metamodel-docs/expected/INDEX.md index 037fc86cf..d36b475f5 100644 --- a/fixtures/metamodel-docs/expected/INDEX.md +++ b/fixtures/metamodel-docs/expected/INDEX.md @@ -55,7 +55,7 @@ children, and cardinality of a subtype. Universal documentation attributes | `object.entity` | An object that owns its data: own identity, writable sources, and lifecycle. The default object subtype — a bare `object:` key resolves to entity. May co-locate templates (template.prompt and friends) with the owning entity. | [types/object.md#objectentity](types/object.md#objectentity) | | `object.projection` | A derived read-only representation of entities. Its fields are extends-bound / origin-derived / self-declared-under-external-assembly, all read-only at the subtype level. Identity is optional and MUST extend an entity identity; sources are restricted to read-only @kinds. The declared field set IS the exposure (inclusive, fail-closed). | [types/object.md#objectprojection](types/object.md#objectprojection) | | `object.value` | A value object — pure shape with NO identity and NO source, ever. Constructed (by caller / embedding), never populated from a store. May `extends` an entity's fields to reuse shape. Equality is by content. | [types/object.md#objectvalue](types/object.md#objectvalue) | -| `origin.aggregate` | A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup of @of). | [types/origin.md#originaggregate](types/origin.md#originaggregate) | +| `origin.aggregate` | A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted). | [types/origin.md#originaggregate](types/origin.md#originaggregate) | | `origin.base` | Abstract base origin — the shared root subtype for field-level provenance. A field carrying any origin.* is derived ⇒ read-only wherever it lives. The base carries no attrs of its own; concrete subtypes add their provenance attrs. | [types/origin.md#originbase](types/origin.md#originbase) | | `origin.computed` | A row-level value computed from the base entity's own fields via a structured expression tree (@expr). No related rows, no @via. Read-only; the expression's inferred type must equal the field's declared subType. | [types/origin.md#origincomputed](types/origin.md#origincomputed) | | `origin.first` | The single related row selected by @orderBy along @via, projecting its @of column (argmax then project). Latest = @orderBy desc. Read-only; empty related set (after @filter) → null, so the field must not be @required. | [types/origin.md#originfirst](types/origin.md#originfirst) | diff --git a/fixtures/metamodel-docs/expected/types/origin.md b/fixtures/metamodel-docs/expected/types/origin.md index c7f2d2d38..7e876c350 100644 --- a/fixtures/metamodel-docs/expected/types/origin.md +++ b/fixtures/metamodel-docs/expected/types/origin.md @@ -12,11 +12,11 @@ documentation attributes are omitted here (see [providers.md](../providers.md)). ### origin.aggregate -A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup of @of). +A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted). **Owning provider:** metaobjects-core-types -**Rules:** @via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max/collect and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only. +**Rules:** @via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only. On @agg:collect @of is OPTIONAL: omitting it declares a WHOLE-OBJECT rollup, which collects each related row as the carrying field's declared @objectRef value object instead of one scalar column. A whole-object rollup requires a field.object carrying @objectRef, requires that @objectRef to name an object.value, requires an explicit @via (there is no @of entity to infer the path from), and refuses @distinct. Its @orderBy keys resolve against the @via TERMINAL entity, not the head or a middle hop. Its value-object members bind to the terminal entity's fields BY NAME — member name == terminal field name, deliberately NOT extends, so one value object stays collectable from two different entities — and every member must match a terminal field agreeing on BOTH field. and array-ness. **When to use:** A projection needs a value derived by reducing related rows — a count/sum/avg/min/max, a 'did any/every related row match' flag, or an array of collected values. Declare it instead of hand-writing the aggregate query — it stays consistent and regenerates. @@ -24,10 +24,10 @@ A value reduced from the related row-set reached along a relationship path (@via | Attribute | Type | Required | Default | Allowed values | Provider | Description | | --- | --- | --- | --- | --- | --- | --- | -| `@agg` | string | yes | | `count`, `sum`, `avg`, `min`, `max`, `any`, `all`, `collect` | metaobjects-core-types | The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup of @of — the field must be isArray). | -| `@distinct` | boolean | no | | | metaobjects-core-types | Set (collect-only) to dedupe collected values (set semantics). | +| `@agg` | string | yes | | `count`, `sum`, `avg`, `min`, `max`, `any`, `all`, `collect` | metaobjects-core-types | The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted; the field must be isArray). | +| `@distinct` | boolean | no | | | metaobjects-core-types | Set (collect-only) to dedupe collected values (set semantics). Not supported on a whole-object collect (@of omitted): it is a guaranteed no-op whenever the value object carries the primary key, and a silent no-op is worse than a refusal. | | `@filter` | filter | no | | | metaobjects-core-types | Optional structured predicate scoping which related rows the aggregate spans (required for any/all, where it is the quantified predicate). A portable attr.filter object (eq/ne/in/isNull with and/or), desugared to canonical { field: { op: value } } at parse time; codegen renders it per target (e.g. SQL FILTER (WHERE ...) or SQLite CASE WHEN for a relational view). | -| `@of` | string | no | | | metaobjects-core-types | Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max/collect; forbidden for any/all (which quantify over rows via @filter, not a column). | +| `@of` | string | no | | | metaobjects-core-types | Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max; OPTIONAL for collect, where absent means a whole-object rollup of the field's declared @objectRef value object; forbidden for any/all (which quantify over rows via @filter, not a column). | | `@orderBy` | string[] | no | | | metaobjects-core-types | Ordering keys as 'field[:asc\|desc]' (default asc) over the related entity's fields; nulls sort last. On @agg:collect sets element order (non-distinct only); on origin.first (required) selects the row. Semantic — carries no SQL syntax. | | `@via` | string | no | | | metaobjects-core-types | Dotted relationship path from the base entity to the aggregated rows (e.g. 'Program.weeks' or 'Program.weeks.workouts'). May be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (FR-024, ADR-0029). | diff --git a/fixtures/registry-conformance/expected-registry.json b/fixtures/registry-conformance/expected-registry.json index 41a088f38..5256cf97f 100644 --- a/fixtures/registry-conformance/expected-registry.json +++ b/fixtures/registry-conformance/expected-registry.json @@ -3363,8 +3363,8 @@ { "type": "origin", "subType": "aggregate", - "description": "A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup of @of).", - "rules": "@via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max/collect and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only.", + "description": "A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted).", + "rules": "@via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only. On @agg:collect @of is OPTIONAL: omitting it declares a WHOLE-OBJECT rollup, which collects each related row as the carrying field's declared @objectRef value object instead of one scalar column. A whole-object rollup requires a field.object carrying @objectRef, requires that @objectRef to name an object.value, requires an explicit @via (there is no @of entity to infer the path from), and refuses @distinct. Its @orderBy keys resolve against the @via TERMINAL entity, not the head or a middle hop. Its value-object members bind to the terminal entity's fields BY NAME — member name == terminal field name, deliberately NOT extends, so one value object stays collectable from two different entities — and every member must match a terminal field agreeing on BOTH field. and array-ness.", "whenToUse": "A projection needs a value derived by reducing related rows — a count/sum/avg/min/max, a 'did any/every related row match' flag, or an array of collected values. Declare it instead of hand-writing the aggregate query — it stays consistent and regenerates.", "attrs": [ { @@ -3382,14 +3382,14 @@ "all", "collect" ], - "description": "The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup of @of — the field must be isArray)." + "description": "The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted; the field must be isArray)." }, { "name": "distinct", "valueType": "boolean", "isArray": false, "required": false, - "description": "Set (collect-only) to dedupe collected values (set semantics)." + "description": "Set (collect-only) to dedupe collected values (set semantics). Not supported on a whole-object collect (@of omitted): it is a guaranteed no-op whenever the value object carries the primary key, and a silent no-op is worse than a refusal." }, { "name": "filter", @@ -3403,7 +3403,7 @@ "valueType": "string", "isArray": false, "required": false, - "description": "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max/collect; forbidden for any/all (which quantify over rows via @filter, not a column)." + "description": "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max; OPTIONAL for collect, where absent means a whole-object rollup of the field's declared @objectRef value object; forbidden for any/all (which quantify over rows via @filter, not a column)." }, { "name": "orderBy", diff --git a/server/csharp/MetaObjects/Persistence/Origin/OriginSchema.cs b/server/csharp/MetaObjects/Persistence/Origin/OriginSchema.cs index 4fed267f6..5bff8120c 100644 --- a/server/csharp/MetaObjects/Persistence/Origin/OriginSchema.cs +++ b/server/csharp/MetaObjects/Persistence/Origin/OriginSchema.cs @@ -39,7 +39,7 @@ public static class OriginSchema Required: true, // #195 — the full @agg vocabulary (numeric reduces + any/all quantifiers + collect). AllowedValues: [.. OriginConstants.ORIGIN_AGG_VALUES], - Description: "The reducing function applied over the related row-set: count/sum/avg/min/max, any/all (predicate quantifiers over @filter), or collect (array rollup of @of)."), + Description: "The reducing function applied over the related row-set: count/sum/avg/min/max, any/all (predicate quantifiers over @filter), or collect (array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted)."), // #195 — @of relaxed to optional (any/all forbid it, the rest require it); // presence is enforced per-@agg in the origin-path validation pass. @@ -47,7 +47,7 @@ public static class OriginSchema Name: OriginConstants.ORIGIN_AGGREGATE_ATTR_OF, ValueType: AttrConstants.ATTR_SUBTYPE_STRING, Required: false, - Description: "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max/collect; forbidden for any/all."), + Description: "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max; OPTIONAL for collect, where absent means a whole-object rollup of the field's declared @objectRef value object; forbidden for any/all."), new AttrSchema( Name: OriginConstants.ORIGIN_AGGREGATE_ATTR_VIA, @@ -66,7 +66,7 @@ public static class OriginSchema Name: OriginConstants.ORIGIN_ATTR_DISTINCT, ValueType: AttrConstants.ATTR_SUBTYPE_BOOLEAN, Required: false, - Description: "Set (collect-only) to dedupe collected values (set semantics)."), + Description: "Set (collect-only) to dedupe collected values (set semantics). Not supported on a whole-object collect (@of omitted): it is a guaranteed no-op whenever the value object carries the primary key."), new AttrSchema( Name: OriginConstants.ORIGIN_ATTR_ORDER_BY, diff --git a/server/csharp/MetaObjects/SpecMetamodel/origin.json b/server/csharp/MetaObjects/SpecMetamodel/origin.json index e6be6d33b..6ddd1dbb7 100644 --- a/server/csharp/MetaObjects/SpecMetamodel/origin.json +++ b/server/csharp/MetaObjects/SpecMetamodel/origin.json @@ -20,15 +20,15 @@ { "type": "origin", "subType": "aggregate", - "description": "A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup of @of).", + "description": "A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted).", "whenToUse": "A projection needs a value derived by reducing related rows — a count/sum/avg/min/max, a 'did any/every related row match' flag, or an array of collected values. Declare it instead of hand-writing the aggregate query — it stays consistent and regenerates.", - "rules": "@via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max/collect and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only.", + "rules": "@via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only. On @agg:collect @of is OPTIONAL: omitting it declares a WHOLE-OBJECT rollup, which collects each related row as the carrying field's declared @objectRef value object instead of one scalar column. A whole-object rollup requires a field.object carrying @objectRef, requires that @objectRef to name an object.value, requires an explicit @via (there is no @of entity to infer the path from), and refuses @distinct. Its @orderBy keys resolve against the @via TERMINAL entity, not the head or a middle hop. Its value-object members bind to the terminal entity's fields BY NAME — member name == terminal field name, deliberately NOT extends, so one value object stays collectable from two different entities — and every member must match a terminal field agreeing on BOTH field. and array-ness.", "children": [ - { "type": "attr", "subType": "string", "name": "agg", "min": 1, "max": 1, "allowedValues": ["count", "sum", "avg", "min", "max", "any", "all", "collect"], "description": "The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup of @of — the field must be isArray)." }, - { "type": "attr", "subType": "string", "name": "of", "min": 0, "max": 1, "description": "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max/collect; forbidden for any/all (which quantify over rows via @filter, not a column)." }, + { "type": "attr", "subType": "string", "name": "agg", "min": 1, "max": 1, "allowedValues": ["count", "sum", "avg", "min", "max", "any", "all", "collect"], "description": "The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted; the field must be isArray)." }, + { "type": "attr", "subType": "string", "name": "of", "min": 0, "max": 1, "description": "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max; OPTIONAL for collect, where absent means a whole-object rollup of the field's declared @objectRef value object; forbidden for any/all (which quantify over rows via @filter, not a column)." }, { "type": "attr", "subType": "string", "name": "via", "min": 0, "max": 1, "description": "Dotted relationship path from the base entity to the aggregated rows (e.g. 'Program.weeks' or 'Program.weeks.workouts'). May be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (FR-024, ADR-0029)." }, { "type": "attr", "subType": "filter", "name": "filter", "min": 0, "max": 1, "description": "Optional structured predicate scoping which related rows the aggregate spans (required for any/all, where it is the quantified predicate). A portable attr.filter object (eq/ne/in/isNull with and/or), desugared to canonical { field: { op: value } } at parse time; codegen renders it per target (e.g. SQL FILTER (WHERE ...) or SQLite CASE WHEN for a relational view)." }, - { "type": "attr", "subType": "boolean", "name": "distinct", "min": 0, "max": 1, "description": "Set (collect-only) to dedupe collected values (set semantics)." }, + { "type": "attr", "subType": "boolean", "name": "distinct", "min": 0, "max": 1, "description": "Set (collect-only) to dedupe collected values (set semantics). Not supported on a whole-object collect (@of omitted): it is a guaranteed no-op whenever the value object carries the primary key, and a silent no-op is worse than a refusal." }, { "type": "attr", "subType": "string", "name": "orderBy", "isArray": true, "min": 0, "max": 1, "description": "Ordering keys as 'field[:asc|desc]' (default asc) over the related entity's fields; nulls sort last. On @agg:collect sets element order (non-distinct only); on origin.first (required) selects the row. Semantic — carries no SQL syntax." } ] }, diff --git a/server/python/src/metaobjects/spec_metamodel/origin.json b/server/python/src/metaobjects/spec_metamodel/origin.json index e6be6d33b..6ddd1dbb7 100644 --- a/server/python/src/metaobjects/spec_metamodel/origin.json +++ b/server/python/src/metaobjects/spec_metamodel/origin.json @@ -20,15 +20,15 @@ { "type": "origin", "subType": "aggregate", - "description": "A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup of @of).", + "description": "A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted).", "whenToUse": "A projection needs a value derived by reducing related rows — a count/sum/avg/min/max, a 'did any/every related row match' flag, or an array of collected values. Declare it instead of hand-writing the aggregate query — it stays consistent and regenerates.", - "rules": "@via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max/collect and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only.", + "rules": "@via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only. On @agg:collect @of is OPTIONAL: omitting it declares a WHOLE-OBJECT rollup, which collects each related row as the carrying field's declared @objectRef value object instead of one scalar column. A whole-object rollup requires a field.object carrying @objectRef, requires that @objectRef to name an object.value, requires an explicit @via (there is no @of entity to infer the path from), and refuses @distinct. Its @orderBy keys resolve against the @via TERMINAL entity, not the head or a middle hop. Its value-object members bind to the terminal entity's fields BY NAME — member name == terminal field name, deliberately NOT extends, so one value object stays collectable from two different entities — and every member must match a terminal field agreeing on BOTH field. and array-ness.", "children": [ - { "type": "attr", "subType": "string", "name": "agg", "min": 1, "max": 1, "allowedValues": ["count", "sum", "avg", "min", "max", "any", "all", "collect"], "description": "The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup of @of — the field must be isArray)." }, - { "type": "attr", "subType": "string", "name": "of", "min": 0, "max": 1, "description": "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max/collect; forbidden for any/all (which quantify over rows via @filter, not a column)." }, + { "type": "attr", "subType": "string", "name": "agg", "min": 1, "max": 1, "allowedValues": ["count", "sum", "avg", "min", "max", "any", "all", "collect"], "description": "The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted; the field must be isArray)." }, + { "type": "attr", "subType": "string", "name": "of", "min": 0, "max": 1, "description": "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max; OPTIONAL for collect, where absent means a whole-object rollup of the field's declared @objectRef value object; forbidden for any/all (which quantify over rows via @filter, not a column)." }, { "type": "attr", "subType": "string", "name": "via", "min": 0, "max": 1, "description": "Dotted relationship path from the base entity to the aggregated rows (e.g. 'Program.weeks' or 'Program.weeks.workouts'). May be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (FR-024, ADR-0029)." }, { "type": "attr", "subType": "filter", "name": "filter", "min": 0, "max": 1, "description": "Optional structured predicate scoping which related rows the aggregate spans (required for any/all, where it is the quantified predicate). A portable attr.filter object (eq/ne/in/isNull with and/or), desugared to canonical { field: { op: value } } at parse time; codegen renders it per target (e.g. SQL FILTER (WHERE ...) or SQLite CASE WHEN for a relational view)." }, - { "type": "attr", "subType": "boolean", "name": "distinct", "min": 0, "max": 1, "description": "Set (collect-only) to dedupe collected values (set semantics)." }, + { "type": "attr", "subType": "boolean", "name": "distinct", "min": 0, "max": 1, "description": "Set (collect-only) to dedupe collected values (set semantics). Not supported on a whole-object collect (@of omitted): it is a guaranteed no-op whenever the value object carries the primary key, and a silent no-op is worse than a refusal." }, { "type": "attr", "subType": "string", "name": "orderBy", "isArray": true, "min": 0, "max": 1, "description": "Ordering keys as 'field[:asc|desc]' (default asc) over the related entity's fields; nulls sort last. On @agg:collect sets element order (non-distinct only); on origin.first (required) selects the row. Semantic — carries no SQL syntax." } ] }, diff --git a/server/typescript/packages/metadata/src/persistence/origin/origin-definition.embedded.ts b/server/typescript/packages/metadata/src/persistence/origin/origin-definition.embedded.ts index 14d14588d..f01ff4160 100644 --- a/server/typescript/packages/metadata/src/persistence/origin/origin-definition.embedded.ts +++ b/server/typescript/packages/metadata/src/persistence/origin/origin-definition.embedded.ts @@ -49,9 +49,9 @@ export const ORIGIN_DEFINITION: ProviderDefinition = { { "type": "origin", "subType": "aggregate", - "description": "A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup of @of).", + "description": "A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted).", "whenToUse": "A projection needs a value derived by reducing related rows — a count/sum/avg/min/max, a 'did any/every related row match' flag, or an array of collected values. Declare it instead of hand-writing the aggregate query — it stays consistent and regenerates.", - "rules": "@via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max/collect and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only.", + "rules": "@via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only. On @agg:collect @of is OPTIONAL: omitting it declares a WHOLE-OBJECT rollup, which collects each related row as the carrying field's declared @objectRef value object instead of one scalar column. A whole-object rollup requires a field.object carrying @objectRef, requires that @objectRef to name an object.value, requires an explicit @via (there is no @of entity to infer the path from), and refuses @distinct. Its @orderBy keys resolve against the @via TERMINAL entity, not the head or a middle hop. Its value-object members bind to the terminal entity's fields BY NAME — member name == terminal field name, deliberately NOT extends, so one value object stays collectable from two different entities — and every member must match a terminal field agreeing on BOTH field. and array-ness.", "children": [ { "type": "attr", @@ -69,7 +69,7 @@ export const ORIGIN_DEFINITION: ProviderDefinition = { "all", "collect" ], - "description": "The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup of @of — the field must be isArray)." + "description": "The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted; the field must be isArray)." }, { "type": "attr", @@ -77,7 +77,7 @@ export const ORIGIN_DEFINITION: ProviderDefinition = { "name": "of", "min": 0, "max": 1, - "description": "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max/collect; forbidden for any/all (which quantify over rows via @filter, not a column)." + "description": "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max; OPTIONAL for collect, where absent means a whole-object rollup of the field's declared @objectRef value object; forbidden for any/all (which quantify over rows via @filter, not a column)." }, { "type": "attr", @@ -101,7 +101,7 @@ export const ORIGIN_DEFINITION: ProviderDefinition = { "name": "distinct", "min": 0, "max": 1, - "description": "Set (collect-only) to dedupe collected values (set semantics)." + "description": "Set (collect-only) to dedupe collected values (set semantics). Not supported on a whole-object collect (@of omitted): it is a guaranteed no-op whenever the value object carries the primary key, and a silent no-op is worse than a refusal." }, { "type": "attr", diff --git a/spec/metamodel/origin.json b/spec/metamodel/origin.json index e6be6d33b..6ddd1dbb7 100644 --- a/spec/metamodel/origin.json +++ b/spec/metamodel/origin.json @@ -20,15 +20,15 @@ { "type": "origin", "subType": "aggregate", - "description": "A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup of @of).", + "description": "A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted).", "whenToUse": "A projection needs a value derived by reducing related rows — a count/sum/avg/min/max, a 'did any/every related row match' flag, or an array of collected values. Declare it instead of hand-writing the aggregate query — it stays consistent and regenerates.", - "rules": "@via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max/collect and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only.", + "rules": "@via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only. On @agg:collect @of is OPTIONAL: omitting it declares a WHOLE-OBJECT rollup, which collects each related row as the carrying field's declared @objectRef value object instead of one scalar column. A whole-object rollup requires a field.object carrying @objectRef, requires that @objectRef to name an object.value, requires an explicit @via (there is no @of entity to infer the path from), and refuses @distinct. Its @orderBy keys resolve against the @via TERMINAL entity, not the head or a middle hop. Its value-object members bind to the terminal entity's fields BY NAME — member name == terminal field name, deliberately NOT extends, so one value object stays collectable from two different entities — and every member must match a terminal field agreeing on BOTH field. and array-ness.", "children": [ - { "type": "attr", "subType": "string", "name": "agg", "min": 1, "max": 1, "allowedValues": ["count", "sum", "avg", "min", "max", "any", "all", "collect"], "description": "The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup of @of — the field must be isArray)." }, - { "type": "attr", "subType": "string", "name": "of", "min": 0, "max": 1, "description": "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max/collect; forbidden for any/all (which quantify over rows via @filter, not a column)." }, + { "type": "attr", "subType": "string", "name": "agg", "min": 1, "max": 1, "allowedValues": ["count", "sum", "avg", "min", "max", "any", "all", "collect"], "description": "The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted; the field must be isArray)." }, + { "type": "attr", "subType": "string", "name": "of", "min": 0, "max": 1, "description": "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max; OPTIONAL for collect, where absent means a whole-object rollup of the field's declared @objectRef value object; forbidden for any/all (which quantify over rows via @filter, not a column)." }, { "type": "attr", "subType": "string", "name": "via", "min": 0, "max": 1, "description": "Dotted relationship path from the base entity to the aggregated rows (e.g. 'Program.weeks' or 'Program.weeks.workouts'). May be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (FR-024, ADR-0029)." }, { "type": "attr", "subType": "filter", "name": "filter", "min": 0, "max": 1, "description": "Optional structured predicate scoping which related rows the aggregate spans (required for any/all, where it is the quantified predicate). A portable attr.filter object (eq/ne/in/isNull with and/or), desugared to canonical { field: { op: value } } at parse time; codegen renders it per target (e.g. SQL FILTER (WHERE ...) or SQLite CASE WHEN for a relational view)." }, - { "type": "attr", "subType": "boolean", "name": "distinct", "min": 0, "max": 1, "description": "Set (collect-only) to dedupe collected values (set semantics)." }, + { "type": "attr", "subType": "boolean", "name": "distinct", "min": 0, "max": 1, "description": "Set (collect-only) to dedupe collected values (set semantics). Not supported on a whole-object collect (@of omitted): it is a guaranteed no-op whenever the value object carries the primary key, and a silent no-op is worse than a refusal." }, { "type": "attr", "subType": "string", "name": "orderBy", "isArray": true, "min": 0, "max": 1, "description": "Ordering keys as 'field[:asc|desc]' (default asc) over the related entity's fields; nulls sort last. On @agg:collect sets element order (non-distinct only); on origin.first (required) selects the row. Semantic — carries no SQL syntax." } ] }, From d006d55414f7897f7e6d55ca3ad6c8541695796f Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 23 Aug 2026 15:49:48 -0400 Subject: [PATCH 22/25] feat(codegen-ts): lower an @of-less collect to a jsonb whole-object rollup (#335) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CLOSES THE MERGE GATE. Since the loader started accepting an @of-absent @agg:collect, `extract-view-spec.ts` still bailed on `if (!of_) continue;` ABOVE the collect branch — so the metadata LOADED clean and codegen silently DROPPED the column. Proven empirically before this commit: the positive conformance fixture extracted `columns = [productId]` with `supplierBriefs` simply gone, i.e. a CREATE VIEW missing a column the generated type still declares. This branch was not mergeable until this landed. A new `collectObjectAgg` SelectColumn kind carries a MEMBER LIST rather than one source column — deliberately a separate kind, not an arm of `collectAgg`: a union would force `isInflationSensitive` and `buildGroupBy` to re-narrow. Both are updated; a whole-object collect is unconditionally inflation-sensitive, because the loader refuses @distinct on this form, so it can never be the distinct case that makes `collectAgg` safe. Extraction resolves the related entity from @via's TERMINAL hop, exactly as the any/all arm does — there is no @of entity. The value object's declared member list IS the exposure: a field the terminal entity has but the VO omits is not projected, which is the #270 guarantee this rollup has to keep. The JSON key is the MEMBER name while the value reads the terminal entity's PHYSICAL column; those differ whenever a field carries @column, and a test pins that they stay distinct. SQL: `jsonb`, not `json`. Verified against a real PG 15 — `json` has neither an equality nor an ordering operator, so the `json_agg(json_build_object(…) ORDER BY …)` form does not run at all. Postgres gets COALESCE(jsonb_agg(jsonb_build_object(…) ORDER BY ASC) FILTER (WHERE IS NOT NULL), '[]'::jsonb); SQLite gets json_group_array(json_object(…)) with json_array() as the empty value. In-aggregate ORDER BY needs SQLite >= 3.44, which is not a new constraint — the scalar collect already emits it and D1's baseline is pinned there. Default element order is the related entity's PK ascending, not "value ascending": ordering rows by a serialized object is meaningless. An explicit @orderBy leads with the PK appended as a tie-break. The SCALAR arm keeps its no-tie-break behaviour deliberately — adding one would change emitted SQL for every existing project using @orderBy — and three no-churn tests pin that asymmetry, byte for byte, in the same file. Tasks 10 and 11 of the plan are ONE commit rather than two. Splitting them would leave an intermediate commit that does not BUILD: adding the union member without its emit branch fails `tsc` with TS2345 where view-ddl-emit falls through to renderFirst. That exhaustiveness failure is a feature of the union — it is what guarantees no kind can be added and silently unlowered — but it makes the split commit a bisect trap. codegen-ts 1312 pass / 0 fail (7 new DDL + 5 new extract), package typecheck clean, whole-workspace `bun run --filter '*' build` and `typecheck` clean. The two issue-214-read-half-compile failures seen mid-task were the documented unbuilt-workspace precondition (missing `@metaobjectsdev/runtime-ts/drizzle-fastify`), not a regression — they pass after the build. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq --- .../src/projection/build-projection-views.ts | 9 +- .../src/projection/extract-view-spec.ts | 65 +++++++- .../src/projection/view-ddl-emit.ts | 27 ++++ .../codegen-ts/src/projection/view-spec.ts | 25 +++ .../collect-whole-object-ddl.test.ts | 126 +++++++++++++++ .../collect-whole-object-spec.test.ts | 144 ++++++++++++++++++ 6 files changed, 389 insertions(+), 7 deletions(-) create mode 100644 server/typescript/packages/codegen-ts/test/projection/collect-whole-object-ddl.test.ts create mode 100644 server/typescript/packages/codegen-ts/test/projection/collect-whole-object-spec.test.ts diff --git a/server/typescript/packages/codegen-ts/src/projection/build-projection-views.ts b/server/typescript/packages/codegen-ts/src/projection/build-projection-views.ts index 91de41102..cd8a850d9 100644 --- a/server/typescript/packages/codegen-ts/src/projection/build-projection-views.ts +++ b/server/typescript/packages/codegen-ts/src/projection/build-projection-views.ts @@ -315,10 +315,11 @@ function collectViewColumns( const out: ExpectedViewColumn[] = []; for (const c of spec.selectSpec.columns) { - // #195: the four new origin column kinds (predicateAgg/collectAgg/computed/first) do - // not resolve to a single (table, column) SqlType via the prefix rule — computed is - // an expression, first is a correlated subquery, and the array/boolean aggregate - // result types are richer than the OR-REPLACE prefix check models. Per this module's + // #195/#335: the five non-scalar origin column kinds (predicateAgg / collectAgg / + // collectObjectAgg / computed / first) do not resolve to a single (table, column) + // SqlType via the prefix rule — computed is an expression, first is a correlated + // subquery, and the array/jsonb/boolean aggregate result types are richer than the + // OR-REPLACE prefix check models. Per this module's // fail-safe doctrine (unknown → drop+create, never a wrong-but-confident replace), // an unknown column drops the whole list so migrate routes through a gated // drop+create. Precise native typing is a later phase. diff --git a/server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts b/server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts index 3654d3a52..94e9d7497 100644 --- a/server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts +++ b/server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts @@ -1,5 +1,6 @@ import { TYPE_FIELD, + FIELD_ATTR_OBJECT_REF, TYPE_IDENTITY, TYPE_ORIGIN, TYPE_RELATIONSHIP, @@ -886,6 +887,59 @@ function buildSelectSpec( continue; } + // #335 — @of ABSENT on collect is a WHOLE-OBJECT rollup. This arm sits ABOVE + // the `if (!of_) continue;` guard on purpose: that guard used to swallow this + // shape, so the metadata LOADED and the column was silently dropped from the + // CREATE VIEW while the generated type still declared the field. + // The related entity comes from @via's terminal hop, not from @of — exactly as + // the any/all arm above resolves it. + const wholeObjectCollect = + agg === AGG_COLLECT && origin.ownAttr(ORIGIN_AGGREGATE_ATTR_OF) === undefined; + if (wholeObjectCollect) { + // Every `continue` below is a loader-guaranteed impossibility (the load fails + // with ERR_COLLECT_WHOLE_OBJECT / ERR_SUBTYPE_RULE_VIOLATION / + // ERR_COLLECT_MEMBER_UNRESOLVED before codegen runs); they are defence for a + // caller that reached codegen without loading, never a silent-drop path. + const via = origin.ownAttr(ORIGIN_AGGREGATE_ATTR_VIA) as string | undefined; + if (!via) continue; + // ADR-0039: resolving — @objectRef may be inherited via extends. + const objectRef = field.attr(FIELD_ATTR_OBJECT_REF) as string | undefined; + if (!objectRef) continue; + const relatedName = viaTerminalEntity(via, root, projPkg); + if (!relatedName) continue; + const relatedEntity = resolveEntityRef(root, relatedName, projPkg); + if (!relatedEntity) continue; + const sourceAlias = findAliasInTree(joinTree, relatedEntity.resolutionKey()); + if (sourceAlias === undefined) continue; + const joinedPk = primaryKeyColumn(relatedEntity, ctx); + if (joinedPk === undefined) continue; + // ADR-0042 — a bare @objectRef resolves in the projection's package. + const vo = resolveObjectRef(root, objectRef, projPkg).node; + if (!vo) continue; + // ADR-0039: resolving — a value object may inherit members via extends, and the + // terminal entity may inherit fields. The VO's member list IS the exposure: a + // field the terminal has but the VO omits is deliberately not projected (#270). + const members: { memberName: string; sourceColumn: string }[] = []; + for (const m of vo.children().filter((c): c is MetaField => c.type === TYPE_FIELD)) { + const target = relatedEntity.fields().find((f) => f.name === m.name); + if (target === undefined) continue; // loader: ERR_COLLECT_MEMBER_UNRESOLVED + members.push({ memberName: m.name, sourceColumn: sourceColumnNameFor(target, ctx) }); + } + if (members.length === 0) continue; + columns.push({ + kind: "collectObjectAgg", + fieldName: field.name, + dbColAlias: dbCol, + sourceAlias, + joinedPkColumn: joinedPk, + members, + // @orderBy resolves against the @via TERMINAL entity, not @of — there is no + // @of entity. Empty ⇒ the emitter's default of related-PK ascending. + orderBy: resolveOrderByKeys(origin.ownAttr(ORIGIN_ATTR_ORDER_BY), relatedEntity, ctx), + }); + continue; + } + // collect + the scalar reduces (count/sum/avg/min/max) all name @of. const of_ = origin.ownAttr(ORIGIN_AGGREGATE_ATTR_OF) as string | undefined; if (!of_) continue; @@ -1003,6 +1057,9 @@ function buildSelectSpec( function isInflationSensitive(c: SelectColumn): boolean { if (c.kind === "aggregate") return c.agg === "sum" || c.agg === "avg"; if (c.kind === "collectAgg") return !c.distinct; + // #335 — a whole-object collect is ALWAYS non-distinct (the loader refuses @distinct + // on this form), so it is unconditionally inflation-sensitive. + if (c.kind === "collectObjectAgg") return true; return false; } @@ -1015,10 +1072,12 @@ function countManyBranches(joinTree: JoinTree): number { } function buildGroupBy(spec: SelectSpec): string[] { - // predicateAgg (bool_or/bool_and) and collectAgg (array_agg) are real aggregates and - // force GROUP BY too; computed/first are scalar-per-row and never grouped. + // predicateAgg (bool_or/bool_and), collectAgg (array_agg) and collectObjectAgg + // (jsonb_agg) are real aggregates and force GROUP BY too; computed/first are + // scalar-per-row and never grouped. const hasAgg = spec.columns.some( - (c) => c.kind === "aggregate" || c.kind === "predicateAgg" || c.kind === "collectAgg", + (c) => c.kind === "aggregate" || c.kind === "predicateAgg" + || c.kind === "collectAgg" || c.kind === "collectObjectAgg", ); if (!hasAgg) return []; return spec.columns diff --git a/server/typescript/packages/codegen-ts/src/projection/view-ddl-emit.ts b/server/typescript/packages/codegen-ts/src/projection/view-ddl-emit.ts index 91c2c1e69..c98d0d9b6 100644 --- a/server/typescript/packages/codegen-ts/src/projection/view-ddl-emit.ts +++ b/server/typescript/packages/codegen-ts/src/projection/view-ddl-emit.ts @@ -200,6 +200,33 @@ function renderColumn(c: SelectColumn, options: EmitOptions, baseAlias: string): return `COALESCE(array_agg(${distinctKw}${src} ${orderClause}) FILTER (WHERE ${guard}), '{}') AS ${alias}`; } + if (c.kind === "collectObjectAgg") { + // #335 whole-object rollup. jsonb, not json: PG's `json` has neither an equality + // nor an ordering operator, so `json_agg(json_build_object(…) ORDER BY …)` does not + // run — verified against a real PG 15. + const guard = `${c.sourceAlias}.${quoteIfNeeded(c.joinedPkColumn)} IS NOT NULL`; + const pk = `${c.sourceAlias}.${quoteIfNeeded(c.joinedPkColumn)}`; + // Element order: the related entity's PK ascending by default — "value ascending" + // is meaningless for an object (and does not parse on PG json). An explicit + // @orderBy leads, with the PK appended as a tie-break so equal-order rows stay + // byte-deterministic. The SCALAR arm above deliberately keeps its no-tie-break + // behaviour: adding one there would alter emitted SQL for every existing project. + const orderClause = c.orderBy.length > 0 + ? `ORDER BY ${renderOrderKeys(c.orderBy, c.sourceAlias)}, ${pk} ASC` + : `ORDER BY ${pk} ASC`; + // The JSON key is the VO MEMBER name; the value reads the TERMINAL entity's + // physical column. Those two differ whenever a field carries @column. + const pairs = c.members + .map((m) => `'${m.memberName}', ${c.sourceAlias}.${quoteIfNeeded(m.sourceColumn)}`) + .join(", "); + // In-aggregate ORDER BY needs SQLite >= 3.44 — not a new constraint: the scalar + // collect above already emits it, and D1's baseline is pinned at 3.44.0. + if (dialect === "sqlite") { + return `COALESCE(json_group_array(json_object(${pairs}) ${orderClause}) FILTER (WHERE ${guard}), json_array()) AS ${alias}`; + } + return `COALESCE(jsonb_agg(jsonb_build_object(${pairs}) ${orderClause}) FILTER (WHERE ${guard}), '[]'::jsonb) AS ${alias}`; + } + if (c.kind === "computed") { return `${renderExpr(c.expr, dialect)} AS ${alias}`; } diff --git a/server/typescript/packages/codegen-ts/src/projection/view-spec.ts b/server/typescript/packages/codegen-ts/src/projection/view-spec.ts index 40fadc264..295674823 100644 --- a/server/typescript/packages/codegen-ts/src/projection/view-spec.ts +++ b/server/typescript/packages/codegen-ts/src/projection/view-spec.ts @@ -124,6 +124,31 @@ export type SelectColumn = /** Element ordering over the @of entity's columns; empty ⇒ value-ascending default. */ readonly orderBy: readonly ViewOrderKey[]; } + | { + // #335 — origin.aggregate @agg:collect with NO @of — a WHOLE-OBJECT rollup: each + // related row is collected as the carrying field.object's declared @objectRef value + // object rather than as one scalar column. Lowered to + // COALESCE(jsonb_agg(jsonb_build_object(...) ORDER BY ASC) FILTER (WHERE + // joined.pk IS NOT NULL), '[]'::jsonb) on PG; json_group_array(json_object(...)) on + // SQLite. Empty set → []. Default element order is the RELATED entity's PK ascending + // — ordering rows by a serialized object is meaningless, and on PG `json` it does not + // even parse (no ordering operator), which is also why the column is jsonb not json. + // + // A separate kind rather than an arm of collectAgg: the payloads differ (a member + // list vs one source column), so a union would force every consumer to re-narrow. + // @distinct never appears — the loader refuses it on this form. + readonly kind: "collectObjectAgg"; + readonly fieldName: string; + readonly dbColAlias: string; + readonly sourceAlias: string; + readonly joinedPkColumn: string; // related entity's PK column — the LEFT-JOIN phantom guard + /** The declared value object's members, in declaration order. `memberName` is the + * emitted JSON key; `sourceColumn` is the TERMINAL entity's physical column it reads. + * The loader guarantees every member resolves (ERR_COLLECT_MEMBER_UNRESOLVED). */ + readonly members: readonly { readonly memberName: string; readonly sourceColumn: string }[]; + /** Element ordering over the @via terminal entity's columns; empty ⇒ PK ascending. */ + readonly orderBy: readonly ViewOrderKey[]; + } | { // #195 — origin.computed — a row-level value from the base entity's own fields via // a structured @expr tree (no related rows). Lowered by a tree-walk to a SQL scalar diff --git a/server/typescript/packages/codegen-ts/test/projection/collect-whole-object-ddl.test.ts b/server/typescript/packages/codegen-ts/test/projection/collect-whole-object-ddl.test.ts new file mode 100644 index 000000000..102fa7760 --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/projection/collect-whole-object-ddl.test.ts @@ -0,0 +1,126 @@ +// #335 — SQL lowering for a WHOLE-OBJECT `origin.aggregate @agg:collect` column. +// +// jsonb, not json, on Postgres: verified against PG 15 — `json` has neither an +// equality nor an ordering operator, so `json_agg(json_build_object(…) ORDER BY …)` +// does not run at all. `field.object` is already jsonb elsewhere in codegen. +// +// Element order defaults to the RELATED entity's PK ascending, not "value +// ascending": ordering rows by a serialized object is meaningless, and on PG json +// it does not even parse. An explicit @orderBy leads with the PK appended as a +// tie-break so equal-order rows stay byte-deterministic. + +import { describe, test, expect } from "bun:test"; +import type { ViewSpec, ViewOrderKey } from "../../src/projection/view-spec.js"; +import { emitViewDdl } from "../../src/projection/view-ddl-emit.js"; + +const OPTS = { baseTableName: "products", joinTables: { Supplier: "suppliers" } } as const; + +function wholeObject(orderBy: ViewOrderKey[] = []): ViewSpec { + return { + viewName: "v_product_summary", + joinTree: { + baseEntity: "Product", + baseAlias: "p", + joins: [ + { relationship: "suppliers", targetEntity: "Supplier", alias: "s", cardinality: "many", + fkColumn: "product_id", pkColumn: "id", referenceHolder: "target", joinType: "left", children: [] }, + ], + }, + selectSpec: { + columns: [ + { kind: "passthrough", fieldName: "id", dbColAlias: "id", sourceAlias: "p", sourceColumn: "id" }, + { + kind: "collectObjectAgg", + fieldName: "supplierBriefs", + dbColAlias: "supplier_briefs", + sourceAlias: "s", + joinedPkColumn: "id", + members: [ + { memberName: "id", sourceColumn: "id" }, + { memberName: "name", sourceColumn: "supplier_name" }, + ], + orderBy, + }, + ], + }, + groupBy: ["p.id"], + }; +} + +describe("emitViewDdl — #335 whole-object collect", () => { + test("postgres: jsonb_agg of jsonb_build_object, related-PK ascending, empty-set guarded", () => { + const sql = emitViewDdl(wholeObject(), { dialect: "postgres", ...OPTS }); + expect(sql).toContain( + "COALESCE(jsonb_agg(jsonb_build_object('id', s.id, 'name', s.supplier_name) ORDER BY s.id ASC) " + + "FILTER (WHERE s.id IS NOT NULL), '[]'::jsonb) AS supplier_briefs", + ); + }); + + test("sqlite: json_group_array of json_object, related-PK ascending, empty-set guarded", () => { + const sql = emitViewDdl(wholeObject(), { dialect: "sqlite", ...OPTS }); + expect(sql).toContain( + "COALESCE(json_group_array(json_object('id', s.id, 'name', s.supplier_name) ORDER BY s.id ASC) " + + "FILTER (WHERE s.id IS NOT NULL), json_array()) AS supplier_briefs", + ); + }); + + test("an explicit @orderBy leads and the related PK is appended as tie-break", () => { + const sql = emitViewDdl(wholeObject([{ column: "supplier_name", dir: "desc" }]), + { dialect: "postgres", ...OPTS }); + expect(sql).toContain("ORDER BY s.supplier_name DESC NULLS LAST, s.id ASC)"); + }); + + test("the JSON key is the VO MEMBER name; the column read is the terminal entity's", () => { + // `name` -> s.supplier_name. If these two ever collapse into one identifier the + // rollup starts emitting the physical column name into the JSON payload, which the + // generated type would not match. + const sql = emitViewDdl(wholeObject(), { dialect: "postgres", ...OPTS }); + expect(sql).toContain("'name', s.supplier_name"); + expect(sql).not.toContain("'supplier_name', s.supplier_name"); + }); +}); + +// The scalar @of arm shares this branch neighbourhood. Its emitted SQL is in every +// existing project's committed migrations, so it must not move by so much as a byte. +describe("emitViewDdl — #335 no-churn pin on the SCALAR collect arm", () => { + const scalar = (distinct: boolean, orderBy: ViewOrderKey[]): ViewSpec => ({ + viewName: "v_order_summary", + joinTree: { + baseEntity: "Order", + baseAlias: "o", + joins: [ + { relationship: "items", targetEntity: "Item", alias: "i", cardinality: "many", + fkColumn: "order_id", pkColumn: "id", referenceHolder: "target", joinType: "left", children: [] }, + ], + }, + selectSpec: { + columns: [ + { kind: "passthrough", fieldName: "id", dbColAlias: "id", sourceAlias: "o", sourceColumn: "id" }, + { kind: "collectAgg", fieldName: "categories", dbColAlias: "categories", + sourceAlias: "i", sourceColumn: "category", joinedPkColumn: "id", distinct, orderBy }, + ], + }, + groupBy: ["o.id"], + }); + const opts = { baseTableName: "orders", joinTables: { Item: "items" } } as const; + + test("postgres: array_agg, value-ascending default, '{}' empty — unchanged", () => { + expect(emitViewDdl(scalar(false, []), { dialect: "postgres", ...opts })).toContain( + "COALESCE(array_agg(i.category ORDER BY i.category ASC) FILTER (WHERE i.id IS NOT NULL), '{}') AS categories", + ); + }); + + test("postgres: an explicit @orderBy still gets NO PK tie-break — unchanged", () => { + // Deliberately asymmetric with the whole-object arm above: adding a tie-break here + // would change the emitted SQL of every existing project that uses @orderBy. + expect(emitViewDdl(scalar(false, [{ column: "category", dir: "desc" }]), { dialect: "postgres", ...opts })).toContain( + "COALESCE(array_agg(i.category ORDER BY i.category DESC NULLS LAST) FILTER (WHERE i.id IS NOT NULL), '{}') AS categories", + ); + }); + + test("sqlite: json_group_array / json_array() — unchanged", () => { + expect(emitViewDdl(scalar(false, []), { dialect: "sqlite", ...opts })).toContain( + "COALESCE(json_group_array(i.category ORDER BY i.category ASC) FILTER (WHERE i.id IS NOT NULL), json_array()) AS categories", + ); + }); +}); diff --git a/server/typescript/packages/codegen-ts/test/projection/collect-whole-object-spec.test.ts b/server/typescript/packages/codegen-ts/test/projection/collect-whole-object-spec.test.ts new file mode 100644 index 000000000..21eb6c230 --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/projection/collect-whole-object-spec.test.ts @@ -0,0 +1,144 @@ +// #335 — extractViewSpec for a WHOLE-OBJECT `origin.aggregate @agg:collect` +// (no `@of`): the related rows roll up as the carrying field.object's declared +// `@objectRef` value object rather than as one scalar column. +// +// The pre-#335 extractor bailed on `if (!of_) continue;` ABOVE the collect +// branch, so this metadata LOADED and codegen silently dropped the column — a +// CREATE VIEW with no such column while the generated type still declared the +// field. That is the failure this file exists to prevent, so the "column is +// absent" assertions below are load-bearing, not incidental. + +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { extractViewSpec } from "../../src/projection/extract-view-spec.js"; + +async function load(children: unknown[]) { + const json = JSON.stringify({ "metadata.root": { package: "test", children } }); + const result = await new MetaDataLoader().load([new InMemoryStringSource(json)]); + if (result.errors.length > 0) { + throw new Error(`Loader errors:\n${result.errors.map((e) => e.message).join("\n")}`); + } + return result.root; +} + +/** Product 1—* Supplier; SupplierBrief is the curated value object collected per product. + * `voFields` and `orderBy` vary per test; everything else is fixed. */ +function model(opts: { voFields?: unknown[]; orderBy?: string[] } = {}) { + const voFields = opts.voFields ?? [ + { "field.int": { name: "id" } }, + { "field.string": { name: "name" } }, + ]; + const origin: Record = { "@agg": "collect", "@via": "Product.suppliers" }; + if (opts.orderBy !== undefined) origin["@orderBy"] = opts.orderBy; + return [ + { + "object.entity": { + name: "Product", + children: [ + { "source.rdb": { "@table": "products" } }, + { "field.int": { name: "id" } }, + { "field.string": { name: "title" } }, + { "identity.primary": { name: "id", "@fields": "id" } }, + { "relationship.association": { name: "suppliers", "@objectRef": "Supplier", "@cardinality": "many" } }, + ], + }, + }, + { + "object.entity": { + name: "Supplier", + children: [ + { "source.rdb": { "@table": "suppliers" } }, + { "field.int": { name: "id" } }, + { "field.string": { name: "name" } }, + { "field.string": { name: "region" } }, + { "field.int": { name: "productId" } }, + { "identity.primary": { name: "id", "@fields": "id" } }, + { "identity.reference": { name: "ref_product", "@fields": "productId", "@references": "Product" } }, + ], + }, + }, + { "object.value": { name: "SupplierBrief", children: voFields } }, + { + "object.projection": { + name: "ProductSummary", + children: [ + { "source.rdb": { "@kind": "view", "@table": "v_product_summary" } }, + { "field.int": { name: "id", extends: "Product.id" } }, + { "identity.primary": { name: "id", extends: "Product.id" } }, + { + "field.object": { + name: "supplierBriefs", + isArray: true, + "@objectRef": "SupplierBrief", + children: [{ "origin.aggregate": origin }], + }, + }, + ], + }, + }, + ]; +} + +async function specFor(opts: Parameters[0] = {}) { + const root = await load(model(opts)); + const projection = root.objects().find((o) => o.name === "ProductSummary")!; + return extractViewSpec(projection, root, { columnNamingStrategy: "snake_case" }); +} + +describe("extractViewSpec — #335 whole-object collect", () => { + test("an @of-less collect resolves a collectObjectAgg column carrying the VO's members", async () => { + const spec = await specFor(); + const col = spec.selectSpec.columns.find((c) => c.fieldName === "supplierBriefs"); + expect(col).toBeDefined(); + expect(col!.kind).toBe("collectObjectAgg"); + if (col!.kind !== "collectObjectAgg") return; + expect(col!.members).toEqual([ + { memberName: "id", sourceColumn: "id" }, + { memberName: "name", sourceColumn: "name" }, + ]); + expect(col!.orderBy).toEqual([]); // default = related PK ascending, applied at emit + expect(col!.joinedPkColumn).toBe("id"); + expect(col!.dbColAlias).toBe("supplier_briefs"); + }); + + test("the VO's member list is the exposure — a member the VO omits is not projected", async () => { + // Supplier declares `region`; SupplierBrief does not. The column must carry two + // members, not three: the declared value object IS the projection, which is the + // #270 guarantee this rollup has to keep. + const spec = await specFor(); + const col = spec.selectSpec.columns.find((c) => c.fieldName === "supplierBriefs")!; + if (col.kind !== "collectObjectAgg") throw new Error("expected collectObjectAgg"); + expect(col.members.map((m) => m.memberName)).toEqual(["id", "name"]); + expect(col.members.map((m) => m.memberName)).not.toContain("region"); + }); + + test("a member's sourceColumn is the TERMINAL entity's physical column, not the member name", async () => { + // Supplier.name carries @column "supplier_name" — the emitted SQL must read that + // column while the JSON key stays the VO member name. + const children = model(); + const supplier = children[1] as { "object.entity": { children: Record[] } }; + supplier["object.entity"].children[2] = { "field.string": { name: "name", "@column": "supplier_name" } }; + const root = await load(children); + const projection = root.objects().find((o) => o.name === "ProductSummary")!; + const spec = extractViewSpec(projection, root, { columnNamingStrategy: "snake_case" }); + const col = spec.selectSpec.columns.find((c) => c.fieldName === "supplierBriefs")!; + if (col.kind !== "collectObjectAgg") throw new Error("expected collectObjectAgg"); + expect(col.members).toEqual([ + { memberName: "id", sourceColumn: "id" }, + { memberName: "name", sourceColumn: "supplier_name" }, + ]); + }); + + test("@orderBy keys resolve against the @via terminal entity", async () => { + const spec = await specFor({ orderBy: ["name:desc"] }); + const col = spec.selectSpec.columns.find((c) => c.fieldName === "supplierBriefs")!; + if (col.kind !== "collectObjectAgg") throw new Error("expected collectObjectAgg"); + expect(col.orderBy).toEqual([{ column: "name", dir: "desc" }]); + }); + + test("it is a real aggregate — the base passthrough columns are GROUPed BY", async () => { + const spec = await specFor(); + expect(spec.groupBy.length).toBeGreaterThan(0); + expect(spec.groupBy).toContain(`${spec.joinTree.baseAlias}.id`); + }); +}); From f02f132916b7026543c29f348a9eecfcaca990da Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 23 Aug 2026 16:03:18 -0400 Subject: [PATCH 23/25] test(integration): round-trip the whole-object rollup on real PG and SQLite (#335) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Golden SQL is not evidence for new DDL, and this cut is the proof of why. Both view-lifecycle probes gain a `WeekBrief` value object and a `weekBriefs` column carrying an @of-less collect, beside the existing scalar `weekLabels` collect so both arms run through one view on one engine. Each asserts the full emit -> apply -> introspect -> re-diff-is-EMPTY convergence gate, then the actual values: two related rows give an array of objects in related-PK order, zero related rows give [] and not null. The SQLite emitter is CHANGED here, by what the probe found. On SQLite 3.44.0 — D1's pinned baseline — the in-aggregate ORDER BY clause destroys the JSON subtype: json_group_array(json_object(...)) -> nests correctly json_group_array(json_object(...) ORDER BY ...) -> array of QUOTED STRINGS json_group_array(json(json_object(...)) ORDER BY ...)-> array of QUOTED STRINGS So the column came back as ["{\"id\":1,...}", ...] — an array of strings — while the generated type declared an array of objects, and a json() wrapper on the argument does not survive the ORDER BY either. Dropping the ORDER BY was not an option: element order would stop being deterministic and an author's @orderBy would silently do nothing. The emitter now builds the ordered array first and re-wraps it element by element through json_each, which iterates in array order, so the ordering survives while json(value) restores each element's subtype. Still a grouped LEFT JOIN — unlike origin.first it needs no correlation info, so a multi-hop @via lowers exactly as a single-hop one does. Postgres was already correct and is verified, not assumed by symmetry. The unit test in codegen-ts is updated to pin the new SQLite shape, so the two tiers cannot drift. Second finding, pinned rather than normalised away: a `field.long` member arrives inside the rollup as a JSON NUMBER, while the same value as a top-level BIGINT column arrives from node-postgres as a STRING. That asymmetry is inherent to JSON — jsonb has no bigint — and is lossy above 2^53. Both id types are asserted explicitly with a comment, so a future change to either tier has to face it. Ran against a throwaway Testcontainers Postgres, not a local one: this box has sibling-project Postgres containers up, `METAOBJECTS_TEST_PG_URL` was confirmed unset, and startPostgres() provisions its own `metaobjects-test-` container on a fresh port. view-lifecycle-pg 17/0, view-lifecycle-sqlite 1/0, codegen-ts projection suite 148/0, package typecheck clean. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq --- .../src/projection/view-ddl-emit.ts | 29 +++++++++++++- .../collect-whole-object-ddl.test.ts | 10 ++++- .../test/view-lifecycle-pg.test.ts | 38 ++++++++++++++++++- .../test/view-lifecycle-sqlite.test.ts | 26 ++++++++++++- 4 files changed, 96 insertions(+), 7 deletions(-) diff --git a/server/typescript/packages/codegen-ts/src/projection/view-ddl-emit.ts b/server/typescript/packages/codegen-ts/src/projection/view-ddl-emit.ts index c98d0d9b6..3e4a36c49 100644 --- a/server/typescript/packages/codegen-ts/src/projection/view-ddl-emit.ts +++ b/server/typescript/packages/codegen-ts/src/projection/view-ddl-emit.ts @@ -68,6 +68,11 @@ function renderFilterCond(clause: ViewFilterClause, dialect: EmitOptions["dialec return `${lhs} ${op} ${sqlLiteral(clause.value, dialect)}`; } +/** Subquery alias for the json_each re-wrap in the SQLite whole-object collect. Scoped to + * its own scalar subquery, so it cannot collide with a JOIN-tree alias; named distinctly + * anyway so it is obvious in emitted DDL where it came from. */ +const JSON_EACH_ALIAS = "mo_je"; + /** * Render resolved ordering keys, applying the #195 nulls-last pin (`NULLS LAST` in * both directions — PG + SQLite ≥ 3.30). `alias` qualifies each key's column. @@ -222,8 +227,30 @@ function renderColumn(c: SelectColumn, options: EmitOptions, baseAlias: string): // In-aggregate ORDER BY needs SQLite >= 3.44 — not a new constraint: the scalar // collect above already emits it, and D1's baseline is pinned at 3.44.0. if (dialect === "sqlite") { - return `COALESCE(json_group_array(json_object(${pairs}) ${orderClause}) FILTER (WHERE ${guard}), json_array()) AS ${alias}`; + // SQLite cannot do BOTH in-aggregate ORDER BY and JSON nesting in one call. + // Measured on SQLite 3.44.0, D1's pinned baseline: + // + // json_group_array(json_object(…)) -> nests correctly + // json_group_array(json_object(…) ORDER BY …) -> array of QUOTED STRINGS + // json_group_array(json(json_object(…)) ORDER BY …) -> array of QUOTED STRINGS + // + // The ORDER BY clause itself destroys the JSON subtype, and a json() wrapper on the + // argument does not survive it. Dropping ORDER BY is not an option: element order + // would stop being deterministic and an author's @orderBy would silently do nothing. + // + // So build the ordered array first (elements quoted), then re-wrap element by + // element through json_each — which iterates in ARRAY ORDER, so the ordering + // survives while json(value) restores each element's JSON subtype. Still a grouped + // LEFT JOIN: unlike origin.first this needs no correlation info, so a multi-hop + // @via lowers here exactly as a single-hop one does. + // + // Found by the real-engine probe in integration-tests. Emitted SQL text cannot show + // this, which is exactly why golden SQL is not evidence for new DDL. + const ordered = `COALESCE(json_group_array(json_object(${pairs}) ${orderClause}) FILTER (WHERE ${guard}), json_array())`; + return `(SELECT json_group_array(json(${JSON_EACH_ALIAS}.value)) FROM json_each(${ordered}) ${JSON_EACH_ALIAS}) AS ${alias}`; } + // PG's jsonb_build_object already yields real jsonb, so jsonb_agg nests it correctly + // with no wrapper — verified against a real engine, not assumed by symmetry. return `COALESCE(jsonb_agg(jsonb_build_object(${pairs}) ${orderClause}) FILTER (WHERE ${guard}), '[]'::jsonb) AS ${alias}`; } diff --git a/server/typescript/packages/codegen-ts/test/projection/collect-whole-object-ddl.test.ts b/server/typescript/packages/codegen-ts/test/projection/collect-whole-object-ddl.test.ts index 102fa7760..9beb42098 100644 --- a/server/typescript/packages/codegen-ts/test/projection/collect-whole-object-ddl.test.ts +++ b/server/typescript/packages/codegen-ts/test/projection/collect-whole-object-ddl.test.ts @@ -56,11 +56,17 @@ describe("emitViewDdl — #335 whole-object collect", () => { ); }); - test("sqlite: json_group_array of json_object, related-PK ascending, empty-set guarded", () => { + // SQLite needs the json_each re-wrap. Measured on 3.44.0: the in-aggregate ORDER BY + // destroys the JSON subtype, so json_group_array(json_object(...) ORDER BY ...) returns + // an array of QUOTED STRINGS — and a json() wrapper on the argument does not survive it + // either. json_each iterates in array order, so re-wrapping element-by-element restores + // the objects while keeping the ordering. Found by the real-engine probe. + test("sqlite: the ordered array is re-wrapped through json_each so elements stay OBJECTS", () => { const sql = emitViewDdl(wholeObject(), { dialect: "sqlite", ...OPTS }); expect(sql).toContain( + "(SELECT json_group_array(json(mo_je.value)) FROM json_each(" + "COALESCE(json_group_array(json_object('id', s.id, 'name', s.supplier_name) ORDER BY s.id ASC) " + - "FILTER (WHERE s.id IS NOT NULL), json_array()) AS supplier_briefs", + "FILTER (WHERE s.id IS NOT NULL), json_array())) mo_je) AS supplier_briefs", ); }); diff --git a/server/typescript/packages/integration-tests/test/view-lifecycle-pg.test.ts b/server/typescript/packages/integration-tests/test/view-lifecycle-pg.test.ts index 89bd3ceeb..bec18dd73 100644 --- a/server/typescript/packages/integration-tests/test/view-lifecycle-pg.test.ts +++ b/server/typescript/packages/integration-tests/test/view-lifecycle-pg.test.ts @@ -432,6 +432,13 @@ describe("view lifecycle — real Postgres", () => { { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, { "identity.reference": { name: "ref_program", "@fields": "programId", "@references": "Program" } }, ] } }, + // #335 — the curated value object a WHOLE-OBJECT collect rolls Week rows up into. + // Week also declares programId / durationMinutes / createdAt; the brief omits them, + // and the emitted jsonb must contain ONLY the two declared members. + { "object.value": { name: "WeekBrief", children: [ + { "field.long": { name: "id" } }, + { "field.string": { name: "label" } }, + ] } }, { "object.projection": { name: "ProgramSummary", children: [ { "source.rdb": { "@kind": "view", "@table": "v_program_summary" } }, { "identity.primary": { name: "id", extends: "Program.id", "@fields": "id" } }, @@ -445,6 +452,10 @@ describe("view lifecycle — real Postgres", () => { // collectAgg — the week labels as a native array, ordered for determinism. { "field.string": { name: "weekLabels", isArray: true, children: [ { "origin.aggregate": { "@agg": "collect", "@of": "Week.label", "@via": "Program.weeks", "@orderBy": ["label:asc"] } } ] } }, + // #335 collectObjectAgg — the whole WeekBrief per related row, beside the scalar + // collect above, so both collect arms are exercised in ONE view on one engine. + { "field.object": { name: "weekBriefs", isArray: true, "@objectRef": "WeekBrief", children: [ + { "origin.aggregate": { "@agg": "collect", "@via": "Program.weeks" } } ] } }, // first — the most-recent week's label (null over an empty related set). { "field.string": { name: "latestWeekLabel", children: [ { "origin.first": { "@of": "Week.label", "@via": "Program.weeks", "@orderBy": ["createdAt:desc"] } } ] } }, @@ -459,6 +470,11 @@ describe("view lifecycle — real Postgres", () => { expect(up).toMatch(/COALESCE\(bool_or\([^)]*\) FILTER \(WHERE [^)]*\), FALSE\) AS "anyLongWeek"/); expect(up).toMatch(/COALESCE\(bool_and\([^)]*\) FILTER \(WHERE [^)]*\), TRUE\) AS "allLongWeeks"/); expect(up).toMatch(/COALESCE\(array_agg\([^)]*\) FILTER \(WHERE [^)]*\), '\{\}'\) AS "weekLabels"/); + // #335 — jsonb, not json: PG's `json` has no equality or ordering operator, so the + // json_agg(json_build_object(…) ORDER BY …) form does not even run. + expect(up).toContain( + `COALESCE(jsonb_agg(jsonb_build_object('id', w.id, 'label', w.label) ORDER BY w.id ASC) FILTER (WHERE w.id IS NOT NULL), '[]'::jsonb) AS "weekBriefs"`, + ); // THE gate: the view (aggregates + correlated-subquery + computed) round-trips. await assertConverged(expected); @@ -471,12 +487,13 @@ describe("view lifecycle — real Postgres", () => { ).execute(k); const rows = await sql.raw( - `SELECT "id","anyLongWeek","allLongWeeks","weekLabels","latestWeekLabel","isPublished" + `SELECT "id","anyLongWeek","allLongWeeks","weekLabels","weekBriefs","latestWeekLabel","isPublished" FROM "v_program_summary" ORDER BY "id"`, ).execute(k); type Row = { id: string; anyLongWeek: boolean; allLongWeeks: boolean; - weekLabels: string[]; latestWeekLabel: string | null; isPublished: boolean; + weekLabels: string[]; weekBriefs: { id: number; label: string }[]; + latestWeekLabel: string | null; isPublished: boolean; }; const byId = new Map((rows.rows as Row[]).map((r) => [String(r.id), r])); @@ -485,6 +502,22 @@ describe("view lifecycle — real Postgres", () => { expect(full.anyLongWeek).toBe(true); // 90 > 60 expect(full.allLongWeeks).toBe(false); // 30 is NOT > 60 expect(full.weekLabels).toEqual(["A", "B"]); // native PG array, ordered by label + // #335 — the node-postgres driver parses jsonb, so this is a real array of OBJECTS + // (not a string). Members are the DECLARED two, in related-PK order; if programId / + // durationMinutes / createdAt appear, "the declared VO IS the exposure" (#270) has + // broken at the SQL tier. + // + // NOTE the id types, which are deliberately asserted rather than normalised away: + // `Week.id` is a field.long -> BIGINT, and a TOP-LEVEL bigint column arrives as the + // STRING "1" (node-postgres stringifies bigint to avoid precision loss) — see the + // `id: string` row type above. The SAME value inside the rollup arrives as the NUMBER + // 1, because jsonb_build_object serialises it as a JSON number and JSON has no bigint. + // That asymmetry is inherent to a JSON-valued rollup, not a codegen bug, and it is + // lossy above 2^53. Pinned here so a future change to either tier has to face it. + expect(full.weekBriefs).toEqual([ + { id: 1, label: "A" }, + { id: 2, label: "B" }, + ]); expect(full.latestWeekLabel).toBe("B"); // most recent by createdAt expect(full.isPublished).toBe(true); // status = 'PUBLISHED' @@ -493,6 +526,7 @@ describe("view lifecycle — real Postgres", () => { expect(empty.anyLongWeek).toBe(false); // any over ∅ = false expect(empty.allLongWeeks).toBe(true); // all over ∅ = true (vacuous) expect(empty.weekLabels).toEqual([]); // collect over ∅ = [] (NOT null) + expect(empty.weekBriefs).toEqual([]); // whole-object collect over ∅ = [] too expect(empty.latestWeekLabel).toBeNull(); // first over ∅ = null expect(empty.isPublished).toBe(false); // status = 'DRAFT' }); diff --git a/server/typescript/packages/integration-tests/test/view-lifecycle-sqlite.test.ts b/server/typescript/packages/integration-tests/test/view-lifecycle-sqlite.test.ts index cec96ac13..fd7db1828 100644 --- a/server/typescript/packages/integration-tests/test/view-lifecycle-sqlite.test.ts +++ b/server/typescript/packages/integration-tests/test/view-lifecycle-sqlite.test.ts @@ -45,6 +45,13 @@ const META = JSON.stringify({ "metadata.root": { package: "acme", children: [ { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, { "identity.reference": { name: "ref_program", "@fields": "programId", "@references": "Program" } }, ] } }, + // #335 — the curated value object a WHOLE-OBJECT collect rolls Week rows up into. + // Week declares `programId`, `durationMinutes` and `createdAt` too; the brief omits + // them, and the emitted JSON must contain ONLY the two declared members. + { "object.value": { name: "WeekBrief", children: [ + { "field.long": { name: "id" } }, + { "field.string": { name: "label" } }, + ] } }, { "object.projection": { name: "ProgramSummary", children: [ { "source.rdb": { "@kind": "view", "@table": "v_program_summary" } }, { "identity.primary": { name: "id", extends: "Program.id", "@fields": "id" } }, @@ -56,6 +63,8 @@ const META = JSON.stringify({ "metadata.root": { package: "acme", children: [ { "origin.aggregate": { "@agg": "all", "@via": "Program.weeks", "@filter": { durationMinutes: { gt: 60 } } } } ] } }, { "field.string": { name: "weekLabels", isArray: true, children: [ { "origin.aggregate": { "@agg": "collect", "@of": "Week.label", "@via": "Program.weeks", "@orderBy": ["label:asc"] } } ] } }, + { "field.object": { name: "weekBriefs", isArray: true, "@objectRef": "WeekBrief", children: [ + { "origin.aggregate": { "@agg": "collect", "@via": "Program.weeks" } } ] } }, { "field.string": { name: "latestWeekLabel", children: [ { "origin.first": { "@of": "Week.label", "@via": "Program.weeks", "@orderBy": ["createdAt:desc"] } } ] } }, { "field.boolean": { name: "isPublished", children: [ @@ -103,6 +112,11 @@ describe("view value-probe — real SQLite (#195 origins)", () => { // SQLite lowers collect to json_group_array (a JSON string), any/all to MAX/MIN over 1/0. expect(up).toContain(`CREATE VIEW "v_program_summary" AS`); expect(up).toContain("json_group_array"); + // #335 whole-object rollup: json_object per row, related-PK ascending, [] on empty. + // #335 whole-object rollup: an ordered json_object array re-wrapped through json_each + // (SQLite's in-aggregate ORDER BY destroys the JSON subtype — see view-ddl-emit). + expect(up).toContain("json_group_array(json_object('id', w.id, 'label', w.label) ORDER BY w.id ASC)"); + expect(up).toContain("SELECT json_group_array(json(mo_je.value)) FROM json_each("); expect(up).toMatch(/COALESCE\(MAX\(CASE WHEN[^)]*\)[^,]*, 0\) AS "anyLongWeek"/); expect(up).toMatch(/COALESCE\(MIN\(CASE WHEN[^)]*\)[^,]*, 1\) AS "allLongWeeks"/); await applyRaw(up); @@ -124,12 +138,12 @@ describe("view value-probe — real SQLite (#195 origins)", () => { ).execute(k); const rows = await sql.raw( - `SELECT "id","anyLongWeek","allLongWeeks","weekLabels","latestWeekLabel","isPublished" + `SELECT "id","anyLongWeek","allLongWeeks","weekLabels","weekBriefs","latestWeekLabel","isPublished" FROM "v_program_summary" ORDER BY "id"`, ).execute(k); type Row = { id: number; anyLongWeek: number; allLongWeeks: number; - weekLabels: string; latestWeekLabel: string | null; isPublished: number; + weekLabels: string; weekBriefs: string; latestWeekLabel: string | null; isPublished: number; }; const byId = new Map((rows.rows as Row[]).map((r) => [String(r.id), r])); @@ -139,6 +153,13 @@ describe("view value-probe — real SQLite (#195 origins)", () => { expect(full.anyLongWeek).toBe(1); // 90 > 60 expect(full.allLongWeeks).toBe(0); // 30 is NOT > 60 expect(JSON.parse(full.weekLabels)).toEqual(["A", "B"]); + // #335 — the DECLARED members only, in related-PK order. Week also has programId / + // durationMinutes / createdAt; if any of those appear the "declared VO IS the + // exposure" guarantee (#270) has broken at the SQL tier. + expect(JSON.parse(full.weekBriefs)).toEqual([ + { id: 1, label: "A" }, + { id: 2, label: "B" }, + ]); expect(full.latestWeekLabel).toBe("B"); // most recent by createdAt expect(full.isPublished).toBe(1); // status = 'PUBLISHED' @@ -147,6 +168,7 @@ describe("view value-probe — real SQLite (#195 origins)", () => { expect(empty.anyLongWeek).toBe(0); // any over ∅ = false (0) expect(empty.allLongWeeks).toBe(1); // all over ∅ = true (1, vacuous) expect(JSON.parse(empty.weekLabels)).toEqual([]); // collect over ∅ = [] (NOT null) + expect(JSON.parse(empty.weekBriefs)).toEqual([]); // whole-object collect over ∅ = [] too expect(empty.latestWeekLabel).toBeNull(); // first over ∅ = null expect(empty.isPublished).toBe(0); // status = 'DRAFT' }); From 42633adae35f38a3923eedc86cce048a999a3599 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 23 Aug 2026 16:03:30 -0400 Subject: [PATCH 24/25] docs(migrations): a whole-object origin.collection on a PROJECTION is replaced, not deleted (#335) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The retirement guide's "delete the child and change nothing else" advice is correct for a payload or object.value host and SILENTLY WRONG for a view-kind projection: a projection field with no origin.* is treated as a plain base column, so the emitted DDL becomes `SELECT base."supplierBriefs"` against a column that does not exist. The metadata loads clean; the failure surfaces at apply. §2 now splits by host — 2a keeps the delete for payload/value hosts, 2b replaces the child with `origin.aggregate @agg: collect @via: …` for projections and names the three constraints the old subtype did not have (@via required, @distinct refused, members must match the @via terminal on both type axes), each of which is a load error, so a mistake there fails loudly. The section that stated the gap as a future promise is replaced by the migration itself, now that #335 has shipped. fixtures/conformance/README.md's "coverage genuinely lost" entry is closed with the reasoning, not just a status flip: the array-of-value-object-carrying-an-origin shape lives in the dedicated `collect-whole-object` fixture rather than back in `flattened-kitchen-sink`, because that fixture's concern is `@storage: flattened` and a whole-object rollup exercises none of it — folding them would make each harder to read in a five-language investigation. It also names where the shape is round-tripped against real engines. TS conformance 606/0. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq --- .../origin-collection-retirement.md | 55 +++++++++++++++---- fixtures/conformance/README.md | 16 +++--- 2 files changed, 52 insertions(+), 19 deletions(-) diff --git a/docs/features/migrations/origin-collection-retirement.md b/docs/features/migrations/origin-collection-retirement.md index 61e83c1db..41020d94d 100644 --- a/docs/features/migrations/origin-collection-retirement.md +++ b/docs/features/migrations/origin-collection-retirement.md @@ -59,10 +59,16 @@ gain `@filter`, `@orderBy` and `@distinct`, none of which `collection` could exp `collect` preserves the **element** type: the array field's own `field.` must equal the `@of` column's, and the field must be `isArray: true`. -### 2. A whole-object rollup → delete the child +### 2. A whole-object rollup → depends on the HOST -If the field collected whole nested value objects (`field.object @objectRef … isArray: true`), -**delete the `origin.collection` child and change nothing else**: +What to do splits by where the field lives, and getting it wrong on a projection is +**silent** — read both cases before editing. + +#### 2a. On a payload / `object.value` host → delete the child + +If the field collected whole nested value objects (`field.object @objectRef … isArray: true`) +and its host is a payload or value object, **delete the `origin.collection` child and change +nothing else**: ```jsonc { "field.object": { @@ -78,15 +84,40 @@ The declared shape is unchanged, and payload typing has been **declared-authorit 0.20.16** (#270) — the type came from `field.object` + `isArray` + `@objectRef`, never from the origin. Generated payload records, output parsers and render helpers are byte-identical. -**State it plainly, because it is the one real gap:** no surviving origin expresses a -whole-object rollup along a relationship — `@agg: collect` reduces a *column* via `@of`. If -that field was on a `source.rdb @kind: view` **projection** and you were relying on the view -DDL, note that no port ever emitted DDL for `origin.collection` either, so nothing regresses; -but you also cannot now declare the provenance. That shape returns with -[#335](https://github.com/metaobjectsdev/metaobjects/issues/335), which makes `@of` **optional** -on `@agg: collect` (absent = whole-object rollup, typed by the declared `@objectRef` + -`isArray`, never derived from the `@via` target) and ships the view lowering with it. It is -**additive**, so it needs no breaking window. +#### 2b. On a view-kind PROJECTION host → replace the child, never delete it + +**Deleting the child here produces metadata that loads clean and generates a broken view.** +A projection field with no `origin.*` is treated as a plain base column, so the emitted DDL +becomes `SELECT base."supplierBriefs"` against a column that does not exist. The load says +nothing; the failure surfaces at apply. + +Replace the child instead: + +```jsonc +{ "field.object": { + "name": "supplierBriefs", "isArray": true, + "@objectRef": "acme::catalog::SupplierBrief", + "children": [ +- { "origin.collection": { "@via": "acme::catalog::Product.suppliers" } } ++ { "origin.aggregate": { ++ "@agg": "collect", ++ "@via": "acme::catalog::Product.suppliers" } } + ] +}} +``` + +That is [#335](https://github.com/metaobjectsdev/metaobjects/issues/335)'s whole-object +rollup: `@of` is **optional** on `@agg: collect`, and omitting it means "collect the related +rows as the declared `@objectRef` value object". The element type comes from the declared +`@objectRef` + `isArray`, never from the `@via` target (#270), and the view lowering ships +with it — `jsonb_agg(jsonb_build_object(…))` on Postgres, `json_group_array(json_object(…))` +on SQLite, `[]` on an empty related set. + +Three constraints the old `origin.collection` did not have: `@via` is **required** (there is +no `@of` entity to infer a single hop from), `@distinct` is **refused** (it is a guaranteed +no-op whenever the value object carries the primary key), and every value-object member must +match a field on the `@via` **terminal** entity by name, with the same `field.` and +array-ness. Each is a load error, so a mistake here fails loudly rather than silently. ### 3. `ASSEMBLY_ORIGIN_SUBTYPES` shrinks to three diff --git a/fixtures/conformance/README.md b/fixtures/conformance/README.md index 5b658dc08..0eaecd03a 100644 --- a/fixtures/conformance/README.md +++ b/fixtures/conformance/README.md @@ -158,13 +158,15 @@ silent. So every removal records what it covered and where that coverage went. **Coverage genuinely lost, stated rather than faked:** `flattened-kitchen-sink` dropped its `supplierBriefs` field, the corpus's only array-of-value-object carrying an origin. No surviving origin expresses a whole-object rollup along a -relationship — `origin.aggregate @agg:collect` reduces a *column* via `@of`. That -shape becomes expressible again when [#335](https://github.com/metaobjectsdev/metaobjects/issues/335) -makes `@of` optional on `collect`; the fixture is the place to restore it. -**Restored** — see "Whole-object rollup (#335)" below: `collect-whole-object` is -a dedicated fixture, not a restoration of `flattened-kitchen-sink` itself (that -fixture stays scoped to its own concern), but it is the corpus's positive case -for exactly this shape. +relationship — `origin.aggregate @agg:collect` reduces a *column* via `@of`. **RESTORED by [#335](https://github.com/metaobjectsdev/metaobjects/issues/335)**, +which makes `@of` optional on `collect` — see "Whole-object rollup (#335)" below. +The coverage lives in a DEDICATED fixture, `collect-whole-object`, rather than +back in `flattened-kitchen-sink`: that fixture's concern is `@storage: flattened`, +and a whole-object rollup exercises none of it, so folding the two would make +each harder to read in a five-language investigation. `collect-whole-object` is +an array-of-value-object carrying an origin — the exact shape recorded as lost — +and it is round-tripped against real Postgres and SQLite in +`server/typescript/packages/integration-tests/test/view-lifecycle-{pg,sqlite}.test.ts`. ## Generated fixtures (differential testing) From d98c0b9e06d42f0d3e51b95485442f1350ec6960 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Sun, 23 Aug 2026 16:03:45 -0400 Subject: [PATCH 25/25] docs(changelog): the whole-object rollup, and teach it where authors will look (#335) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The CHANGELOG entry leads with the gap rather than the feature: a projection could roll related rows up into an array of one COLUMN and had no way to roll them into an array of OBJECTS, so the shape needed a second round-trip or a hand-written view — unmanaged, and therefore invisible to `meta verify --db`. It records the rulings that will otherwise be re-litigated: why the declared value object is the exposure and binds BY NAME rather than by `extends`; why @distinct is refused by choice rather than engine limit; why the new error code exists at all (the corpus compares code + source and never message text, so five negative fixtures passed against three ports containing none of the rules); why jsonb and not json; why SQLite needs the json_each re-wrap; and the bigint-inside-jsonb asymmetry an adopter will hit above 2^53. Half B's array filter/sort rules are recorded in the same entry. The authoring skill is updated in all SIX copies (the source plus its five byte-gated conformance mirrors). That matters for a specific reason: #342 shipped with the skill teaching a form the loader had just made ILLEGAL. This is the same failure in the other direction — a skill that cannot express the new form leaves adopters reaching for a hand-written view, which is exactly what the feature exists to remove. agent-context-conformance 5/0. `downstream-metadata-decisions.md` gains it as a Step 0 entry rather than a feature note, because "I need an array of objects" is the case most often mistaken for missing vocabulary: the @of form reads as the only form, and the natural next thought is a custom view. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq --- CHANGELOG.md | 80 +++++++++++++++++++ .../skills/metaobjects-authoring/SKILL.md | 12 ++- .../features/downstream-metadata-decisions.md | 7 ++ .../skills/metaobjects-authoring/SKILL.md | 12 ++- .../skills/metaobjects-authoring/SKILL.md | 12 ++- .../skills/metaobjects-authoring/SKILL.md | 12 ++- .../skills/metaobjects-authoring/SKILL.md | 12 ++- .../skills/metaobjects-authoring/SKILL.md | 12 ++- 8 files changed, 147 insertions(+), 12 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 56fb6cb86..755e18f8f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,86 @@ this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm ## [Unreleased] +### Added — `@of` becomes optional on `@agg: collect`: the whole-object rollup ([#335](https://github.com/metaobjectsdev/metaobjects/issues/335)) + +**A projection could roll related rows up into an array of one COLUMN, and had no way to +roll them up into an array of OBJECTS.** `origin.aggregate @agg: collect` required `@of`, so +"every supplier's `{id, name}` for this product" was inexpressible — the shape had to be a +second round-trip, or a hand-written view, which is unmanaged and invisible to +`meta verify --db`. When `origin.collection` retired in `0.24.0` this became the one +coverage gap the retirement guide had to state rather than close. + +**`@of` is now OPTIONAL on `collect`, and omitting it means a whole-object rollup:** the +carrying `field.object @isArray @objectRef` collects each related row as its declared value +object. + +```jsonc +{ "field.object": { + "name": "supplierBriefs", "isArray": true, "@objectRef": "SupplierBrief", + "children": [ + { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers" } } + ] +}} +``` + +**The declared value object IS the exposure.** Members bind to the `@via` terminal entity's +fields BY NAME, and a field the entity has but the value object omits is simply not +projected. That is deliberate — it is the [#270](https://github.com/metaobjectsdev/metaobjects/issues/270) +guarantee (a curated value object must not silently become the full entity) carried down to +the DDL tier. Name matching, rather than `extends`, is also deliberate: it keeps one value +object collectable from two different entities, which `extends` would forbid. The convention +is written into the byte-gated `origin.aggregate` registry prose so no port has to infer it. + +**Eight load errors, in all five ports.** The carrying field must be a `field.object` +declaring `@objectRef`; that `@objectRef` must name an `object.value`; `@via` is required +(there is no `@of` entity to infer a single hop from); the path must be to-many; `@distinct` +is refused; `@orderBy` keys must resolve against the `@via` **terminal** entity; every value +object member must match a terminal field; and a matched member must agree on **both** type +axes — subtype and array-ness. `@distinct` is refused by CHOICE, not engine limit: it works +on both engines, but it is a guaranteed no-op whenever the value object carries the primary +key, and a silent no-op is worse than a refusal. + +**A new error code, `ERR_COLLECT_WHOLE_OBJECT`,** carries the five refusals that would +otherwise have shared `ERR_INVALID_ORIGIN`. That is not taxonomy for its own sake: the shared +corpus compares error **code + source** and never message text, and `ERR_INVALID_ORIGIN` is +exactly what a loader that still *requires* `@of` emits for this same metadata — so five of +the eight negative fixtures passed against three ports containing none of the rules. With the +distinct code they fail, and Task-by-task porting has a real signal. + +**View lowering, both dialects.** Postgres emits +`COALESCE(jsonb_agg(jsonb_build_object(…) ORDER BY ASC) FILTER (WHERE IS NOT NULL), '[]'::jsonb)`. +`jsonb`, not `json`: PG's `json` type has neither an equality nor an ordering operator, so +the `json_agg(json_build_object(… ORDER BY …))` form does not run at all. Default element +order is the related entity's **primary key** ascending — ordering rows by a serialized +object is meaningless — and an explicit `@orderBy` leads with the PK appended as a tie-break. +The scalar `@of` arm deliberately keeps its existing no-tie-break behaviour, since changing +it would move the emitted SQL of every project already using `@orderBy`. + +**SQLite needed a shape nobody would have guessed, and only a real engine found it.** On +SQLite 3.44 (D1's pinned baseline) the in-aggregate `ORDER BY` clause **destroys the JSON +subtype**: `json_group_array(json_object(…) ORDER BY …)` returns an array of quoted STRINGS +rather than objects, and wrapping the argument in `json()` does not survive it either. +Dropping the `ORDER BY` was not an option — element order would stop being deterministic and +an author's `@orderBy` would silently do nothing. So the ordered array is built first and +re-wrapped element-by-element through `json_each`, which iterates in array order. The +emitted SQL text alone could never have shown this; it was caught by the emit → apply → +introspect → re-diff round-trip against a real engine, which is the standing rule that +golden SQL is not evidence for new DDL. + +Also worth knowing: inside the rollup a `field.long` member arrives as a JSON **number**, +while the same value as a top-level `BIGINT` column arrives as a string from +node-postgres. That is inherent to JSON, not a codegen choice, and it is lossy above 2^53. + +**Also in this change — array fields are not filterable or sortable.** A +`field. isArray: true` carrying `@filterable: true` or `@sortable: true` is now a +load error in all five ports (`ERR_FILTERABLE_UNSUPPORTED_SUBTYPE` / +`ERR_SORTABLE_UNSUPPORTED_SUBTYPE`): no operator in the FR-009 scalar band applies to a +collection column, and no dialect can `ORDER BY` one. `@sortable` also gains the subtype +validation `@filterable` already had. + +`metamodelVersion` moves `0.11` → `0.12`. + + ## [0.24.1] — npm `0.24.1` · PyPI `0.24.1` · NuGet `0.24.1` · Maven `7.24.1` ### Fixed — an expression index was undeclarable, and the one spelling that loaded was half-ignored ([#342](https://github.com/metaobjectsdev/metaobjects/issues/342)) diff --git a/agent-context/skills/metaobjects-authoring/SKILL.md b/agent-context/skills/metaobjects-authoring/SKILL.md index 98b16b4c5..1dbb1fe1d 100644 --- a/agent-context/skills/metaobjects-authoring/SKILL.md +++ b/agent-context/skills/metaobjects-authoring/SKILL.md @@ -838,8 +838,16 @@ amendment 2026-08-06.) **Origin vocabulary (#195).** `origin.aggregate @agg` takes `count`/`sum`/`avg`/`min`/`max` (numeric reduces over `@of`), `any`/`all` (predicate quantifiers over a `@filter`; `@of` -forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup of `@of` -into an `isArray` field, with optional `@distinct` / `@orderBy`). Any aggregate may be +forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup +into an `isArray` field, with optional `@distinct` / `@orderBy`). **`collect` is the one +`@agg` where `@of` is OPTIONAL (#335):** name a column with `@of` to collect scalars, or +omit `@of` on a `field.object @objectRef` to collect each related row as that declared +value object — a **whole-object rollup**, lowered to `jsonb_agg(jsonb_build_object(…))` +on Postgres. The whole-object form requires an explicit `@via`, refuses `@distinct` (it is +a no-op whenever the value object carries the primary key), and requires every value-object +member to match a field on the `@via` **terminal** entity by name, with the same subtype +and array-ness. The declared value object IS the exposure: a field the entity has and the +value object omits is not projected. Any aggregate may be row-scoped with `@filter`. `origin.computed` carries a closed structured `@expr` tree (a derived scalar). `origin.first` picks one related row's column (`@of`) along `@via`, ordered by a **required `@orderBy`** (`["field:asc|desc", …]`, with the PK as tie-break) — diff --git a/docs/features/downstream-metadata-decisions.md b/docs/features/downstream-metadata-decisions.md index 54118af0b..a9be37d3b 100644 --- a/docs/features/downstream-metadata-decisions.md +++ b/docs/features/downstream-metadata-decisions.md @@ -29,6 +29,13 @@ Most "I need a new type" needs die here. Before you register anything: - **Is it a one-off author-supplied property?** The registered `attr.properties` bag is the sanctioned escape hatch for arbitrary key/values — it does not require a new attribute and stays inside strict provenance (ADR-0023). +- **Do you need an array of OBJECTS on a projection?** That is + `origin.aggregate @agg: collect` with **`@of` omitted** — the whole-object rollup + (#335), on a `field.object @isArray @objectRef`. It is the case most often + mistaken for missing vocabulary, because the `@of` form reads as the only form and + the natural next thought is a hand-written view. Its declared value object is the + exposure, and the `CREATE VIEW` is generated — a hand-written one is unmanaged and + therefore invisible to `meta verify --db`. ## Step 1 — check whether core (or the roadmap) already models it diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md index 98b16b4c5..1dbb1fe1d 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -838,8 +838,16 @@ amendment 2026-08-06.) **Origin vocabulary (#195).** `origin.aggregate @agg` takes `count`/`sum`/`avg`/`min`/`max` (numeric reduces over `@of`), `any`/`all` (predicate quantifiers over a `@filter`; `@of` -forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup of `@of` -into an `isArray` field, with optional `@distinct` / `@orderBy`). Any aggregate may be +forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup +into an `isArray` field, with optional `@distinct` / `@orderBy`). **`collect` is the one +`@agg` where `@of` is OPTIONAL (#335):** name a column with `@of` to collect scalars, or +omit `@of` on a `field.object @objectRef` to collect each related row as that declared +value object — a **whole-object rollup**, lowered to `jsonb_agg(jsonb_build_object(…))` +on Postgres. The whole-object form requires an explicit `@via`, refuses `@distinct` (it is +a no-op whenever the value object carries the primary key), and requires every value-object +member to match a field on the `@via` **terminal** entity by name, with the same subtype +and array-ness. The declared value object IS the exposure: a field the entity has and the +value object omits is not projected. Any aggregate may be row-scoped with `@filter`. `origin.computed` carries a closed structured `@expr` tree (a derived scalar). `origin.first` picks one related row's column (`@of`) along `@via`, ordered by a **required `@orderBy`** (`["field:asc|desc", …]`, with the PK as tie-break) — diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md index 98b16b4c5..1dbb1fe1d 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -838,8 +838,16 @@ amendment 2026-08-06.) **Origin vocabulary (#195).** `origin.aggregate @agg` takes `count`/`sum`/`avg`/`min`/`max` (numeric reduces over `@of`), `any`/`all` (predicate quantifiers over a `@filter`; `@of` -forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup of `@of` -into an `isArray` field, with optional `@distinct` / `@orderBy`). Any aggregate may be +forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup +into an `isArray` field, with optional `@distinct` / `@orderBy`). **`collect` is the one +`@agg` where `@of` is OPTIONAL (#335):** name a column with `@of` to collect scalars, or +omit `@of` on a `field.object @objectRef` to collect each related row as that declared +value object — a **whole-object rollup**, lowered to `jsonb_agg(jsonb_build_object(…))` +on Postgres. The whole-object form requires an explicit `@via`, refuses `@distinct` (it is +a no-op whenever the value object carries the primary key), and requires every value-object +member to match a field on the `@via` **terminal** entity by name, with the same subtype +and array-ness. The declared value object IS the exposure: a field the entity has and the +value object omits is not projected. Any aggregate may be row-scoped with `@filter`. `origin.computed` carries a closed structured `@expr` tree (a derived scalar). `origin.first` picks one related row's column (`@of`) along `@via`, ordered by a **required `@orderBy`** (`["field:asc|desc", …]`, with the PK as tie-break) — diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md index 98b16b4c5..1dbb1fe1d 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -838,8 +838,16 @@ amendment 2026-08-06.) **Origin vocabulary (#195).** `origin.aggregate @agg` takes `count`/`sum`/`avg`/`min`/`max` (numeric reduces over `@of`), `any`/`all` (predicate quantifiers over a `@filter`; `@of` -forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup of `@of` -into an `isArray` field, with optional `@distinct` / `@orderBy`). Any aggregate may be +forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup +into an `isArray` field, with optional `@distinct` / `@orderBy`). **`collect` is the one +`@agg` where `@of` is OPTIONAL (#335):** name a column with `@of` to collect scalars, or +omit `@of` on a `field.object @objectRef` to collect each related row as that declared +value object — a **whole-object rollup**, lowered to `jsonb_agg(jsonb_build_object(…))` +on Postgres. The whole-object form requires an explicit `@via`, refuses `@distinct` (it is +a no-op whenever the value object carries the primary key), and requires every value-object +member to match a field on the `@via` **terminal** entity by name, with the same subtype +and array-ness. The declared value object IS the exposure: a field the entity has and the +value object omits is not projected. Any aggregate may be row-scoped with `@filter`. `origin.computed` carries a closed structured `@expr` tree (a derived scalar). `origin.first` picks one related row's column (`@of`) along `@via`, ordered by a **required `@orderBy`** (`["field:asc|desc", …]`, with the PK as tie-break) — diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md index 98b16b4c5..1dbb1fe1d 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -838,8 +838,16 @@ amendment 2026-08-06.) **Origin vocabulary (#195).** `origin.aggregate @agg` takes `count`/`sum`/`avg`/`min`/`max` (numeric reduces over `@of`), `any`/`all` (predicate quantifiers over a `@filter`; `@of` -forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup of `@of` -into an `isArray` field, with optional `@distinct` / `@orderBy`). Any aggregate may be +forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup +into an `isArray` field, with optional `@distinct` / `@orderBy`). **`collect` is the one +`@agg` where `@of` is OPTIONAL (#335):** name a column with `@of` to collect scalars, or +omit `@of` on a `field.object @objectRef` to collect each related row as that declared +value object — a **whole-object rollup**, lowered to `jsonb_agg(jsonb_build_object(…))` +on Postgres. The whole-object form requires an explicit `@via`, refuses `@distinct` (it is +a no-op whenever the value object carries the primary key), and requires every value-object +member to match a field on the `@via` **terminal** entity by name, with the same subtype +and array-ness. The declared value object IS the exposure: a field the entity has and the +value object omits is not projected. Any aggregate may be row-scoped with `@filter`. `origin.computed` carries a closed structured `@expr` tree (a derived scalar). `origin.first` picks one related row's column (`@of`) along `@via`, ordered by a **required `@orderBy`** (`["field:asc|desc", …]`, with the PK as tie-break) — diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md index 98b16b4c5..1dbb1fe1d 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -838,8 +838,16 @@ amendment 2026-08-06.) **Origin vocabulary (#195).** `origin.aggregate @agg` takes `count`/`sum`/`avg`/`min`/`max` (numeric reduces over `@of`), `any`/`all` (predicate quantifiers over a `@filter`; `@of` -forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup of `@of` -into an `isArray` field, with optional `@distinct` / `@orderBy`). Any aggregate may be +forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup +into an `isArray` field, with optional `@distinct` / `@orderBy`). **`collect` is the one +`@agg` where `@of` is OPTIONAL (#335):** name a column with `@of` to collect scalars, or +omit `@of` on a `field.object @objectRef` to collect each related row as that declared +value object — a **whole-object rollup**, lowered to `jsonb_agg(jsonb_build_object(…))` +on Postgres. The whole-object form requires an explicit `@via`, refuses `@distinct` (it is +a no-op whenever the value object carries the primary key), and requires every value-object +member to match a field on the `@via` **terminal** entity by name, with the same subtype +and array-ness. The declared value object IS the exposure: a field the entity has and the +value object omits is not projected. Any aggregate may be row-scoped with `@filter`. `origin.computed` carries a closed structured `@expr` tree (a derived scalar). `origin.first` picks one related row's column (`@of`) along `@via`, ordered by a **required `@orderBy`** (`["field:asc|desc", …]`, with the PK as tie-break) —