diff --git a/CHANGELOG.md b/CHANGELOG.md index 56fb6cb86..755e18f8f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,86 @@ this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm ## [Unreleased] +### Added — `@of` becomes optional on `@agg: collect`: the whole-object rollup ([#335](https://github.com/metaobjectsdev/metaobjects/issues/335)) + +**A projection could roll related rows up into an array of one COLUMN, and had no way to +roll them up into an array of OBJECTS.** `origin.aggregate @agg: collect` required `@of`, so +"every supplier's `{id, name}` for this product" was inexpressible — the shape had to be a +second round-trip, or a hand-written view, which is unmanaged and invisible to +`meta verify --db`. When `origin.collection` retired in `0.24.0` this became the one +coverage gap the retirement guide had to state rather than close. + +**`@of` is now OPTIONAL on `collect`, and omitting it means a whole-object rollup:** the +carrying `field.object @isArray @objectRef` collects each related row as its declared value +object. + +```jsonc +{ "field.object": { + "name": "supplierBriefs", "isArray": true, "@objectRef": "SupplierBrief", + "children": [ + { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers" } } + ] +}} +``` + +**The declared value object IS the exposure.** Members bind to the `@via` terminal entity's +fields BY NAME, and a field the entity has but the value object omits is simply not +projected. That is deliberate — it is the [#270](https://github.com/metaobjectsdev/metaobjects/issues/270) +guarantee (a curated value object must not silently become the full entity) carried down to +the DDL tier. Name matching, rather than `extends`, is also deliberate: it keeps one value +object collectable from two different entities, which `extends` would forbid. The convention +is written into the byte-gated `origin.aggregate` registry prose so no port has to infer it. + +**Eight load errors, in all five ports.** The carrying field must be a `field.object` +declaring `@objectRef`; that `@objectRef` must name an `object.value`; `@via` is required +(there is no `@of` entity to infer a single hop from); the path must be to-many; `@distinct` +is refused; `@orderBy` keys must resolve against the `@via` **terminal** entity; every value +object member must match a terminal field; and a matched member must agree on **both** type +axes — subtype and array-ness. `@distinct` is refused by CHOICE, not engine limit: it works +on both engines, but it is a guaranteed no-op whenever the value object carries the primary +key, and a silent no-op is worse than a refusal. + +**A new error code, `ERR_COLLECT_WHOLE_OBJECT`,** carries the five refusals that would +otherwise have shared `ERR_INVALID_ORIGIN`. That is not taxonomy for its own sake: the shared +corpus compares error **code + source** and never message text, and `ERR_INVALID_ORIGIN` is +exactly what a loader that still *requires* `@of` emits for this same metadata — so five of +the eight negative fixtures passed against three ports containing none of the rules. With the +distinct code they fail, and Task-by-task porting has a real signal. + +**View lowering, both dialects.** Postgres emits +`COALESCE(jsonb_agg(jsonb_build_object(…) ORDER BY ASC) FILTER (WHERE IS NOT NULL), '[]'::jsonb)`. +`jsonb`, not `json`: PG's `json` type has neither an equality nor an ordering operator, so +the `json_agg(json_build_object(… ORDER BY …))` form does not run at all. Default element +order is the related entity's **primary key** ascending — ordering rows by a serialized +object is meaningless — and an explicit `@orderBy` leads with the PK appended as a tie-break. +The scalar `@of` arm deliberately keeps its existing no-tie-break behaviour, since changing +it would move the emitted SQL of every project already using `@orderBy`. + +**SQLite needed a shape nobody would have guessed, and only a real engine found it.** On +SQLite 3.44 (D1's pinned baseline) the in-aggregate `ORDER BY` clause **destroys the JSON +subtype**: `json_group_array(json_object(…) ORDER BY …)` returns an array of quoted STRINGS +rather than objects, and wrapping the argument in `json()` does not survive it either. +Dropping the `ORDER BY` was not an option — element order would stop being deterministic and +an author's `@orderBy` would silently do nothing. So the ordered array is built first and +re-wrapped element-by-element through `json_each`, which iterates in array order. The +emitted SQL text alone could never have shown this; it was caught by the emit → apply → +introspect → re-diff round-trip against a real engine, which is the standing rule that +golden SQL is not evidence for new DDL. + +Also worth knowing: inside the rollup a `field.long` member arrives as a JSON **number**, +while the same value as a top-level `BIGINT` column arrives as a string from +node-postgres. That is inherent to JSON, not a codegen choice, and it is lossy above 2^53. + +**Also in this change — array fields are not filterable or sortable.** A +`field. isArray: true` carrying `@filterable: true` or `@sortable: true` is now a +load error in all five ports (`ERR_FILTERABLE_UNSUPPORTED_SUBTYPE` / +`ERR_SORTABLE_UNSUPPORTED_SUBTYPE`): no operator in the FR-009 scalar band applies to a +collection column, and no dialect can `ORDER BY` one. `@sortable` also gains the subtype +validation `@filterable` already had. + +`metamodelVersion` moves `0.11` → `0.12`. + + ## [0.24.1] — npm `0.24.1` · PyPI `0.24.1` · NuGet `0.24.1` · Maven `7.24.1` ### Fixed — an expression index was undeclarable, and the one spelling that loaded was half-ignored ([#342](https://github.com/metaobjectsdev/metaobjects/issues/342)) diff --git a/agent-context/skills/metaobjects-authoring/SKILL.md b/agent-context/skills/metaobjects-authoring/SKILL.md index 98b16b4c5..1dbb1fe1d 100644 --- a/agent-context/skills/metaobjects-authoring/SKILL.md +++ b/agent-context/skills/metaobjects-authoring/SKILL.md @@ -838,8 +838,16 @@ amendment 2026-08-06.) **Origin vocabulary (#195).** `origin.aggregate @agg` takes `count`/`sum`/`avg`/`min`/`max` (numeric reduces over `@of`), `any`/`all` (predicate quantifiers over a `@filter`; `@of` -forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup of `@of` -into an `isArray` field, with optional `@distinct` / `@orderBy`). Any aggregate may be +forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup +into an `isArray` field, with optional `@distinct` / `@orderBy`). **`collect` is the one +`@agg` where `@of` is OPTIONAL (#335):** name a column with `@of` to collect scalars, or +omit `@of` on a `field.object @objectRef` to collect each related row as that declared +value object — a **whole-object rollup**, lowered to `jsonb_agg(jsonb_build_object(…))` +on Postgres. The whole-object form requires an explicit `@via`, refuses `@distinct` (it is +a no-op whenever the value object carries the primary key), and requires every value-object +member to match a field on the `@via` **terminal** entity by name, with the same subtype +and array-ness. The declared value object IS the exposure: a field the entity has and the +value object omits is not projected. Any aggregate may be row-scoped with `@filter`. `origin.computed` carries a closed structured `@expr` tree (a derived scalar). `origin.first` picks one related row's column (`@of`) along `@via`, ordered by a **required `@orderBy`** (`["field:asc|desc", …]`, with the PK as tie-break) — diff --git a/docs/features/downstream-metadata-decisions.md b/docs/features/downstream-metadata-decisions.md index 54118af0b..a9be37d3b 100644 --- a/docs/features/downstream-metadata-decisions.md +++ b/docs/features/downstream-metadata-decisions.md @@ -29,6 +29,13 @@ Most "I need a new type" needs die here. Before you register anything: - **Is it a one-off author-supplied property?** The registered `attr.properties` bag is the sanctioned escape hatch for arbitrary key/values — it does not require a new attribute and stays inside strict provenance (ADR-0023). +- **Do you need an array of OBJECTS on a projection?** That is + `origin.aggregate @agg: collect` with **`@of` omitted** — the whole-object rollup + (#335), on a `field.object @isArray @objectRef`. It is the case most often + mistaken for missing vocabulary, because the `@of` form reads as the only form and + the natural next thought is a hand-written view. Its declared value object is the + exposure, and the `CREATE VIEW` is generated — a hand-written one is unmanaged and + therefore invisible to `meta verify --db`. ## Step 1 — check whether core (or the roadmap) already models it diff --git a/docs/features/migrations/origin-collection-retirement.md b/docs/features/migrations/origin-collection-retirement.md index 61e83c1db..41020d94d 100644 --- a/docs/features/migrations/origin-collection-retirement.md +++ b/docs/features/migrations/origin-collection-retirement.md @@ -59,10 +59,16 @@ gain `@filter`, `@orderBy` and `@distinct`, none of which `collection` could exp `collect` preserves the **element** type: the array field's own `field.` must equal the `@of` column's, and the field must be `isArray: true`. -### 2. A whole-object rollup → delete the child +### 2. A whole-object rollup → depends on the HOST -If the field collected whole nested value objects (`field.object @objectRef … isArray: true`), -**delete the `origin.collection` child and change nothing else**: +What to do splits by where the field lives, and getting it wrong on a projection is +**silent** — read both cases before editing. + +#### 2a. On a payload / `object.value` host → delete the child + +If the field collected whole nested value objects (`field.object @objectRef … isArray: true`) +and its host is a payload or value object, **delete the `origin.collection` child and change +nothing else**: ```jsonc { "field.object": { @@ -78,15 +84,40 @@ The declared shape is unchanged, and payload typing has been **declared-authorit 0.20.16** (#270) — the type came from `field.object` + `isArray` + `@objectRef`, never from the origin. Generated payload records, output parsers and render helpers are byte-identical. -**State it plainly, because it is the one real gap:** no surviving origin expresses a -whole-object rollup along a relationship — `@agg: collect` reduces a *column* via `@of`. If -that field was on a `source.rdb @kind: view` **projection** and you were relying on the view -DDL, note that no port ever emitted DDL for `origin.collection` either, so nothing regresses; -but you also cannot now declare the provenance. That shape returns with -[#335](https://github.com/metaobjectsdev/metaobjects/issues/335), which makes `@of` **optional** -on `@agg: collect` (absent = whole-object rollup, typed by the declared `@objectRef` + -`isArray`, never derived from the `@via` target) and ships the view lowering with it. It is -**additive**, so it needs no breaking window. +#### 2b. On a view-kind PROJECTION host → replace the child, never delete it + +**Deleting the child here produces metadata that loads clean and generates a broken view.** +A projection field with no `origin.*` is treated as a plain base column, so the emitted DDL +becomes `SELECT base."supplierBriefs"` against a column that does not exist. The load says +nothing; the failure surfaces at apply. + +Replace the child instead: + +```jsonc +{ "field.object": { + "name": "supplierBriefs", "isArray": true, + "@objectRef": "acme::catalog::SupplierBrief", + "children": [ +- { "origin.collection": { "@via": "acme::catalog::Product.suppliers" } } ++ { "origin.aggregate": { ++ "@agg": "collect", ++ "@via": "acme::catalog::Product.suppliers" } } + ] +}} +``` + +That is [#335](https://github.com/metaobjectsdev/metaobjects/issues/335)'s whole-object +rollup: `@of` is **optional** on `@agg: collect`, and omitting it means "collect the related +rows as the declared `@objectRef` value object". The element type comes from the declared +`@objectRef` + `isArray`, never from the `@via` target (#270), and the view lowering ships +with it — `jsonb_agg(jsonb_build_object(…))` on Postgres, `json_group_array(json_object(…))` +on SQLite, `[]` on an empty related set. + +Three constraints the old `origin.collection` did not have: `@via` is **required** (there is +no `@of` entity to infer a single hop from), `@distinct` is **refused** (it is a guaranteed +no-op whenever the value object carries the primary key), and every value-object member must +match a field on the `@via` **terminal** entity by name, with the same `field.` and +array-ness. Each is a load error, so a mistake here fails loudly rather than silently. ### 3. `ASSEMBLY_ORIGIN_SUBTYPES` shrinks to three diff --git a/docs/superpowers/plans/2026-08-22-issue-335-whole-object-rollup.md b/docs/superpowers/plans/2026-08-22-issue-335-whole-object-rollup.md new file mode 100644 index 000000000..fcc861a73 --- /dev/null +++ b/docs/superpowers/plans/2026-08-22-issue-335-whole-object-rollup.md @@ -0,0 +1,1541 @@ +# Whole-Object Rollup (#335) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make `@of` optional on `origin.aggregate @agg: collect`, so a projection can roll up related rows as a JSON array of a declared value-object — and close the two holes in the queryable-projection contract that such a column would otherwise land in. + +**Architecture:** Two halves. **Half B first** (two loader rules making the filter/sort contract honest about array columns) so Half A lands into a correct tier. **Half A** then splits `collect` out of the `@of`-required gate in all four loaders, adds a value-object member→column resolution rule, and emits a new `collectObjectAgg` view column lowered to `jsonb_agg(jsonb_build_object(…))` on Postgres and `json_group_array(json_object(…))` on SQLite. Codegen is TypeScript-only (ADR-0015); the loader rules are cross-port. + +**Tech Stack:** TypeScript (Bun), C#, Java, Python. Shared JSON conformance corpus. Postgres + SQLite via Testcontainers. + +**Spec:** [`docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md`](../specs/2026-08-22-issue-335-whole-object-rollup-design.md) + +## Global Constraints + +- **`metamodelVersion` is NOT edited by this work.** It is already `0.11` (moved by #342) and that bump is unreleased. **Re-check before merging:** if `0.24.1` has shipped by then, `0.11` is released and this needs `node scripts/check-metamodel-version.mjs --set 0.12`. +- **ADR-0039 — never use own-only accessors.** Read `isArray` as `field.resolvedIsArray()`, attrs as `field.attr(...)`, children as `.children()`. `field.isArray` and `ownAttr`/`ownChildren` silently drop everything inherited via `extends`. The one exception in this plan is reading an `origin.*` node's own attrs (`origin.ownAttr(...)`), because `origin.*` never inherits (ADR-0029) — the existing code does this and carries that comment. +- **ADR-0023 — never invent an attribute.** This work adds **no** new registered attribute. `@of` is already `"required": false` structurally. +- **Named constants only.** Never inline `"collect"`, `"field"`, `"object"` etc. Import from `packages/metadata/src/constants.ts` (TS) and each port's equivalent. +- **Public repository.** No private/other-project names, no absolute home paths, in code, docs, fixtures **or commit messages**. +- **Every new load error needs a shared corpus fixture that triggers it.** A port-local unit test does not substitute — it proves one port enforces the rule, which is the true-but-insufficient assertion behind three separate #342 findings. +- **Scalar-arm output must stay byte-identical.** A `collect` **with** `@of` emits exactly the SQL it emits today. This is pinned by an explicit no-churn test, not assumed. +- **Test scoping.** Never run a bare `bun test` at the repo root. Scope to the package. + +## File Structure + +**Half B — loader only (4 ports):** +- `server/typescript/packages/metadata/src/loader/validation-passes.ts` — extend `validateFilterableHasSupportedOps` +- `server/csharp/MetaObjects/Loader/ValidationPasses.cs` — same rule +- `server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java` — same rule +- `server/python/src/metaobjects/loader/validation_passes.py` — same rule +- `fixtures/conformance/error-filterable-array-field/` — new negative fixture +- `fixtures/conformance/error-sortable-array-field/` — new negative fixture + +**Half A — loader (4 ports):** same four files, the `origin.aggregate` block. + +**Half A — registry prose (7 byte-gated files):** +- `spec/metamodel/origin.json` +- `fixtures/registry-conformance/expected-registry.json` +- `fixtures/metamodel-docs/expected/types/origin.md` +- `server/csharp/MetaObjects/Persistence/Origin/OriginSchema.cs` +- `server/csharp/MetaObjects/SpecMetamodel/origin.json` +- `server/python/src/metaobjects/spec_metamodel/origin.json` +- `server/typescript/packages/metadata/src/persistence/origin/origin-definition.embedded.ts` + +**Half A — codegen (TypeScript only):** +- `server/typescript/packages/codegen-ts/src/projection/view-spec.ts` — new `collectObjectAgg` kind +- `server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts` — object arm +- `server/typescript/packages/codegen-ts/src/projection/view-ddl-emit.ts` — both dialects + +**Half A — gates:** +- `fixtures/conformance/collect-whole-object/` + 6 negative fixtures +- `fixtures/conformance/flattened-kitchen-sink/` — restore `supplierBriefs` +- `server/typescript/packages/integration-tests/test/view-lifecycle-{pg,sqlite}.test.ts` — round-trip + +--- + +## Task 1: Half B — array fields cannot be `@filterable` (TypeScript) + +**Files:** +- Modify: `server/typescript/packages/metadata/src/loader/validation-passes.ts` (function `validateFilterableHasSupportedOps`, currently at ~`:436-456`) +- Test: `server/typescript/packages/metadata/test/validation-filterable-array.test.ts` (create) + +**Interfaces:** +- Consumes: nothing from earlier tasks. +- Produces: the extended `validateFilterableHasSupportedOps(root: MetaData): ParseError[]`, emitting `ERR_FILTERABLE_UNSUPPORTED_SUBTYPE` for array fields. Task 2 ports this rule; Task 3 gates it cross-port. + +**Why:** `filterSubTypeFor` in `codegen-ts/src/templates/filter-allowlist.ts` falls through to `"string"` for anything unrecognised and **nothing in that file consults `isArray`**, so `field.string isArray: true @filterable: true` emits a `like`/`eq` rule against a `text[]` column — SQL that cannot execute. No operator in the FR-009 band applies to an array, which is the same reason `field.object` is already rejected, so it reuses the same error code and the same door. + +- [ ] **Step 1: Write the failing test** + +Create `server/typescript/packages/metadata/test/validation-filterable-array.test.ts`: + +```ts +import { test, expect, describe } from "bun:test"; +import { MetaDataLoader } from "../src/loader/meta-data-loader.js"; + +const model = (fieldJson: string) => `{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { + "name": "Product", + "children": [ + { "source.rdb": { "@kind": "table", "@table": "products" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "@fields": ["id"] } }, + ${fieldJson} + ] + }} + ] + } +}`; + +describe("@filterable on an array field", () => { + test("an array field marked @filterable fails to load", () => { + const src = model( + `{ "field.string": { "name": "tags", "isArray": true, "@filterable": true } }`, + ); + const loader = new MetaDataLoader(); + expect(() => loader.loadFromString(src, "meta.demo.json")).toThrow( + /ERR_FILTERABLE_UNSUPPORTED_SUBTYPE/, + ); + }); + + test("the same field WITHOUT isArray still loads", () => { + const src = model( + `{ "field.string": { "name": "tags", "@filterable": true } }`, + ); + const loader = new MetaDataLoader(); + expect(() => loader.loadFromString(src, "meta.demo.json")).not.toThrow(); + }); + + test("an array field NOT marked @filterable still loads", () => { + const src = model( + `{ "field.string": { "name": "tags", "isArray": true } }`, + ); + const loader = new MetaDataLoader(); + expect(() => loader.loadFromString(src, "meta.demo.json")).not.toThrow(); + }); +}); +``` + +> If `loadFromString` is not the loader's string entry point in this codebase, find the one the neighbouring tests in `server/typescript/packages/metadata/test/` use and match it exactly — do not invent an API. + +- [ ] **Step 2: Run the test to verify it fails** + +```bash +cd server/typescript/packages/metadata && bun test test/validation-filterable-array.test.ts +``` + +Expected: the first test FAILS (no error is thrown); the other two PASS. + +- [ ] **Step 3: Extend the validation pass** + +In `validation-passes.ts`, inside `validateFilterableHasSupportedOps`, replace the single-condition body of the field loop with an array check ahead of the subtype check: + +```ts + // ADR-0039: resolving — a concrete field may inherit @filterable via extends. + if (field.attr(FIELD_ATTR_FILTERABLE) !== true) continue; + + // #335 Half B — an ARRAY field has no operator band either. Every FR-009 + // operator (eq/ne/gt/gte/lt/lte/in/like/isNull) is a scalar comparison; + // none applies to a collection column. The allowlist template does not + // consult isArray and falls through to the "string" band, so this + // previously emitted a `like` rule against a text[] column — SQL that + // cannot execute. Same reason as the subtype check below, so same code. + // ADR-0039: resolvedIsArray(), never the own `isArray` flag. + if (field.resolvedIsArray()) { + errors.push( + new ParseError( + `Field "${obj.name}.${field.name}" has @filterable: true but is an array ` + + `(isArray: true). No filter operator applies to a collection column. ` + + `Remove @filterable from this field.`, + { code: "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE", source: field.source }, + ), + ); + continue; + } + + if (opsForSubType(field.subType).length > 0) continue; +``` + +- [ ] **Step 4: Run the test to verify it passes** + +```bash +cd server/typescript/packages/metadata && bun test test/validation-filterable-array.test.ts +``` + +Expected: 3 pass, 0 fail. + +- [ ] **Step 5: Run the full metadata suite for regressions** + +```bash +cd server/typescript/packages/metadata && bun test +``` + +Expected: 0 fail. If an existing fixture breaks, it is declaring `@filterable` on an array field — read it before changing it; that fixture is pinning the defect. + +- [ ] **Step 6: Commit** + +```bash +git add server/typescript/packages/metadata/src/loader/validation-passes.ts \ + server/typescript/packages/metadata/test/validation-filterable-array.test.ts +git commit -m "fix(metamodel): an array field has no filter-operator band either (#335)" +``` + +--- + +## Task 2: Half B — `@sortable` gets the same subtype/array validation (TypeScript) + +**Files:** +- Modify: `server/typescript/packages/metadata/src/loader/validation-passes.ts` +- Test: `server/typescript/packages/metadata/test/validation-sortable-array.test.ts` (create) + +**Interfaces:** +- Consumes: Task 1's extended `validateFilterableHasSupportedOps`. +- Produces: `validateSortableHasSupportedSubtype(root: MetaData): ParseError[]`, exported alongside it and registered in the same pass list. Task 3 gates it. + +**Why:** `@sortable` defaults from `@filterable`, so it is only independently set when explicit — and **nothing validates it at all**, versus a hard error for `@filterable`. `@sortable: true` on a JSON or array column currently passes the loader and emits a sort entry over a column that cannot be ordered. + +- [ ] **Step 1: Write the failing test** + +Create `server/typescript/packages/metadata/test/validation-sortable-array.test.ts`: + +```ts +import { test, expect, describe } from "bun:test"; +import { MetaDataLoader } from "../src/loader/meta-data-loader.js"; + +const model = (fieldJson: string) => `{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { + "name": "Product", + "children": [ + { "source.rdb": { "@kind": "table", "@table": "products" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "@fields": ["id"] } }, + ${fieldJson} + ] + }} + ] + } +}`; + +describe("@sortable subtype validation", () => { + test("an array field marked @sortable fails to load", () => { + const src = model( + `{ "field.string": { "name": "tags", "isArray": true, "@sortable": true } }`, + ); + const loader = new MetaDataLoader(); + expect(() => loader.loadFromString(src, "meta.demo.json")).toThrow( + /ERR_SORTABLE_UNSUPPORTED_SUBTYPE/, + ); + }); + + test("a field.object marked @sortable fails to load", () => { + const src = model( + `{ "field.object": { "name": "spec", "@objectRef": "acme::shop::Spec", "@sortable": true } }`, + ); + const loader = new MetaDataLoader(); + expect(() => loader.loadFromString(src, "meta.demo.json")).toThrow( + /ERR_SORTABLE_UNSUPPORTED_SUBTYPE/, + ); + }); + + test("a plain scalar marked @sortable still loads", () => { + const src = model( + `{ "field.string": { "name": "sku", "@sortable": true } }`, + ); + const loader = new MetaDataLoader(); + expect(() => loader.loadFromString(src, "meta.demo.json")).not.toThrow(); + }); +}); +``` + +- [ ] **Step 2: Run the test to verify it fails** + +```bash +cd server/typescript/packages/metadata && bun test test/validation-sortable-array.test.ts +``` + +Expected: the first two FAIL (nothing thrown); the third PASSES. + +- [ ] **Step 3: Add the error code to the ledger** + +`ERR_SORTABLE_UNSUPPORTED_SUBTYPE` is new. Add it to **all** of: +- `fixtures/conformance/ERROR-CODES.json` +- `server/typescript/packages/metadata/src/errors.ts` (exact-bidirectional — the TS ledger test fails if the sets differ) +- `server/python/src/metaobjects/errors.py` (superset) +- `server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java` +- `server/csharp/MetaObjects/Errors.cs` + +Copy the description style of the neighbouring `ERR_FILTERABLE_UNSUPPORTED_SUBTYPE` entry. Do not paraphrase the rule differently in different ports — the strings are compared. + +- [ ] **Step 4: Write the validation pass** + +In `validation-passes.ts`, directly below `validateFilterableHasSupportedOps`: + +```ts +// @sortable on a subtype or shape that cannot be ordered (#335 Half B) +// --------------------------------------------------------------------------- +// @sortable defaults FROM @filterable, so it is independently set only when +// explicit — and nothing validated it, while @filterable has had a hard error +// since SP-H Unit9. A @sortable JSON or array column emits a sort entry over a +// column no dialect can ORDER BY meaningfully. → ERR_SORTABLE_UNSUPPORTED_SUBTYPE. + +export function validateSortableHasSupportedSubtype(root: MetaData): ParseError[] { + const errors: ParseError[] = []; + // ADR-0039: root has no super; children()==ownChildren() but resolving is the default. + for (const obj of root.children().filter((c) => c.type === TYPE_OBJECT)) { + // children() — inherited @sortable fields (via extends:/super:) are visible. + for (const field of obj.children().filter((c) => c.type === TYPE_FIELD)) { + // ADR-0039: resolving — a concrete field may inherit @sortable via extends. + if (field.attr(FIELD_ATTR_SORTABLE) !== true) continue; + // ADR-0039: resolvedIsArray(), never the own `isArray` flag. + const isArray = field.resolvedIsArray(); + if (!isArray && opsForSubType(field.subType).length > 0) continue; + errors.push( + new ParseError( + `Field "${obj.name}.${field.name}" has @sortable: true but ` + + (isArray + ? `is an array (isArray: true) — a collection column has no ordering.` + : `its subtype "${field.subType}" cannot be ordered.`) + + ` Remove @sortable from this field.`, + { code: "ERR_SORTABLE_UNSUPPORTED_SUBTYPE", source: field.source }, + ), + ); + } + } + return errors; +} +``` + +Register it in the same list that calls `validateFilterableHasSupportedOps` — grep for that name to find the pass registry and add the new function immediately after it. + +If `FIELD_ATTR_SORTABLE` does not exist in `constants.ts`, add it there beside `FIELD_ATTR_FILTERABLE`; do not inline `"sortable"`. + +- [ ] **Step 5: Run the test to verify it passes** + +```bash +cd server/typescript/packages/metadata && bun test test/validation-sortable-array.test.ts +``` + +Expected: 3 pass, 0 fail. + +- [ ] **Step 6: Run the full metadata suite** + +```bash +cd server/typescript/packages/metadata && bun test +``` + +Expected: 0 fail. + +- [ ] **Step 7: Commit** + +```bash +git add server/typescript/packages/metadata/src/loader/validation-passes.ts \ + server/typescript/packages/metadata/src/errors.ts \ + server/typescript/packages/metadata/test/validation-sortable-array.test.ts \ + fixtures/conformance/ERROR-CODES.json \ + server/python/src/metaobjects/errors.py \ + server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java \ + server/csharp/MetaObjects/Errors.cs +git commit -m "fix(metamodel): @sortable gets the subtype validation @filterable already had (#335)" +``` + +--- + +## Task 3: Half B — shared conformance fixtures + +**Files:** +- Create: `fixtures/conformance/error-filterable-array-field/input/meta.demo.json` +- Create: `fixtures/conformance/error-filterable-array-field/expected-errors.json` +- Create: `fixtures/conformance/error-sortable-array-field/input/meta.demo.json` +- Create: `fixtures/conformance/error-sortable-array-field/expected-errors.json` +- Modify: `fixtures/conformance/README.md` + +**Interfaces:** +- Consumes: the two rules from Tasks 1–2. +- Produces: the cross-port gate. Task 4's port work is verified against these. + +**Why:** This is the point of Half B. A load error with no fixture that triggers it is the same blind spot one layer up — "no fixture covers it" and "every port enforces it" are indistinguishable on a green suite. A structural scan of 1321 JSON and 124 YAML files found **zero** fields carrying both `isArray: true` and `@filterable`/`@sortable: true`, so nothing in the corpus exercises an array field through the filter tier at all. + +- [ ] **Step 1: Create the filterable fixture input** + +`fixtures/conformance/error-filterable-array-field/input/meta.demo.json`: + +```json +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Product", + "children": [ + { "source.rdb": { "@kind": "table", "@table": "products" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "@fields": ["id"] } }, + { "field.string": { "name": "tags", "isArray": true, "@filterable": true } } + ] + } + } + ] + } +} +``` + +- [ ] **Step 2: Create the filterable expected-errors** + +`fixtures/conformance/error-filterable-array-field/expected-errors.json`: + +```json +{ + "errors": [ + { + "code": "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE", + "source": { + "format": "json", + "files": ["meta.demo.json"], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[3]['field.string']" + } + } + ], + "warnings": [] +} +``` + +- [ ] **Step 3: Create the sortable fixture** + +Same two files under `fixtures/conformance/error-sortable-array-field/`, with the field changed to `"@sortable": true` and the code to `ERR_SORTABLE_UNSUPPORTED_SUBTYPE`. + +- [ ] **Step 4: Run the TS conformance corpus** + +```bash +cd server/typescript/packages/metadata && bun test test/conformance.test.ts +``` + +Expected: PASS, count increased by 2. If the `jsonPath` in `expected-errors.json` does not match, the failure message prints the actual path — copy it verbatim rather than guessing. + +- [ ] **Step 5: Document the fixtures in the corpus README** + +In `fixtures/conformance/README.md`, add both fixtures to the case list, each with one line saying which rule it covers. State that they are the **only** cases exercising an array field through the filter/sort tier. + +- [ ] **Step 6: Commit** + +```bash +git add fixtures/conformance/error-filterable-array-field \ + fixtures/conformance/error-sortable-array-field \ + fixtures/conformance/README.md +git commit -m "test(conformance): gate the array filter/sort rules cross-port (#335)" +``` + +--- + +## Task 4: Half B — port the two rules to C#, Java, Python + +**Files:** +- Modify: `server/csharp/MetaObjects/Loader/ValidationPasses.cs` +- Modify: `server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java` +- Modify: `server/python/src/metaobjects/loader/validation_passes.py` + +**Interfaces:** +- Consumes: Tasks 1–3. The TypeScript implementation is the reference; the fixtures are the contract. +- Produces: all four ports green on the two fixtures from Task 3. + +**Why:** The loader contract is cross-port. Four ports hand-rolling the same read is what produced the `@fields` divergence in #342 — so implement each against the fixture, not against a reading of the TS source. + +- [ ] **Step 1: Find each port's filterable pass** + +```bash +grep -rn "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE" server/csharp server/java server/python --include=*.cs --include=*.java --include=*.py +``` + +Read each one before editing. Each port already has this rule; you are adding the array condition beside it and adding the sortable pass after it. + +- [ ] **Step 2: Use each port's RESOLVING array accessor** + +This is the single highest-risk detail. Do **not** read a raw `isArray` field or a raw attr. + +| Port | Correct accessor | +|---|---| +| TypeScript | `field.resolvedIsArray()` | +| Python | `node.resolved_is_array()` | +| C# / Java | grep the port's existing `@agg:collect` array check — it already reads array-ness the resolving way; copy that call exactly | + +For C# and Java: + +```bash +grep -rn "collect produces a list" server/csharp server/java +``` + +The line above that error is the resolving accessor to reuse. + +- [ ] **Step 3: Implement in each port** + +Mirror the TypeScript semantics exactly: +- `@filterable: true` + array ⇒ `ERR_FILTERABLE_UNSUPPORTED_SUBTYPE` +- `@sortable: true` + (array **or** no operator band) ⇒ `ERR_SORTABLE_UNSUPPORTED_SUBTYPE` +- Neither fires when the attr is absent or `false`. + +- [ ] **Step 4: Run each port's conformance suite** + +```bash +cd server/python && uv run pytest --extra integration -k conformance +cd server/csharp && dotnet build && dotnet test --filter "FullyQualifiedName~Conformance" +cd server/java && mvn -q -pl metadata test +``` + +Expected: all green on the two new fixtures. + +**Two traps, both previously shipped:** `dotnet test` prints `Passed!` even when a project failed to COMPILE — grep the output for `error CS` and confirm the build succeeded separately. Never pipe a Maven run through `tail`; the exit status becomes `tail`'s. + +- [ ] **Step 5: Commit** + +```bash +git add server/csharp/MetaObjects/Loader/ValidationPasses.cs \ + server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java \ + server/python/src/metaobjects/loader/validation_passes.py +git commit -m "fix(metamodel): array filter/sort rules in the remaining three ports (#335)" +``` + +--- + +## Task 5: Half A — `@of` optional on `collect` (TypeScript loader) + +**Files:** +- Modify: `server/typescript/packages/metadata/src/loader/validation-passes.ts` — the block under the comment `// --- count/sum/avg/min/max/collect: @of REQUIRED ---` +- Test: `server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts` (create) + +**Interfaces:** +- Consumes: nothing from Half B (independent rules, same file). +- Produces: the `@of`-absent collect branch. Task 6 adds member resolution; Task 8 ports it. + +**Why:** Today `@of` is required for `collect`, so only a single scalar column can be rolled up. `@of` is already `"required": false` in the registry — the constraint is validation-only. + +> ### CORRECTION — use this model, not the one written below +> +> The model in Step 1 is **wrong** and was verified so by execution. It produces two structural +> errors that have nothing to do with #335: `ERR_SUBTYPE_RULE_VIOLATION` (*"a projection may +> only extend another projection"* — it has `object.projection extends "Product"`, an entity) +> and `ERR_PROJECTION_IDENTITY_NOT_EXTENDED` (a projection identity must `extends` an entity +> identity, not declare fresh `@fields`). +> +> Use this shape instead — copied from the corpus's own `error-origin-aggregate-no-to-many` +> and **verified**: the scalar baseline loads with **0 errors**, and the whole-object form +> produces **exactly one** error, `ERR_INVALID_ORIGIN … missing @of`. +> +> ```jsonc +> { "object.projection": { "name": "ProductWithSuppliers", "children": [ +> { "field.uuid": { "name": "productId", "extends": "acme::Product.id" } }, +> , +> { "identity.primary": { "name": "id", "extends": "acme::Product.id" } } +> ]}} +> ``` +> +> The projection carries **no** object-level `extends`; each field carries its own, and the +> identity uses `extends` rather than `@fields`. Entities are plain: `source.rdb @table`, +> `field.uuid id`, `identity.primary { name: "id", @fields: ["id"] }`, and `Product` holds +> `relationship.association { name: "suppliers", @objectRef: "acme::Supplier", @cardinality: "many" }`. +> +> Also note: `MetaDataLoader.load()` is **async** and returns errors on the result — it does +> not throw. `await` it. Copy the exact harness from +> `server/typescript/packages/metadata/test/validation-filterable-array.test.ts`. + +- [ ] **Step 1: Write the failing test** + +Create `server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts`. Include a **positive** case and each **negative** arm: + +```ts +import { test, expect, describe } from "bun:test"; +import { MetaDataLoader } from "../src/loader/meta-data-loader.js"; + +/** Product 1:N Supplier, plus a projection rolling suppliers up as objects. */ +const model = (collectField: string, extra = "") => `{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { "name": "Supplier", "children": [ + { "source.rdb": { "@kind": "table", "@table": "suppliers" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "name" } }, + { "field.long": { "name": "productId" } }, + { "identity.primary": { "@fields": ["id"] } }, + { "identity.reference": { "name": "product", "@references": "Product", "@fields": ["productId"] } } + ]}}, + { "object.entity": { "name": "Product", "children": [ + { "source.rdb": { "@kind": "table", "@table": "products" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "@fields": ["id"] } }, + { "relationship.association": { "name": "suppliers", "@cardinality": "many", "@objectRef": "Supplier" } } + ]}}, + { "object.value": { "name": "SupplierBrief", "children": [ + { "field.long": { "name": "id" } }, + { "field.string": { "name": "name" } } + ]}}, + ${extra} + { "object.projection": { "name": "ProductWithSuppliers", "extends": "Product", "children": [ + { "source.rdb": { "@kind": "view", "@view": "v_product_suppliers" } }, + { "identity.primary": { "@fields": ["id"] } }, + ${collectField} + ]}} + ] + } +}`; + +const load = (src: string) => new MetaDataLoader().loadFromString(src, "meta.demo.json"); + +const WHOLE_OBJECT = `{ "field.object": { + "name": "supplierBriefs", "isArray": true, "@objectRef": "SupplierBrief", + "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers" } } ] +}}`; + +describe("@of-absent collect (whole-object rollup)", () => { + test("loads on a field.object @objectRef isArray with @via", () => { + expect(() => load(model(WHOLE_OBJECT))).not.toThrow(); + }); + + test("fails without @objectRef", () => { + expect(() => load(model(`{ "field.object": { + "name": "supplierBriefs", "isArray": true, + "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers" } } ] + }}`))).toThrow(/ERR_INVALID_ORIGIN/); + }); + + test("fails when @objectRef targets an entity, not a value", () => { + expect(() => load(model(`{ "field.object": { + "name": "supplierBriefs", "isArray": true, "@objectRef": "Supplier", + "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers" } } ] + }}`))).toThrow(/ERR_SUBTYPE_RULE_VIOLATION/); + }); + + test("fails without @via (nothing to infer the relation from)", () => { + expect(() => load(model(`{ "field.object": { + "name": "supplierBriefs", "isArray": true, "@objectRef": "SupplierBrief", + "children": [ { "origin.aggregate": { "@agg": "collect" } } ] + }}`))).toThrow(/ERR_INVALID_ORIGIN/); + }); + + test("fails when @distinct is declared", () => { + expect(() => load(model(`{ "field.object": { + "name": "supplierBriefs", "isArray": true, "@objectRef": "SupplierBrief", + "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers", "@distinct": true } } ] + }}`))).toThrow(/ERR_INVALID_ORIGIN/); + }); + + test("a collect WITH @of is unaffected", () => { + expect(() => load(model(`{ "field.string": { + "name": "supplierNames", "isArray": true, + "children": [ { "origin.aggregate": { "@agg": "collect", "@of": "Supplier.name", "@via": "Product.suppliers" } } ] + }}`))).not.toThrow(); + }); +}); +``` + +- [ ] **Step 2: Run the test to verify it fails** + +```bash +cd server/typescript/packages/metadata && bun test test/validation-collect-whole-object.test.ts +``` + +Expected: the positive cases FAIL with `missing @of`; the negatives may pass for the wrong reason. That is fine — Step 3 makes all six correct. + +- [ ] **Step 3: Split `collect` out of the `@of`-required gate** + +Replace the block beginning `// --- count/sum/avg/min/max/collect: @of REQUIRED ---` with: + +```ts + // --- @of: REQUIRED for count/sum/avg/min/max; OPTIONAL for collect --- + // #335 — an @of-absent collect is a WHOLE-OBJECT rollup: collect the + // related rows as an array of the field's declared @objectRef value + // object rather than an array of one scalar column. + if (!ofPresent) { + if (!isCollect) { + errors.push(new ParseError( + `origin.aggregate on ${obj.name}.${field.name}: missing @of.`, + { code: "ERR_INVALID_ORIGIN", source: src })); + continue; + } + // Whole-object rollup. The carrying field must be a field.object + // naming a value object, and @via must be explicit (there is no @of + // entity to infer the single-hop relation from). + // ADR-0039: resolving — @objectRef may be inherited via extends. + const objectRef = field.attr(FIELD_ATTR_OBJECT_REF); + if (field.subType !== FIELD_SUBTYPE_OBJECT || typeof objectRef !== "string" || objectRef === "") { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: @of is omitted, so this is a ` + + `whole-object rollup — the carrying field must be a field.object declaring @objectRef ` + + `(add @of to collect a single column instead).`, + { code: "ERR_INVALID_ORIGIN", source: src })); + continue; + } + // #210's value-only rule is PAYLOAD-scoped and never reaches a + // projection-hosted field, so this branch enforces it itself. + // Without it an @objectRef to an entity silently rolls up the FULL + // entity — the #270 shape, this time baked into DDL. + const refPkg = field.parent?.package ?? obj.package ?? ""; + const refTarget = resolveObjectRef(root, objectRef, refPkg).node; + if (refTarget !== undefined && refTarget.subType !== OBJECT_SUBTYPE_VALUE) { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: @objectRef '${objectRef}' ` + + `resolves to ${TYPE_OBJECT}.${refTarget.subType} — a whole-object rollup must target an ` + + `object.value (#210, ADR-0028).`, + { code: "ERR_SUBTYPE_RULE_VIOLATION", source: src })); + continue; + } + // ADR-0039: own — origin.* never inherits (ADR-0029). + const viaAttr = origin.ownAttr(ORIGIN_AGGREGATE_ATTR_VIA); + if (typeof viaAttr !== "string" || viaAttr === "") { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: @via is required on a ` + + `whole-object rollup — there is no @of entity to infer the relationship from.`, + { code: "ERR_INVALID_ORIGIN", source: src })); + continue; + } + // @distinct is refused on the object form. It is NOT an engine limit + // (both engines dedupe JSON objects); it is a guaranteed no-op + // whenever the value object carries the entity's primary key, which + // is the common case, and a silent no-op is worse than a refusal. + if (hasDistinct) { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: @distinct is not supported on a ` + + `whole-object rollup (it is a no-op whenever the value object carries the primary key).`, + { code: "ERR_INVALID_ORIGIN", source: src })); + continue; + } + const hops = _validateViaPath(viaAttr, root, obj, field.name, src, errors); + if (hops !== undefined) _checkAggregateCardinality(hops, obj, field.name, src, errors); + // @orderBy keys resolve against the @via TERMINAL entity, not @of. + if (hasOrderBy) { + const terminal = _viaTerminalEntityNode(viaAttr, root, obj); + _validateOrderByKeys(orderBy, terminal, obj, field.name, "origin.aggregate @agg:collect", src, errors); + } + continue; + } +``` + +Add a `_viaTerminalEntityNode` helper beside `_validateViaPath` that walks the same dotted `@via` segments and returns the terminal entity node (or `undefined`). `extract-view-spec.ts:477` has the codegen-side equivalent (`viaTerminalEntity`) — mirror its hop-walking logic, returning the loader's node type. + +Import any constants not already imported (`FIELD_SUBTYPE_OBJECT`, `FIELD_ATTR_OBJECT_REF`, `OBJECT_SUBTYPE_VALUE`, `resolveObjectRef`). + +- [ ] **Step 4: Run the test to verify it passes** + +```bash +cd server/typescript/packages/metadata && bun test test/validation-collect-whole-object.test.ts +``` + +Expected: 6 pass, 0 fail. + +- [ ] **Step 5: Run the full metadata suite** + +```bash +cd server/typescript/packages/metadata && bun test +``` + +Expected: 0 fail. + +- [ ] **Step 6: Commit** + +```bash +git add server/typescript/packages/metadata/src/loader/validation-passes.ts \ + server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts +git commit -m "feat(metamodel): @of is optional on @agg:collect — whole-object rollup (#335)" +``` + +--- + +## Task 6: Half A — value-object member resolution (the #270 guard) + +**Files:** +- Modify: `server/typescript/packages/metadata/src/loader/validation-passes.ts` (the branch from Task 5) +- Modify: the five error ledgers (as in Task 2, Step 3) +- Test: `server/typescript/packages/metadata/test/validation-collect-member-resolution.test.ts` (create) + +**Interfaces:** +- Consumes: Task 5's `@of`-absent branch and its `_viaTerminalEntityNode` helper. +- Produces: `ERR_COLLECT_MEMBER_UNRESOLVED`. Task 10's emitter relies on every member resolving. + +**Why:** The lowering projects exactly the declared value-object's members. A member that matches no column on the `@via` terminal entity must be an **error** — failing open here is how #270 turned a curated value object into the full entity. Element type stays declared-authoritative. + +- [ ] **Step 1: Write the failing test** + +Create `server/typescript/packages/metadata/test/validation-collect-member-resolution.test.ts`, reusing the model helper from Task 5 (copy it — the engineer may read tasks out of order) with two cases: + +```ts + test("a VO member with no matching column on the @via terminal fails", () => { + // SupplierBrief declares `nickname`; Supplier has no such field. + expect(() => load(modelWithVo( + `{ "field.long": { "name": "id" } }, { "field.string": { "name": "nickname" } }`, + WHOLE_OBJECT, + ))).toThrow(/ERR_COLLECT_MEMBER_UNRESOLVED/); + }); + + test("a VO member whose type differs from the matched column fails", () => { + // SupplierBrief declares `name` as a long; Supplier.name is a string. + expect(() => load(modelWithVo( + `{ "field.long": { "name": "id" } }, { "field.long": { "name": "name" } }`, + WHOLE_OBJECT, + ))).toThrow(/ERR_INVALID_ORIGIN/); + }); +``` + +Write `modelWithVo(voFields, collectField)` as a variant of Task 5's `model` that parameterises `SupplierBrief`'s field list. + +- [ ] **Step 2: Run the test to verify it fails** + +```bash +cd server/typescript/packages/metadata && bun test test/validation-collect-member-resolution.test.ts +``` + +Expected: both FAIL (nothing thrown). + +- [ ] **Step 3: Add `ERR_COLLECT_MEMBER_UNRESOLVED` to all five ledgers** + +Same five files as Task 2, Step 3. + +- [ ] **Step 4: Implement member resolution** + +Inside Task 5's branch, after the `@via` hop validation and before its `continue`: + +```ts + // Member resolution — the lowering projects EXACTLY the declared value + // object's members, matched by NAME against the @via terminal entity. + // An unmatched member must error: failing open is how #270 turned a + // curated value object into the full entity. + const terminalEntity = _viaTerminalEntityNode(viaAttr, root, obj); + if (terminalEntity !== undefined && refTarget !== undefined) { + // children() — a value object may inherit members via extends. + const terminalFields = terminalEntity.children().filter((c) => c.type === TYPE_FIELD); + for (const member of refTarget.children().filter((c) => c.type === TYPE_FIELD)) { + const match = terminalFields.find((f) => f.name === member.name); + if (match === undefined) { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: value-object member ` + + `'${member.name}' has no matching field on '${terminalEntity.name}' — a whole-object ` + + `rollup projects exactly the declared members.`, + { code: "ERR_COLLECT_MEMBER_UNRESOLVED", source: src })); + continue; + } + // Per-member type agreement — the object-form analogue of the + // scalar element-type check, same #185 type-preserving doctrine. + if (member.subType !== match.subType) { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: value-object member ` + + `'${member.name}' is field.${member.subType} but '${terminalEntity.name}.${match.name}' ` + + `is field.${match.subType} — a whole-object rollup preserves each member's type.`, + { code: "ERR_INVALID_ORIGIN", source: src })); + } + } + } +``` + +- [ ] **Step 5: Run both Half A tests** + +```bash +cd server/typescript/packages/metadata && bun test test/validation-collect-member-resolution.test.ts test/validation-collect-whole-object.test.ts +``` + +Expected: 8 pass, 0 fail. + +- [ ] **Step 6: Commit** + +```bash +git add server/typescript/packages/metadata/src/loader/validation-passes.ts \ + server/typescript/packages/metadata/src/errors.ts \ + server/typescript/packages/metadata/test/validation-collect-member-resolution.test.ts \ + fixtures/conformance/ERROR-CODES.json \ + server/python/src/metaobjects/errors.py \ + server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java \ + server/csharp/MetaObjects/Errors.cs +git commit -m "feat(metamodel): whole-object rollup resolves VO members against the @via terminal (#335)" +``` + +--- + +## Task 7: Half A — conformance fixtures (positive + 6 negative arms) + +**Files:** +- Create: `fixtures/conformance/collect-whole-object/{input/meta.demo.json,expected.json}` +- Create: six `fixtures/conformance/error-collect-*/` directories, each with `input/meta.demo.json` + `expected-errors.json` +- Modify: `fixtures/conformance/README.md` + +**Interfaces:** +- Consumes: Tasks 5–6. +- Produces: the cross-port contract Task 8 implements against. + +The six negative arms, one fixture each: + +| Fixture | Arm | Code | +|---|---|---| +| `error-collect-no-object-ref` | no `@objectRef` | `ERR_INVALID_ORIGIN` | +| `error-collect-ref-not-value` | `@objectRef` → entity | `ERR_SUBTYPE_RULE_VIOLATION` | +| `error-collect-no-via` | `@via` absent | `ERR_INVALID_ORIGIN` | +| `error-collect-via-to-one` | every hop to-one | `ERR_ORIGIN_CARDINALITY` | +| `error-collect-member-unresolved` | VO member not on terminal | `ERR_COLLECT_MEMBER_UNRESOLVED` | +| `error-collect-distinct` | `@distinct` declared | `ERR_INVALID_ORIGIN` | + +- [ ] **Step 1: Build the positive fixture** + +Use the model from Task 5's test as `input/meta.demo.json`. Generate `expected.json` by running the corpus and copying the canonical serialization the runner prints on mismatch — **do not hand-write it**. + +- [ ] **Step 2: Build the six negative fixtures** + +Each is the positive input with one thing changed. Copy the `expected-errors.json` shape from `fixtures/conformance/error-origin-aggregate-no-to-many/expected-errors.json`; take the `jsonPath` from the runner's failure output rather than deriving it. + +- [ ] **Step 3: Run the corpus** + +```bash +cd server/typescript/packages/metadata && bun test test/conformance.test.ts +``` + +Expected: PASS, count increased by 7. + +- [ ] **Step 4: Document in the README** + +Add all seven to the case list. For the positive one, state that it is the corpus's only whole-object rollup. + +- [ ] **Step 5: Commit** + +```bash +git add fixtures/conformance/collect-whole-object fixtures/conformance/error-collect-* fixtures/conformance/README.md +git commit -m "test(conformance): gate the whole-object rollup and its six error arms (#335)" +``` + +--- + +## Task 7b: Half A — mint `ERR_COLLECT_WHOLE_OBJECT` so the fixtures discriminate + +**Added after Task 7, ruled by Doug 2026-08-23.** Not in the original plan. + +**Files:** +- Modify: the six error ledgers — `fixtures/conformance/ERROR-CODES.json`, + `server/typescript/packages/metadata/src/errors.ts`, + `server/python/src/metaobjects/errors.py`, `server/csharp/MetaObjects/Errors.cs`, + `server/java/.../com/metaobjects/ErrorCode.java`, + `server/java/.../com/metaobjects/util/ErrorMessageConstants.java` +- Modify: `server/typescript/packages/metadata/src/loader/validation-passes.ts` +- Modify: the five fixtures' `expected-errors.json` +- Modify: `server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts` + +**Interfaces:** +- Consumes: Tasks 5–7 (the eight rules and the nine fixtures). +- Produces: a corpus that goes RED on an unported port, which is Task 8's signal. + +**Why:** five of the eight negative fixtures assert `ERR_INVALID_ORIGIN` — the same code +a loader that still REQUIRES `@of` emits for that same metadata. The corpus compares only +`code` + `source` (`spec/conformance-tests.md` §"Errors and warnings"; *"Error message text +is never compared"*), so those five pass **vacuously** on a port that implements nothing. +Verified: Python conformance failed 4 of the 9 before this task. + +**Scope — the five arms that move**, all inside the `@of`-absent branch: +carrier-not-`field.object`/no-`@objectRef`; `@via` required; `@distinct` refused; +`@orderBy` key not on the `@via` TERMINAL entity; value-object member type disagreement. +Rules 2/5/7 keep their existing distinct codes (`ERR_SUBTYPE_RULE_VIOLATION`, +`ERR_ORIGIN_CARDINALITY`, `ERR_COLLECT_MEMBER_UNRESOLVED`) — untouched. + +- [x] **Step 1: Six ledgers.** Mirror the `ERR_COLLECT_MEMBER_UNRESOLVED` entries added in + `e67cd9340`. Java needs BOTH `ErrorCode.java` and `ErrorMessageConstants.java`. + The TS ledger test compares SORTED sets, so placement is free — keep it beside its sibling. + +- [x] **Step 2: The `@orderBy` arm goes through a SHARED helper.** `_validateOrderByKeys` is + also called by the scalar `@of` arm and by `origin.first`. Give it a trailing + `code: ErrorCode = "ERR_INVALID_ORIGIN"` parameter and pass the new code from the + whole-object call site only — the other two call sites keep their envelope byte-for-byte. + +- [x] **Step 3: Retarget the five fixtures.** In `error-collect-no-object-ref` change only the + FIRST entry; `ERR_OBJECT_FIELD_WITHOUT_OBJECT_REF` stays second (the envelope assertion is + positional, and all four ports run origin validation before field-object-storage validation). + +- [x] **Step 4: Prove the signal is real.** Run an UNPORTED port's corpus and confirm the + failure count rises from 4 to 9. + +```bash +cd server/python && uv run --extra integration pytest -q -k conformance +``` + +- [x] **Step 5: Commit** + +```bash +git add fixtures/conformance/ERROR-CODES.json fixtures/conformance/error-collect-* \ + server/typescript/packages/metadata/src/errors.ts \ + server/typescript/packages/metadata/src/loader/validation-passes.ts \ + server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts \ + server/python/src/metaobjects/errors.py server/csharp/MetaObjects/Errors.cs \ + server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java \ + server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java +git commit -m "feat(metamodel): the whole-object arm gets its own error code (#335)" +``` + +--- + +## Task 8: Half A — port the loader rules to C#, Java, Python + +**Files:** +- Modify: `server/csharp/MetaObjects/Loader/ValidationPasses.cs` +- Modify: `server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java` +- Modify: `server/python/src/metaobjects/loader/validation_passes.py` + +**Interfaces:** +- Consumes: Tasks 5–7. The NINE fixtures are the contract (8 negative + 1 positive). + Task 7 shipped eight rules, not the six this plan was written against — the ledger + recorded the scope correction; this text was stale. Port all EIGHT. +- Produces: four ports green on the whole corpus. + +- [ ] **Step 1: Locate each port's gate** + +```bash +grep -rn "missing @of" server/csharp server/java server/python +``` + +Python phrases it differently — search for its `AGG_COLLECT` handling near the `collect produces a list` message instead. + +- [ ] **Step 2: Implement the same EIGHT rules per port** + +Object-ref required; ref must be `object.value`; `@via` required; cardinality; member +resolution; member TYPE agreement (subtype AND array-ness — compare the +`field.[]` label, as TS does); `@orderBy` keys resolve against the `@via` +TERMINAL entity (not the head or a middle hop); `@distinct` refused. Use each port's **resolving** accessors throughout (see Task 4, Step 2). + +- [ ] **Step 3: Run each port's conformance suite** + +```bash +cd server/python && uv run pytest --extra integration -k conformance +cd server/csharp && dotnet build && dotnet test --filter "FullyQualifiedName~Conformance" +cd server/java && mvn -q -pl metadata test +``` + +Expected: green on all nine new fixtures. Same two traps as Task 4, Step 4. + +**Task 7b closed the vacuous-pass hole.** Those five negatives used to assert +`ERR_INVALID_ORIGIN` — what an unported loader emits for the old "missing @of" — so they +passed before a line was written. They now assert `ERR_COLLECT_WHOLE_OBJECT`, and all NINE +fail on an unported port (Python: 9 failed / 568 passed, measured). So the corpus IS a +red-before/green-after signal per rule. + +It still is not proof that each fixture fails for its OWN reason rather than a neighbour's +guard. Verify that the way Task 7's review did: neutralise one guard at a time +(`if (false && )`) in a COPY under /tmp and re-run that fixture. Never `git stash` — +worktrees share one stash list, so a stash op reaches other sessions' work. + +- [ ] **Step 4: Commit** + +```bash +git add server/csharp/MetaObjects/Loader/ValidationPasses.cs \ + server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java \ + server/python/src/metaobjects/loader/validation_passes.py +git commit -m "feat(metamodel): whole-object rollup in the remaining three ports (#335)" +``` + +--- + +## Task 9: Half A — registry prose in seven byte-gated files + +**Files:** the seven listed under **File Structure**. + +**Interfaces:** +- Consumes: Tasks 5–8 (the rule the prose describes). +- Produces: `registry-conformance` green in all five ports. + +**Why:** `@of` is already `"required": false` structurally, so nothing changes there — but its `description` says *"Required for count/sum/avg/min/max/collect"*, which is now false. That string is byte-gated in seven files; change fewer than seven and `registry-conformance` goes red in whichever port you missed. + +- [ ] **Step 1: Find every copy** + +```bash +grep -rln "Required for count/sum/avg/min/max/collect" . --exclude-dir=docs +``` + +Expected: exactly **7 product files** — the seven listed under **File Structure**. + +**Exclude `docs/superpowers/`.** Without that flag the grep returns **9**, because this plan and +its spec both quote the sentence. Verified. Editing those two is harmless but pointless; they +are prose about the change, not copies of the contract. + +- [ ] **Step 2: Rewrite the sentence identically in all seven** + +New text, byte-identical everywhere: + +``` +Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max; OPTIONAL for collect, where absent means a whole-object rollup of the field's declared @objectRef value object; forbidden for any/all (which quantify over rows via @filter, not a column). +``` + +- [ ] **Step 3: Regenerate the derived docs fixture** + +`fixtures/metamodel-docs/expected/types/origin.md` is generated. Find its generator command in `fixtures/metamodel-docs/README.md` and run it rather than hand-editing. + +- [ ] **Step 4: Run registry conformance in every port** + +```bash +cd server/typescript/packages/metadata && bun test test/registry-conformance.test.ts +cd server/python && uv run pytest --extra integration -k registry +cd server/csharp && dotnet build && dotnet test --filter "FullyQualifiedName~Registry" +cd server/java && mvn -q -pl metadata test -Dtest='*Registry*' +``` + +Expected: all green. + +- [ ] **Step 5: Confirm `metamodelVersion` still needs no move** + +```bash +node scripts/check-metamodel-version.mjs --explain +git tag --list 'v0.*' --sort=-v:refname | head -1 +``` + +If the newest release tag is still `v0.24.0`, `0.11` is unreleased and no edit is needed. If `v0.24.1` has shipped, run `node scripts/check-metamodel-version.mjs --set 0.12`. + +- [ ] **Step 6: Commit** + +```bash +git add spec/metamodel/origin.json fixtures/registry-conformance/expected-registry.json \ + fixtures/metamodel-docs/expected/types/origin.md \ + server/csharp/MetaObjects/Persistence/Origin/OriginSchema.cs \ + server/csharp/MetaObjects/SpecMetamodel/origin.json \ + server/python/src/metaobjects/spec_metamodel/origin.json \ + server/typescript/packages/metadata/src/persistence/origin/origin-definition.embedded.ts +git commit -m "docs(metamodel): @of is optional on collect, in all seven byte-gated copies (#335)" +``` + +--- + +## Task 10: Half A — the `collectObjectAgg` view column (TypeScript codegen) + +**Files:** +- Modify: `server/typescript/packages/codegen-ts/src/projection/view-spec.ts` +- Modify: `server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts` +- Test: `server/typescript/packages/codegen-ts/test/projection/collect-whole-object-spec.test.ts` (create) + +**Interfaces:** +- Consumes: the loader guarantees from Tasks 5–8 (every member resolves; `@objectRef` is a value; `@via` present). +- Produces: + +```ts +{ + readonly kind: "collectObjectAgg"; + readonly fieldName: string; + readonly dbColAlias: string; + readonly sourceAlias: string; + readonly joinedPkColumn: string; + readonly members: readonly { readonly memberName: string; readonly sourceColumn: string }[]; + readonly orderBy: readonly ViewOrderKey[]; +} +``` + +Task 11 lowers this to SQL. + +**Why a separate kind, not an arm of `collectAgg`:** the payloads differ (a member list vs one source column), and `collectAgg` is consumed by `viewOrderKeysAreDeterministic` (`:1005`) and the real-aggregate predicate (`:1021`), which a union would force every consumer to re-narrow. + +- [ ] **Step 1: Add the kind to `view-spec.ts`** + +Add the interface above to the `ViewColumn` union, beside the existing `collectAgg` member. + +- [ ] **Step 2: Write the failing test** + +Create the test file; build the same Product/Supplier/SupplierBrief model as Task 5, run `extractViewSpec`, and assert: + +```ts + const col = spec.columns.find((c) => c.fieldName === "supplierBriefs"); + expect(col?.kind).toBe("collectObjectAgg"); + expect(col?.members).toEqual([ + { memberName: "id", sourceColumn: "id" }, + { memberName: "name", sourceColumn: "name" }, + ]); + expect(col?.orderBy).toEqual([]); // default = PK asc, applied at emit + expect(col?.joinedPkColumn).toBe("id"); +``` + +- [ ] **Step 3: Run to verify it fails** + +```bash +cd server/typescript/packages/codegen-ts && bun test test/projection/collect-whole-object-spec.test.ts +``` + +Expected: FAIL — the column is absent entirely, because `if (!of_) continue;` skips it. + +- [ ] **Step 4: Restructure the branch** + +In `extract-view-spec.ts`, the `if (!of_) continue;` at ~`:891` sits **above** the `AGG_COLLECT` branch and must no longer swallow the object form. Add the object arm **before** that guard, mirroring the `any`/`all` branch at `:861-885`: + +```ts + // #335 — @of ABSENT on collect is a whole-object rollup. Resolve the related + // entity from @via's terminal hop (not from @of), exactly as any/all does. + if (agg === AGG_COLLECT && !of_) { + const via = origin.ownAttr(ORIGIN_AGGREGATE_ATTR_VIA) as string | undefined; + if (!via) continue; // loader already errored + const objectRef = field.attr(FIELD_ATTR_OBJECT_REF) as string | undefined; + if (!objectRef) continue; // loader already errored + const relatedName = viaTerminalEntity(via, root, projPkg); + if (!relatedName) continue; + const relatedEntity = resolveEntityRef(root, relatedName, projPkg); + const sourceAlias = findAliasInTree(joinTree, relatedName); + if (!relatedEntity || sourceAlias === undefined) continue; + const joinedPk = primaryKeyColumn(relatedEntity, ctx); + if (joinedPk === undefined) continue; + const vo = resolveObjectRef(root, objectRef, projPkg).node; + if (!vo) continue; + // children() — a value object may inherit members via extends. + const members = vo.children() + .filter((c): c is MetaField => c.type === TYPE_FIELD) + .map((m) => { + const target = relatedEntity.fields().find((f) => f.name === m.name); + // The loader guarantees every member resolves (ERR_COLLECT_MEMBER_UNRESOLVED). + return target === undefined + ? undefined + : { memberName: m.name, sourceColumn: sourceColumnNameFor(target, ctx) }; + }) + .filter((m): m is { memberName: string; sourceColumn: string } => m !== undefined); + if (members.length === 0) continue; + columns.push({ + kind: "collectObjectAgg", + fieldName: field.name, + dbColAlias: dbCol, + sourceAlias, + joinedPkColumn: joinedPk, + members, + orderBy: resolveOrderByKeys(origin.ownAttr(ORIGIN_ATTR_ORDER_BY), relatedEntity, ctx), + }); + continue; + } +``` + +Then update `:1005` and `:1021` to treat `collectObjectAgg` the same way they treat `collectAgg` (deterministic ordering; a real aggregate). + +- [ ] **Step 5: Run the test to verify it passes** + +```bash +cd server/typescript/packages/codegen-ts && bun test test/projection/collect-whole-object-spec.test.ts +``` + +Expected: PASS. + +- [ ] **Step 6: Commit** + +```bash +git add server/typescript/packages/codegen-ts/src/projection/view-spec.ts \ + server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts \ + server/typescript/packages/codegen-ts/test/projection/collect-whole-object-spec.test.ts +git commit -m "feat(codegen-ts): extract a collectObjectAgg column from an @of-less collect (#335)" +``` + +--- + +## Task 11: Half A — SQL lowering, both dialects + +**Files:** +- Modify: `server/typescript/packages/codegen-ts/src/projection/view-ddl-emit.ts` +- Test: `server/typescript/packages/codegen-ts/test/projection/collect-whole-object-ddl.test.ts` (create) + +**Interfaces:** +- Consumes: Task 10's `collectObjectAgg`. +- Produces: the emitted SQL Task 13 applies to real engines. + +**Why `jsonb`, not `json`:** verified against PostgreSQL 15.15 — `json` has neither an equality nor an ordering operator (`ERROR: operator does not exist: json = json`; `could not identify an ordering operator for type json`), so `json_agg(json_build_object(…) ORDER BY …)` **does not run**. `field.object` is already `jsonb` in `column-mapper.ts:610-613`. + +- [ ] **Step 1: Write the failing test** + +```ts + test("postgres: jsonb_agg of jsonb_build_object, PK-asc, empty-set guarded", () => { + expect(emitColumn(col, "postgres")).toBe( + `COALESCE(jsonb_agg(jsonb_build_object('id', s."id", 'name', s."name") ORDER BY s."id" ASC) ` + + `FILTER (WHERE s."id" IS NOT NULL), '[]'::jsonb) AS "supplierBriefs"`, + ); + }); + + test("sqlite: json_group_array of json_object, PK-asc, empty-set guarded", () => { + expect(emitColumn(col, "sqlite")).toBe( + `COALESCE(json_group_array(json_object('id', s."id", 'name', s."name") ORDER BY s."id" ASC) ` + + `FILTER (WHERE s."id" IS NOT NULL), json_array()) AS "supplierBriefs"`, + ); + }); +``` + +Match `emitColumn`'s real name and signature from the neighbouring tests in that directory. + +- [ ] **Step 2: Run to verify it fails** + +```bash +cd server/typescript/packages/codegen-ts && bun test test/projection/collect-whole-object-ddl.test.ts +``` + +Expected: FAIL — no branch handles the kind. + +- [ ] **Step 3: Implement the lowering** + +In `view-ddl-emit.ts`, directly after the `collectAgg` branch at `:187`: + +```ts + if (c.kind === "collectObjectAgg") { + const guard = `${c.sourceAlias}.${quoteIfNeeded(c.joinedPkColumn)} IS NOT NULL`; + // Element order: the related entity's PK ascending by default — "value + // ascending" is meaningless for an object and does not even parse on PG json. + // An explicit @orderBy leads, with the PK appended as tie-break so equal-order + // rows stay byte-deterministic (mirrors renderFirst at :131-137). The SCALAR + // arm deliberately keeps its no-tie-break behaviour: changing it would alter + // emitted SQL for every existing project using @orderBy. + const pk = `${c.sourceAlias}.${quoteIfNeeded(c.joinedPkColumn)}`; + const orderClause = c.orderBy.length > 0 + ? `ORDER BY ${renderOrderKeys(c.orderBy, c.sourceAlias)}, ${pk} ASC` + : `ORDER BY ${pk} ASC`; + const pairs = c.members + .map((m) => `'${m.memberName}', ${c.sourceAlias}.${quoteIfNeeded(m.sourceColumn)}`) + .join(", "); + if (dialect === "sqlite") { + return `COALESCE(json_group_array(json_object(${pairs}) ${orderClause}) FILTER (WHERE ${guard}), json_array()) AS ${alias}`; + } + return `COALESCE(jsonb_agg(jsonb_build_object(${pairs}) ${orderClause}) FILTER (WHERE ${guard}), '[]'::jsonb) AS ${alias}`; + } +``` + +In-aggregate `ORDER BY` needs SQLite **≥ 3.44**. Not a new constraint: the existing scalar `collect` already emits it and D1's baseline is pinned at `3.44.0` (`introspect/d1.ts:44`). + +- [ ] **Step 4: Run to verify it passes** + +```bash +cd server/typescript/packages/codegen-ts && bun test test/projection/collect-whole-object-ddl.test.ts +``` + +Expected: 2 pass. + +- [ ] **Step 5: Add the scalar no-churn pin** + +In the same file, add a test asserting a `collect` **with** `@of` still emits `array_agg(… ORDER BY … ) FILTER (…) , '{}'` on Postgres and `json_group_array` / `json_array()` on SQLite, byte-for-byte. The two arms share a branch neighbourhood; this is what proves the scalar one did not move. + +- [ ] **Step 6: Run the whole codegen-ts suite** + +```bash +cd server/typescript/packages/codegen-ts && bun test +``` + +Expected: 0 fail. If `issue-214-read-half-compile.test.ts` fails with *"Cannot find module `@metaobjectsdev/runtime-ts/drizzle-fastify`"*, that is a fresh-worktree precondition, not your change — run `bun run --filter '@metaobjectsdev/runtime-ts' build` from the repo root first. + +- [ ] **Step 7: Commit** + +```bash +git add server/typescript/packages/codegen-ts/src/projection/view-ddl-emit.ts \ + server/typescript/packages/codegen-ts/test/projection/collect-whole-object-ddl.test.ts +git commit -m "feat(codegen-ts): lower a whole-object rollup to jsonb_agg / json_group_array (#335)" +``` + +--- + +## Task 12: Half A — real-engine round-trip, both dialects + +**Files:** +- Modify: `server/typescript/packages/integration-tests/test/view-lifecycle-pg.test.ts` +- Modify: `server/typescript/packages/integration-tests/test/view-lifecycle-sqlite.test.ts` + +**Interfaces:** +- Consumes: Tasks 10–11. +- Produces: the only evidence that counts for new DDL. + +**Why:** Golden SQL is not acceptable evidence for new DDL — that is exactly how the migrate defect class got through before. These files live in a **separate package** that only the `ts-slow` lane runs; they will not appear in a `codegen-ts` or `migrate-ts` run. + +- [ ] **Step 1: Add the projection to both fixtures** + +Extend the existing model in each file with `SupplierBrief`, a `Supplier` child table, and a `field.object isArray @objectRef` carrying an `@of`-less collect — beside the existing `weekLabels` scalar collect, so both arms are exercised in one view. + +- [ ] **Step 2: Assert emit → apply → introspect → re-diff is EMPTY** + +Follow the existing round-trip in each file exactly. The re-diff must produce **no** operations; a non-empty second diff means the emitted SQL and the introspected schema disagree, which is permanent false drift on every subsequent `meta migrate`. + +- [ ] **Step 3: Assert the value shape, including the empty set** + +```ts + // two related rows → array of objects, PK-ascending + expect(JSON.parse(full.supplierBriefs)).toEqual([ + { id: 1, name: "Acme" }, + { id: 2, name: "Globex" }, + ]); + // zero related rows → [] and NOT null (the FILTER guard) + expect(JSON.parse(empty.supplierBriefs)).toEqual([]); +``` + +On Postgres the driver may return a parsed object rather than a string; match whatever the neighbouring `weekLabels` assertions do in that same file. + +- [ ] **Step 4: Run both** + +```bash +cd server/typescript/packages/integration-tests && bun test test/view-lifecycle-sqlite.test.ts +cd server/typescript/packages/integration-tests && bun test test/view-lifecycle-pg.test.ts +``` + +Postgres needs Docker for Testcontainers. If the container times out, re-run on a quiet box before treating it as a failure — that timeout fires before test logic and means load, not regression. **Never** point these at an existing local Postgres; print the resolved URL first and confirm it is the Testcontainers one. + +- [ ] **Step 5: Commit** + +```bash +git add server/typescript/packages/integration-tests/test/view-lifecycle-pg.test.ts \ + server/typescript/packages/integration-tests/test/view-lifecycle-sqlite.test.ts +git commit -m "test(integration): round-trip the whole-object rollup on real PG and SQLite (#335)" +``` + +--- + +## Task 13: Half A — restore the corpus coverage the retirement lost + +**Files:** +- Modify: `fixtures/conformance/flattened-kitchen-sink/input/meta.catalog.json` +- Modify: `fixtures/conformance/flattened-kitchen-sink/expected.json` +- Modify: `fixtures/conformance/flattened-kitchen-sink/expected-effective.json` +- Modify: `fixtures/conformance/README.md` +- Modify: `docs/features/migrations/origin-collection-retirement.md` + +**Interfaces:** +- Consumes: Tasks 5–8. +- Produces: closes the documented coverage gap. + +**Why:** `fixtures/conformance/README.md:102` records this as *"coverage genuinely lost"* — `flattened-kitchen-sink` dropped `supplierBriefs`, the corpus's only array-of-value-object carrying an origin — and names #335 as the restore path. + +- [ ] **Step 1: Restore `supplierBriefs`, expressed the new way** + +```jsonc +{ "field.object": { + "name": "supplierBriefs", "isArray": true, + "@objectRef": "acme::catalog::SupplierBrief", + "children": [ + { "origin.aggregate": { "@agg": "collect", "@via": "acme::catalog::Product.suppliers" } } + ] +}} +``` + +- [ ] **Step 2: Regenerate both goldens** + +Run the corpus and copy the canonical output from the mismatch report. Do not hand-edit. + +- [ ] **Step 3: Update the README's coverage table** + +The "Coverage genuinely lost" entry is no longer true. Replace it with a line saying the shape is expressible again via `@of`-less collect and naming this fixture as where it is covered. + +- [ ] **Step 4: Fix the retirement guide's silent case** + +`docs/features/migrations/origin-collection-retirement.md:62-65` tells authors to *"delete the `origin.collection` child and change nothing else"*. That is correct for a **payload** host but wrong for a **view-kind projection**, where a no-origin field lowers to `SELECT base."supplierBriefs"` against a column that does not exist (`extract-view-spec.ts:817-826`). Add a short subsection: on a projection host, replace the child with `origin.aggregate @agg: collect @via: …` rather than deleting it. + +- [ ] **Step 5: Run the corpus in every port** + +```bash +cd server/typescript/packages/metadata && bun test test/conformance.test.ts +cd server/python && uv run pytest --extra integration -k conformance +cd server/csharp && dotnet build && dotnet test --filter "FullyQualifiedName~Conformance" +cd server/java && mvn -q -pl metadata test +``` + +Expected: all green. + +- [ ] **Step 6: Commit** + +```bash +git add fixtures/conformance/flattened-kitchen-sink fixtures/conformance/README.md \ + docs/features/migrations/origin-collection-retirement.md +git commit -m "test(conformance): restore the array-of-value-object origin coverage (#335)" +``` + +--- + +## Task 14: Documentation and changelog + +**Files:** +- Modify: `CHANGELOG.md` +- Modify: `docs/features/downstream-metadata-decisions.md` +- Modify: `agent-context/skills/metaobjects-authoring/SKILL.md` **and its 5 byte-gated copies** under `fixtures/agent-context-conformance/*/expected/.claude/skills/metaobjects-authoring/SKILL.md` + +**Interfaces:** +- Consumes: everything. +- Produces: the adopter-facing record. + +**Why the skill matters:** the skill is what teaches adopters the vocabulary. #342 shipped with the skill teaching a form the loader had just made illegal — do not repeat that in the other direction by leaving the skill unable to express the new one. + +- [ ] **Step 1: Add the changelog entry** + +Cover, in this order: the whole-object rollup with a worked example; **Half B as potentially breaking**, stated with its bounds — in-repo cost measured at zero, what breaks was already emitting SQL that cannot execute, and no documentation ever taught the form; the `@distinct` refusal as a deliberate choice rather than an engine limit; and the `metamodelVersion` position. + +- [ ] **Step 2: Add the authoring-skill section** + +Document `@of`-less collect beside the existing scalar `collect`: the `field.object isArray @objectRef` requirement, `@via` required, `@objectRef` must be an `object.value`, members must exist on the terminal entity with matching types, `@distinct` unsupported, default order PK-ascending. + +- [ ] **Step 3: Regenerate the byte-gated skill copies** + +```bash +grep -rn "regenerate" fixtures/agent-context-conformance/README.md +``` + +Run the generator named there. Do not hand-edit the five copies. + +- [ ] **Step 4: Verify the agent-context corpus** + +```bash +cd server/typescript/packages/cli && bun test +``` + +Expected: `agent-context-conformance` green. + +- [ ] **Step 5: Commit** + +```bash +git add CHANGELOG.md docs/features/downstream-metadata-decisions.md \ + agent-context/skills/metaobjects-authoring/SKILL.md \ + fixtures/agent-context-conformance +git commit -m "docs: whole-object rollup and the array filter/sort rules (#335)" +``` + +--- + +## Task 15: Full-gate verification before merge + +- [ ] **Step 1: Clean-tree check** + +```bash +git clean -ndX # review what would be removed +git clean -fdX && bun install +``` + +The dev box is warm; a lane change validated on a warm tree proves nothing. + +- [ ] **Step 2: Run the affected ports** + +```bash +bash scripts/ci-local.sh --only ts +bash scripts/ci-local.sh --only python +bash scripts/ci-local.sh --only csharp +bash scripts/ci-local.sh --only java +``` + +**Never pipe these through `tail`** — the exit status becomes `tail`'s and a red run reports green. Read the trailing `OK —` / `FAILED —` line directly. + +- [ ] **Step 3: Confirm the scalar arm never moved** + +```bash +git diff v0.24.0 --stat -- server/typescript/packages/codegen-ts/test/golden/ +``` + +Expected: no churn attributable to the scalar `collect` path. Any golden that moved must be explained before merge. + +- [ ] **Step 4: Leak scan** + +```bash +bash scripts/ci-local.sh --quick +``` + +- [ ] **Step 5: Re-check the version question** + +```bash +git tag --list 'v0.*' --sort=-v:refname | head -1 +node scripts/check-metamodel-version.mjs --explain +``` + +If `v0.24.1` shipped while this was in flight, `--set 0.12` and commit before merging. + +- [ ] **Step 6: Coordinate before pushing** + +Another session owns `validation-passes.ts` for #342 and has committed there. `git fetch origin` and rebase before pushing; message that session before landing. + +--- + +## Self-review + +**Spec coverage.** A1 → Tasks 5, 8. A2 → Task 6. A3 → Task 10. A4 → Task 11. A5 → Tasks 5 (`@distinct`), 11 (order). A6 → Task 9. A7 → Tasks 9 Step 5, 15 Step 5. B1 → Tasks 1, 4. B2 → Tasks 2, 4. B3 → Task 14 Step 1. Testing section → Tasks 3, 7, 12, 13. Non-goals are not implemented, by design. + +**Placeholders.** None. Every code step carries real code; every command is runnable. + +**Type consistency.** `collectObjectAgg` and its `members: { memberName, sourceColumn }[]` are defined in Task 10 and consumed unchanged in Tasks 11–12. `_viaTerminalEntityNode` is introduced in Task 5 and reused in Task 6. `ERR_COLLECT_MEMBER_UNRESOLVED` and `ERR_SORTABLE_UNSUPPORTED_SUBTYPE` are added to all five ledgers in the tasks that first emit them. + +**Known soft spots**, flagged rather than hidden: the loader's string entry point is written as `loadFromString`, and the DDL test helper as `emitColumn` — both must be matched to the real names in neighbouring tests. Task 9's exact seven files must come from the grep, not from the list, in case a copy moved. diff --git a/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md b/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md new file mode 100644 index 000000000..72d52efee --- /dev/null +++ b/docs/superpowers/specs/2026-08-22-issue-335-whole-object-rollup-design.md @@ -0,0 +1,377 @@ +# Whole-object rollup on `origin.aggregate @agg: collect` — design + +**Issue:** [#335](https://github.com/metaobjectsdev/metaobjects/issues/335) +**Date:** 2026-08-22 +**Status:** proposed +**Baseline:** `9eede9dd9` + +## Summary + +Make `@of` **optional** on `origin.aggregate @agg: collect`. Absent means a **whole-object +rollup**: collect the related rows as a JSON array of the field's declared `@objectRef` +value-object, rather than an array of one scalar column. + +```jsonc +{ "field.object": { + "name": "supplierBriefs", + "isArray": true, + "@objectRef": "acme::common::SupplierBrief", + "children": [ + { "origin.aggregate": { "@agg": "collect", "@via": "acme::catalog::Product.suppliers" } } + ] +}} +``` + +The work ships in **two halves**. Half A is the feature. Half B closes two holes in the +queryable-projection contract that Half A would otherwise land a column into. + +## Why two halves + +A projection is a **queryable relational surface**, not merely a wire read-shape. That is the +project's intent, and it is already true in code: any object with a readable source gets +`renderFilterAllowlist` / `renderSortAllowlist` and filtered read routes +(`templates/entity-file.ts:171`). + +A whole-object rollup column is a deliberate **leaf** inside that surface — non-filterable by +construction, like a `jsonb` column in any hand-written view. The loader already enforces the +leaf-ness (`@filterable` on `field.object` is `ERR_FILTERABLE_UNSUPPORTED_SUBTYPE`, +`validation-passes.ts:436-455`). + +But the surface contract has two holes today, **independent of this feature**: + +1. **The filter allowlist is not `isArray`-aware.** `filterSubTypeFor` + (`templates/filter-allowlist.ts:38-44`) falls through to `"string"` for any unrecognised + subtype, and nothing in that file consults `isArray` at all. So + `field.string isArray: true @filterable: true` already emits a `like`/`eq` rule against a + `text[]` column. This is live: `weekLabels` is a shipping array projection column. +2. **There is no `@sortable` subtype validation anywhere in the loader** — versus a hard error + for `@filterable`. `@sortable: true` on a JSON or array column passes the loader and emits a + sort entry. + +Shipping Half A without Half B adds a second non-scalar column shape to a query tier that +already mishandles the first. + +## Decision: no new subtype and no new `@kind` + +Considered and rejected: distinguishing a "document projection" from a "relational projection" +in the type system. + +- **ADR-0037's subtype test fails.** A subtype needs its own native type, behaviour, or + attributes. Both forms are read-only views, with the same source `@kind`s and the same + generated artifacts. The only difference is whether one column is non-scalar. +- **`@kind` is already chartered for something else.** On `source.rdb` it names *which physical + relational object backs this* — `table | view | materializedView | storedProc | + tableFunction` — and read-only-ness is **derived** from it (`spec/metamodel/source.json`). A + `documentView` member would be a category error and would break that derivation. +- **The query contract is already per-FIELD**, which is the correct level, because that is + where the truth lives. A projection carrying one nested column stays a queryable surface on + every other column. + +**Consequence recorded deliberately:** #210 made `object.projection` the only legal host for +assembly origins, so "assemble a document" has no other home in the metamodel. If a shipping +consumer ever needs a read shape that is *not* a queryable surface, that earns its own type +then — the ADR-0007 Amendment 2 bar, the same discipline applied to `origin.collection`. Not +now, and not on speculation. + +--- + +# Half A — the whole-object rollup + +## A1. Loader (all five ports) + +Split `collect` out of the `@of`-required gate at `validation-passes.ts:1252`. + +**`@of` PRESENT** — unchanged, including the element-type-preserving check at `:1265-1268`. + +**`@of` ABSENT** — the carrying field must be: + +| Requirement | Error | Already enforced? | +|---|---|---| +| `field.object` with `@objectRef` | `ERR_INVALID_ORIGIN` | new | +| `isArray: true` | `ERR_INVALID_ORIGIN` | yes — `:1192-1197` | +| `@objectRef` targets an `object.value` | `ERR_SUBTYPE_RULE_VIOLATION` | **new — see below** | +| `@via` present | `ERR_INVALID_ORIGIN` | new (no `@of` to infer from) | +| at least one to-many `@via` hop | `ERR_ORIGIN_CARDINALITY` | yes — `_checkAggregateCardinality` | +| `@distinct` absent | `ERR_INVALID_ORIGIN` | new — see A5 | + +**The value-only rule is NOT inherited from #210.** `_checkNestedPayloadRefsValueOnly` +(`validation-passes.ts:213`) walks only from **template-level payload targets** — its own +comment scopes it to "every `field.object @objectRef` reachable from a template-level payload +target". A projection-hosted `field.object` never reaches that pass, so nothing today stops a +projection field's `@objectRef` from resolving to an `object.entity`. This branch must enforce +it itself, reusing `ERR_SUBTYPE_RULE_VIOLATION` for consistency with the payload-side message. +Without it, an `@objectRef` to an entity reintroduces the #270 shape — a curated value silently +becoming the full entity — in DDL. + +**Element type is declared-authoritative (#270).** The element type comes from the declared +`@objectRef`, never from the `@via` relationship's target entity. This is the exact bug #270 +deleted; the rule is restated here because the new branch is where it could be reintroduced. + +**`@orderBy` re-points.** `_validateOrderByKeys` currently resolves keys against +`ofTarget?.entity` (`:1271`). With `@of` absent there is no `ofTarget`, so keys must resolve +against the `@via` **terminal** entity. + +**Multi-hop `@via` is legal.** `viaTerminalEntity` already walks every segment +(`extract-view-spec.ts:477-493`), and the `any`/`all` branch already permits multi-hop. No new +restriction — consistency with the sibling branch is the reason. + +## A2. Member resolution — the #270 guard + +The lowering must project **exactly the declared value-object's members**, matched **by name** +against the `@via` terminal entity's effective fields. + +- A VO member with **no matching field** on the terminal entity → **error**, new code + `ERR_COLLECT_MEMBER_UNRESOLVED`. A new code must be added to the shared ledger and every + registry that gates it — TS `errors.ts` (exact-bidirectional), Python `errors.py` (superset), + Java `ErrorCode.java` — in the same change, or the ledger tests fail. +- A VO member whose `field.` **differs** from the matched field's → **error**, reusing + `ERR_INVALID_ORIGIN`, which is what the scalar element-type check at `:1265-1268` already + emits. This is the object-form analogue of that check, carrying the same #185 + type-preserving doctrine per member. + +Failing open here — silently dropping or substituting members — is precisely how #270 turned a +declared curated value-object into the full entity. The error is the point. + +## A3. Codegen (TypeScript only — ADR-0015) + +`extract-view-spec.ts`: today `if (!of_) continue;` (`:891`) sits **above** the `AGG_COLLECT` +branch (`:902`), and the aggregated entity is resolved *from* `@of`. Restructure so the +object arm resolves from `@via` instead. + +**Mirror the `any`/`all` branch** (`:861-885`) — it is the same shape and already proven: +`viaTerminalEntity(via, …)` → `resolveEntityRef` → `findAliasInTree` → `primaryKeyColumn`. + +New column kind `collectObjectAgg` in `view-spec.ts`, carrying the resolved member list +(`{ memberName, sourceColumn }[]`) plus `sourceAlias`, `joinedPkColumn`, `orderBy`. + +Kept as a **separate kind** rather than an arm of `collectAgg`: the payloads differ (a member +list vs a single source column) and the existing kind is consumed by +`viewOrderKeysAreDeterministic` / the real-aggregate predicate at `:1005` and `:1021`, which a +union type would force every consumer to re-narrow. + +## A4. DDL lowering (`view-ddl-emit.ts`) + +Preserves the existing empty-set guard and ordering discipline. + +**Postgres — `jsonb`, not `json`.** Verified against PostgreSQL 15.15: `json` has neither an +equality nor an ordering operator (`ERROR: operator does not exist: json = json`; +`could not identify an ordering operator for type json`), so the `json_agg(json_build_object(…) +ORDER BY …)` form named in the issue **does not run**. `jsonb` has both, and `field.object` is +already `jsonb` in `column-mapper.ts:610-613`. + +```sql +COALESCE( + jsonb_agg(jsonb_build_object('id', s."id", 'name', s."name") ORDER BY s."id" ASC) + FILTER (WHERE s."id" IS NOT NULL), + '[]'::jsonb +) AS "supplierBriefs" +``` + +**SQLite** — verified on 3.53.0: nested objects propagate (the JSON subtype carries), and +in-aggregate `ORDER BY` and the `FILTER` clause both work. + +```sql +COALESCE( + json_group_array(json_object('id', s."id", 'name', s."name") ORDER BY s."id" ASC) + FILTER (WHERE s."id" IS NOT NULL), + json_array() +) AS "supplierBriefs" +``` + +In-aggregate `ORDER BY` requires SQLite **≥ 3.44**. This is **not a new constraint** — the +existing scalar `collect` already emits it, and D1's baseline is pinned at `3.44.0` +(`introspect/d1.ts:44`). + +**Empty default differs by arm**: the scalar arm keeps `'{}'` (a PG array); the object arm uses +`'[]'::jsonb`. The scalar arm's output must stay byte-identical. + +## A5. Element order and dedupe + +**Default order: the related entity's primary key, ascending.** "Value-ascending" does not +transfer from scalars — ordering rows by a serialized object is meaningless, and on PG `json` +it does not even parse. PK-ascending is deterministic and already the in-emitter precedent: +`renderFirst` appends `childAlias.childPk ASC` as its tie-break (`view-ddl-emit.ts:131-137`). + +**`@orderBy` is supported**, resolving against the `@via` terminal entity, with the PK appended +as tie-break so equal-order rows stay byte-deterministic. + +> **This diverges from the scalar arm, deliberately.** Today a scalar `collect` with `@orderBy` +> emits *only* those keys (`view-ddl-emit.ts:193-196`) — no tie-break — so two rows equal under +> the keys have engine-chosen order. That is tolerable for scalars and not for objects, where +> the value itself cannot break the tie (PG `json` has no ordering operator at all). Appending +> the PK matches `renderFirst`, which already does exactly this. The scalar arm is **not** +> changed: doing so would alter existing emitted SQL for every project using `@orderBy`. + +**`@distinct` is FORBIDDEN — by choice, not by engine limit.** State this honestly: it was +verified to work on both engines (SQLite 3.53 `json_group_array(DISTINCT json_object(…))` +dedupes correctly; PG `jsonb_agg(DISTINCT …)` works). It is forbidden because it is a +guaranteed no-op whenever the value-object carries the entity's primary key, which is the +common case, and a silent no-op is worse than a refusal. Re-entry path if an adopter presents a +real case: allow `@distinct` **with a mandatory `@orderBy`**, inverting today's mutual +exclusion. + +## A6. Registry prose — byte-gated, seven files + +`@of` is already `"required": false` structurally, so no `required` flip. But its `description` +reads *"Required for count/sum/avg/min/max/collect; …"* and that string is byte-gated in seven +places across all five ports: + +``` +fixtures/registry-conformance/expected-registry.json +fixtures/metamodel-docs/expected/types/origin.md +spec/metamodel/origin.json +server/csharp/MetaObjects/Persistence/Origin/OriginSchema.cs +server/csharp/MetaObjects/SpecMetamodel/origin.json +server/python/src/metaobjects/spec_metamodel/origin.json +server/typescript/packages/metadata/src/persistence/origin/origin-definition.embedded.ts +``` + +All seven change in lockstep or `registry-conformance` goes red in every port. + +## A7. `metamodelVersion`: `0.12` — SETTLED and SHIPPED + +Classification: **additive**. Previously-invalid metadata becomes valid; nothing that loaded +before stops loading. This is #210 in reverse — a rule change whose only machine-readable +footprint is prose, which `check-metamodel-version.mjs` can only **WARN** about (`:36`), so no +gate forces the call and it is made by a human. + +**The number is `0.12`, set by `7c99536a7` via `check-metamodel-version.mjs --set 0.12`** (all +five sites at once: the manifest plus the four port constants). + +This section previously said the change rode `#342`'s unreleased `0.11` and needed no edit. That +reasoning held only while `0.11` was unclaimed, and it stopped holding during implementation: +`origin/main`'s `2af77e7cd` names **`0.24.1` as the sole claimant of `0.11`**. The re-check this +section itself demanded — *"if `0.11` ships in a release before this lands, this needs `0.12`"* — +therefore fired. Doug ruled `0.12` on 2026-08-23. + +Worth keeping, because a reviewer re-derived the superseded answer and called the `0.11` ride +correct: "`v0.24.0` shipped `0.10` and main already carries the unreleased bump" is true and +still gives the wrong answer, because it never asks *who else has already claimed `0.11`*. The +gate baselines against the last release TAG, so an unreleased bump looks free right up until +another change in the same cycle spends it. + +One condition remains open: if Half B is judged breaking rather than additive (see B2's note), +the classification — though not necessarily the number, pre-1.0 — changes with it. + +--- + +# Half B — the queryable-surface contract + +## B1. `isArray`-aware filter band + +`filterSubTypeFor` (`templates/filter-allowlist.ts:38-44`) maps subtype → category and falls +through to `"string"`; nothing consults `isArray`. + +**Rule:** an array-typed field is **not filterable** under the FR-009 scalar operator band. A +`field. isArray: true` carrying `@filterable: true` becomes a **load error**, reusing +`ERR_FILTERABLE_UNSUPPORTED_SUBTYPE` — the same door `field.object` already goes through, since +the reason is identical (no operator in the band applies). + +Fail at load, not by filtering the field out of the allowlist: silently dropping a declared +`@filterable` is the same failure mode as the pre-#292 unchecked snapshot — the author believes +something is enforced that is not. + +## B2. `@sortable` subtype validation + +`@sortable` defaults from `@filterable`, so today it is only independently set when explicit — +and nothing validates it. Add the subtype/array check that `@filterable` has, so +`@sortable: true` on a JSON or array column is a load error rather than a generated sort over a +JSON column. + +### B3. How breaking is this, precisely + +B1 and B2 make previously-loading metadata fail to load, so they are breaking on the metadata +contract. Three things bound it, and the bound should be stated rather than the label: + +1. **In-repo cost is zero.** A structural scan (per field node) of 1321 JSON and 124 YAML files + found **no** field carrying both `isArray: true` and `@filterable`/`@sortable: true`. Nothing + here breaks. +2. **What breaks was already broken.** Such a declaration emits a `like`/`eq` rule against a + `text[]` column — SQL that cannot execute. This converts a runtime failure into a load + failure, which is the direction this project already treats as a fix rather than a + regression. +3. **There is no mechanism by which an adopter was taught to write it.** This is the distinction + that matters, and it is worth stating because a sibling change in the same release window is + the *same category* and a **different magnitude**: #342's retired `@fields` + `@expr` + spelling appeared as a worked example in a shipped skill, so there is a plausible route by + which adopters acquired it. The array-`@filterable` form was never documented, never + exemplified, and never emitted by any generator — it is a silently-wrong emission nobody was + told to write. + +So: same category (breaking only for metadata outside this repo), materially lower exposure. +Changelog must still call it out — an adopter who wrote it independently gets a load error with +no warning — but the "we handed out the recipe" argument does not apply here. + +**Do not bundle this decision with another release's.** The version question for #335 is asked +when #335 is cut, on this evidence. Folding it into a ruling already made about a different +change would be deciding it on the wrong facts. + +--- + +## Testing + +**The corpus must gain the case, not just the code.** `fixtures/conformance/README.md:102` +already records `flattened-kitchen-sink`'s dropped `supplierBriefs` as *"coverage genuinely +lost"* and names #335 as the restore path. Restore it there, expressed the new way. + +| Layer | What | +|---|---| +| Conformance (5 ports) | `@of`-absent collect loads on a `field.object @objectRef isArray`; each error arm fails: no `@objectRef`, non-value target, missing `@via`, all-to-one `@via`, unresolved member, member type mismatch | +| No-churn | a `collect` **with** `@of` emits byte-identical SQL — pinned, since the scalar arm shares the branch | +| Real engine, both dialects | emit → apply → introspect → **re-diff must be empty**, then read rows back and assert the array-of-objects shape, including the empty-set case returning `[]` not `null` | +| Half B | a `@filterable` array field fails to load; a `@sortable` array field fails to load — **as shared `fixtures/conformance/` cases, not port-local unit tests** | + +**Every new load error needs a corpus fixture that TRIGGERS it, in the shared corpus.** A rule +with no negative fixture is the same blind spot one layer up: "no fixture covers it" and "every +port enforces it" are indistinguishable from a green suite. Three separate findings in #342 were +this shape — a BOTH-fixture that covered `index.lookup` but not `identity.secondary`, which is +the exact arm where the JVM had diverged; and `@expr` built and shipped end-to-end with no +fixture ever declaring one, so one port was free to refuse it for months. + +A port-local unit test does not substitute. It proves *one* port enforces the rule, which is +precisely the assertion that was true-and-insufficient in every one of those cases. This +matters more than usual here because Half B's in-repo blast radius is **zero** — a structural +scan of 1321 JSON and 124 YAML files found no field carrying both `isArray: true` and +`@filterable`/`@sortable: true`. Nothing in the corpus exercises an array field through the +filter tier at all, which is why the tier has been free to be wrong about it since it shipped. +The fixture is the only thing that changes that. + +**The round-trip tests live in a separate package.** `view-lifecycle-{pg,sqlite}.test.ts` are in +`server/typescript/packages/integration-tests`, which only the `ts-slow` lane runs — they will +not appear in a `codegen-ts` or `migrate-ts` run. Golden SQL alone is not acceptable evidence +for new DDL; that is how the migrate defect class got through before. + +**SQLite byte-stability binds.** `view-fingerprint.ts` explains that Postgres never compares +against the deparser (it hashes the body we emit), **but that "SQLite/D1 need none of this: +`sqlite_master.sql` is the verbatim text we wrote, so the body comparator is exact there."** So +the emitted SQLite expression must be deterministic or every migrate reports drift. + +## Non-goals + +- Non-RDB lowerings — #211's capability matrix. +- Filtering a parent by an embedded child's property. This is the capability a queryable + surface would eventually want, `json_agg` cannot serve it, and it is a separate design. +- Exposing `ObjectManager.attachIncludes` as a generated endpoint. It is wired into the + ObjectManager read path (`object-manager.ts:128,140`) but **no generated endpoint surfaces + it** and it is TS-only. If that is ever built, it is the app-side-merge counterpart to this + view-side join — the two are alternatives, and this design does not foreclose it. + +## Evidence base + +Claims here were verified at `9eede9dd9` rather than carried from the issue text. Four of the +issue's own statements did not survive: + +1. **"No registry change"** — structurally true, but the byte-gated `description` prose must + change in seven files (A6). +2. **The proposed `json_agg(json_build_object(…) ORDER BY …)` does not run on Postgres** — + verified against a real engine (A4). +3. **"One relaxed rule"** understates it — member resolution, `@orderBy` re-pointing, a new + column kind, and a changed empty default all follow. +4. **"`@objectRef` must target an `object.value`, which is already a loader rule from #210"** — + it is not. That rule is payload-scoped and never reaches a projection-hosted field (A1). + This is the most consequential of the four: relying on it would have shipped the branch with + its main #270 guard simply absent. + +Two claims from review that also did not survive checking: this is **not** the first non-scalar +projection column (`weekLabels` ships today, asserted against real PG and SQLite), and +`@distinct` is **not** blocked by either engine. diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md index 98b16b4c5..1dbb1fe1d 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -838,8 +838,16 @@ amendment 2026-08-06.) **Origin vocabulary (#195).** `origin.aggregate @agg` takes `count`/`sum`/`avg`/`min`/`max` (numeric reduces over `@of`), `any`/`all` (predicate quantifiers over a `@filter`; `@of` -forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup of `@of` -into an `isArray` field, with optional `@distinct` / `@orderBy`). Any aggregate may be +forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup +into an `isArray` field, with optional `@distinct` / `@orderBy`). **`collect` is the one +`@agg` where `@of` is OPTIONAL (#335):** name a column with `@of` to collect scalars, or +omit `@of` on a `field.object @objectRef` to collect each related row as that declared +value object — a **whole-object rollup**, lowered to `jsonb_agg(jsonb_build_object(…))` +on Postgres. The whole-object form requires an explicit `@via`, refuses `@distinct` (it is +a no-op whenever the value object carries the primary key), and requires every value-object +member to match a field on the `@via` **terminal** entity by name, with the same subtype +and array-ness. The declared value object IS the exposure: a field the entity has and the +value object omits is not projected. Any aggregate may be row-scoped with `@filter`. `origin.computed` carries a closed structured `@expr` tree (a derived scalar). `origin.first` picks one related row's column (`@of`) along `@via`, ordered by a **required `@orderBy`** (`["field:asc|desc", …]`, with the PK as tie-break) — diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md index 98b16b4c5..1dbb1fe1d 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -838,8 +838,16 @@ amendment 2026-08-06.) **Origin vocabulary (#195).** `origin.aggregate @agg` takes `count`/`sum`/`avg`/`min`/`max` (numeric reduces over `@of`), `any`/`all` (predicate quantifiers over a `@filter`; `@of` -forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup of `@of` -into an `isArray` field, with optional `@distinct` / `@orderBy`). Any aggregate may be +forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup +into an `isArray` field, with optional `@distinct` / `@orderBy`). **`collect` is the one +`@agg` where `@of` is OPTIONAL (#335):** name a column with `@of` to collect scalars, or +omit `@of` on a `field.object @objectRef` to collect each related row as that declared +value object — a **whole-object rollup**, lowered to `jsonb_agg(jsonb_build_object(…))` +on Postgres. The whole-object form requires an explicit `@via`, refuses `@distinct` (it is +a no-op whenever the value object carries the primary key), and requires every value-object +member to match a field on the `@via` **terminal** entity by name, with the same subtype +and array-ness. The declared value object IS the exposure: a field the entity has and the +value object omits is not projected. Any aggregate may be row-scoped with `@filter`. `origin.computed` carries a closed structured `@expr` tree (a derived scalar). `origin.first` picks one related row's column (`@of`) along `@via`, ordered by a **required `@orderBy`** (`["field:asc|desc", …]`, with the PK as tie-break) — diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md index 98b16b4c5..1dbb1fe1d 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -838,8 +838,16 @@ amendment 2026-08-06.) **Origin vocabulary (#195).** `origin.aggregate @agg` takes `count`/`sum`/`avg`/`min`/`max` (numeric reduces over `@of`), `any`/`all` (predicate quantifiers over a `@filter`; `@of` -forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup of `@of` -into an `isArray` field, with optional `@distinct` / `@orderBy`). Any aggregate may be +forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup +into an `isArray` field, with optional `@distinct` / `@orderBy`). **`collect` is the one +`@agg` where `@of` is OPTIONAL (#335):** name a column with `@of` to collect scalars, or +omit `@of` on a `field.object @objectRef` to collect each related row as that declared +value object — a **whole-object rollup**, lowered to `jsonb_agg(jsonb_build_object(…))` +on Postgres. The whole-object form requires an explicit `@via`, refuses `@distinct` (it is +a no-op whenever the value object carries the primary key), and requires every value-object +member to match a field on the `@via` **terminal** entity by name, with the same subtype +and array-ness. The declared value object IS the exposure: a field the entity has and the +value object omits is not projected. Any aggregate may be row-scoped with `@filter`. `origin.computed` carries a closed structured `@expr` tree (a derived scalar). `origin.first` picks one related row's column (`@of`) along `@via`, ordered by a **required `@orderBy`** (`["field:asc|desc", …]`, with the PK as tie-break) — diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md index 98b16b4c5..1dbb1fe1d 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -838,8 +838,16 @@ amendment 2026-08-06.) **Origin vocabulary (#195).** `origin.aggregate @agg` takes `count`/`sum`/`avg`/`min`/`max` (numeric reduces over `@of`), `any`/`all` (predicate quantifiers over a `@filter`; `@of` -forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup of `@of` -into an `isArray` field, with optional `@distinct` / `@orderBy`). Any aggregate may be +forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup +into an `isArray` field, with optional `@distinct` / `@orderBy`). **`collect` is the one +`@agg` where `@of` is OPTIONAL (#335):** name a column with `@of` to collect scalars, or +omit `@of` on a `field.object @objectRef` to collect each related row as that declared +value object — a **whole-object rollup**, lowered to `jsonb_agg(jsonb_build_object(…))` +on Postgres. The whole-object form requires an explicit `@via`, refuses `@distinct` (it is +a no-op whenever the value object carries the primary key), and requires every value-object +member to match a field on the `@via` **terminal** entity by name, with the same subtype +and array-ness. The declared value object IS the exposure: a field the entity has and the +value object omits is not projected. Any aggregate may be row-scoped with `@filter`. `origin.computed` carries a closed structured `@expr` tree (a derived scalar). `origin.first` picks one related row's column (`@of`) along `@via`, ordered by a **required `@orderBy`** (`["field:asc|desc", …]`, with the PK as tie-break) — diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md index 98b16b4c5..1dbb1fe1d 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -838,8 +838,16 @@ amendment 2026-08-06.) **Origin vocabulary (#195).** `origin.aggregate @agg` takes `count`/`sum`/`avg`/`min`/`max` (numeric reduces over `@of`), `any`/`all` (predicate quantifiers over a `@filter`; `@of` -forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup of `@of` -into an `isArray` field, with optional `@distinct` / `@orderBy`). Any aggregate may be +forbidden; empty set → `any=false`, `all=true`), and `collect` (an array rollup +into an `isArray` field, with optional `@distinct` / `@orderBy`). **`collect` is the one +`@agg` where `@of` is OPTIONAL (#335):** name a column with `@of` to collect scalars, or +omit `@of` on a `field.object @objectRef` to collect each related row as that declared +value object — a **whole-object rollup**, lowered to `jsonb_agg(jsonb_build_object(…))` +on Postgres. The whole-object form requires an explicit `@via`, refuses `@distinct` (it is +a no-op whenever the value object carries the primary key), and requires every value-object +member to match a field on the `@via` **terminal** entity by name, with the same subtype +and array-ness. The declared value object IS the exposure: a field the entity has and the +value object omits is not projected. Any aggregate may be row-scoped with `@filter`. `origin.computed` carries a closed structured `@expr` tree (a derived scalar). `origin.first` picks one related row's column (`@of`) along `@via`, ordered by a **required `@orderBy`** (`["field:asc|desc", …]`, with the PK as tie-break) — diff --git a/fixtures/conformance/ERROR-CODES.json b/fixtures/conformance/ERROR-CODES.json index 2ec7f79c9..410b0e94e 100644 --- a/fixtures/conformance/ERROR-CODES.json +++ b/fixtures/conformance/ERROR-CODES.json @@ -69,7 +69,10 @@ "ERR_DISCRIMINATOR_VALUE_TYPE_MISMATCH": "FR-014: @discriminatorValue is not a member of the discriminator field's @values (when the discriminator field is field.enum) or cannot be coerced to its subtype.", "ERR_YAML_COERCION": "A YAML 1.2 silent type coercion produced a value whose runtime type differs from the attribute's declared valueType (e.g. an unquoted `column: TRUE` parsed as boolean for a string-typed attr). Emitted by every port's YAML loader; canonical JSON is unaffected.", "ERR_MERGE_CONFLICT": "FR5c: multi-file overlay merge produced an attribute conflict \u2014 two contributing files declared the same @attr with different non-empty values on the same logical node. Envelope is `format: \"merged\"` with `contributors[]` naming both files.", - "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE": "SP-H Unit9: a field carries @filterable: true but its subtype has no filter-operator band (e.g. field.object). Generating a filter for it would emit an empty operator set \u2014 a filter route that rejects every request. Use a filterable subtype (string/enum/uuid/number/currency/date/boolean) or remove @filterable.", + "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE": "#335 Half B / SP-H Unit9: a field carries @filterable: true but cannot be filtered — either it is an array (isArray: true; a collection column has no filter-operator band) or its subtype has no filter-operator band (e.g. field.object), the same subtype-support signal @sortable uses. Generating a filter for it would emit an empty operator set — a filter route that rejects every request. Use a filterable subtype (string/enum/uuid/number/currency/date/boolean), drop isArray, or remove @filterable.", + "ERR_SORTABLE_UNSUPPORTED_SUBTYPE": "#335 Half B: a field carries @sortable: true but cannot be ordered \u2014 either it is an array (isArray: true; a collection column has no ordering) or its subtype has no filter-operator band (e.g. field.object), the same subtype-support signal @filterable uses (SP-H Unit9). Generating a sort entry for it would emit an ORDER BY no dialect can execute. Use a sortable subtype (string/enum/uuid/number/currency/date/boolean) or remove @sortable.", + "ERR_COLLECT_WHOLE_OBJECT": "#335 Half A: a whole-object origin.aggregate @agg:collect \u2014 one with no @of, which rolls the related rows up as the carrying field's declared @objectRef value object rather than as one scalar column \u2014 is malformed. Raised when the carrying field is not a field.object declaring @objectRef; when @via is absent (there is no @of entity to infer the relationship from); when @distinct is declared (refused \u2014 a guaranteed no-op whenever the value object carries the primary key, and a silent no-op is worse than a refusal); when an @orderBy key does not resolve against the @via TERMINAL entity; or when a value-object member's declared field.[] disagrees with the matched terminal field's. Distinct from ERR_INVALID_ORIGIN so a conformance fixture can tell \"the whole-object arm rejected this\" from \"a loader that still requires @of rejected this\" \u2014 indistinguishable otherwise, since the corpus compares only code + source envelope.", + "ERR_COLLECT_MEMBER_UNRESOLVED": "#335 Half A: a whole-object origin.aggregate @agg:collect (no @of \u2014 the carrying field.object collects related rows as its declared @objectRef value object) has a value-object member with no matching field (by name) on the @via terminal entity. The lowering projects exactly the declared members; failing open here is how #270 turned a curated value object into the full entity, invisible in a diff because the metadata still read as curated.", "ERR_REGISTRY_SEALED": "ADR-0023: a registration (register / registerType / extend / registerCommonAttrs / setDefaultSubType / addConstraint / registerProviders) was attempted against a registry sealed after its agreed metamodel-provider bootstrap. Made-up metamodel attributes/types are structurally disallowed in the library: codegen cannot register post-bootstrap. A new metamodel attribute requires a registered provider + human agreement. Downstream apps that need extra vocabulary compose their own (unsealed) registry.", "ERR_UNKNOWN": "An internal loader error with no stable error code.", "ERR_ENTITY_PRIMARY_SOURCE_READONLY": "FR-024 (ADR-0028) hard cutover: an object.entity's PRIMARY source has a read-only @kind (view/materializedView/storedProc/tableFunction). Read-only kinds are legal only in non-primary roles; a derived read model is an object.projection.", diff --git a/fixtures/conformance/README.md b/fixtures/conformance/README.md index 0f738b714..0eaecd03a 100644 --- a/fixtures/conformance/README.md +++ b/fixtures/conformance/README.md @@ -80,6 +80,63 @@ API-surface checks; a fixture may have both. Fixtures assert error *codes*, never message prose. Codes are registered in `ERROR-CODES.json`. Adding a code is an additive edit to that file. +## Array fields through the filter/sort tier (#335) + +- `error-filterable-array-field` — a `field.string @isArray @filterable` fails + `ERR_FILTERABLE_UNSUPPORTED_SUBTYPE`: no FR-009 operator (eq/ne/gt/gte/lt/lte/in/like/isNull) + applies to a collection column. +- `error-sortable-array-field` — the same shape with `@sortable` in place of + `@filterable` fails `ERR_SORTABLE_UNSUPPORTED_SUBTYPE`: a collection column has + no ordering. + +A structural scan of the whole repo (1321 JSON + 124 YAML files) found **zero** +fields carrying both `isArray: true` and `@filterable`/`@sortable: true` before +these two fixtures were added — they are the **only** cases in the corpus that +exercise an array field through the filter/sort tier, so they are what makes the +rule (added to the TypeScript loader for #335) a cross-port contract rather than +a TS-only unit test. + +## Whole-object rollup (#335) + +`origin.aggregate @agg:collect` with `@of` omitted is a whole-object rollup: the +carrying field must be a `field.object @isArray @objectRef`, and the origin +assembles the related rows as an array of that declared value object — each +member matched by name against the `@via` path's TERMINAL entity — instead of +an array of one scalar column. + +- `collect-whole-object` — the positive case: a `field.object @isArray + @objectRef` collects the related entity's rows as an array of the declared + value object. **This is the corpus's only whole-object rollup** — every other + `@agg:collect` fixture (`origin-agg-collect`, `error-agg-collect-*`) collects a + single scalar column via `@of`. +- `error-collect-no-object-ref` — the carrying field declares no `@objectRef`. + Gates `ERR_INVALID_ORIGIN` (the whole-object-rollup rule itself) **and** + `ERR_OBJECT_FIELD_WITHOUT_OBJECT_REF` (a `field.object` always requires + `@objectRef`, independent of collect) — both fire on the same shape, so both + are asserted. +- `error-collect-ref-not-value` — `@objectRef` resolves to an `object.entity` + instead of an `object.value`. Gates `ERR_SUBTYPE_RULE_VIOLATION` (#210, + ADR-0028: a whole-object rollup must target a value). +- `error-collect-no-via` — `@via` is absent, and with no `@of` there is no + entity to infer the relationship from. Gates `ERR_INVALID_ORIGIN`. +- `error-collect-via-to-one` — every hop on `@via` is to-one (a passthrough + shape, not a rollup). Gates `ERR_ORIGIN_CARDINALITY`. +- `error-collect-distinct` — `@distinct` is declared on a whole-object rollup (a + no-op once the value object carries the primary key). Gates + `ERR_INVALID_ORIGIN`. +- `error-collect-member-unresolved` — a declared value-object member has no + matching field, by name, on the `@via` terminal entity. Gates + `ERR_COLLECT_MEMBER_UNRESOLVED` — failing open here is how #270 turned a + curated value object into the full entity, invisible in a diff. +- `error-collect-member-type-mismatch` — a declared value-object member's + `field.` differs from the matched terminal-entity field's. Gates + `ERR_INVALID_ORIGIN` (a whole-object rollup preserves each member's declared + type). +- `error-collect-orderby-not-terminal` — `@orderBy` names a field that exists on + a middle hop of a multi-hop `@via` path but not on the path's TERMINAL entity. + Gates `ERR_INVALID_ORIGIN` — `@orderBy` keys always resolve against the + terminal, never the head or an intermediate hop. + ## Adding a fixture Create a directory; add `input/` and expectation files. No runner code changes — @@ -101,9 +158,15 @@ silent. So every removal records what it covered and where that coverage went. **Coverage genuinely lost, stated rather than faked:** `flattened-kitchen-sink` dropped its `supplierBriefs` field, the corpus's only array-of-value-object carrying an origin. No surviving origin expresses a whole-object rollup along a -relationship — `origin.aggregate @agg:collect` reduces a *column* via `@of`. That -shape becomes expressible again when [#335](https://github.com/metaobjectsdev/metaobjects/issues/335) -makes `@of` optional on `collect`; the fixture is the place to restore it. +relationship — `origin.aggregate @agg:collect` reduces a *column* via `@of`. **RESTORED by [#335](https://github.com/metaobjectsdev/metaobjects/issues/335)**, +which makes `@of` optional on `collect` — see "Whole-object rollup (#335)" below. +The coverage lives in a DEDICATED fixture, `collect-whole-object`, rather than +back in `flattened-kitchen-sink`: that fixture's concern is `@storage: flattened`, +and a whole-object rollup exercises none of it, so folding the two would make +each harder to read in a five-language investigation. `collect-whole-object` is +an array-of-value-object carrying an origin — the exact shape recorded as lost — +and it is round-tripped against real Postgres and SQLite in +`server/typescript/packages/integration-tests/test/view-lifecycle-{pg,sqlite}.test.ts`. ## Generated fixtures (differential testing) diff --git a/fixtures/conformance/collect-whole-object/expected.json b/fixtures/conformance/collect-whole-object/expected.json new file mode 100644 index 000000000..76cedec37 --- /dev/null +++ b/fixtures/conformance/collect-whole-object/expected.json @@ -0,0 +1,142 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Supplier", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "suppliers" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + }, + { + "field.long": { + "name": "productId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "product", + "@fields": [ + "productId" + ], + "@references": "Product" + } + } + ] + } + }, + { + "object.entity": { + "name": "Product", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "products" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "relationship.association": { + "name": "suppliers", + "@cardinality": "many", + "@objectRef": "Supplier" + } + } + ] + } + }, + { + "object.value": { + "name": "SupplierBrief", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + } + ] + } + }, + { + "object.projection": { + "name": "ProductWithSuppliers", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_product_suppliers" + } + }, + { + "field.long": { + "name": "productId", + "extends": "Product.id" + } + }, + { + "identity.primary": { + "name": "id", + "extends": "Product.id" + } + }, + { + "field.object": { + "name": "supplierBriefs", + "isArray": true, + "@objectRef": "SupplierBrief", + "children": [ + { + "origin.aggregate": { + "@agg": "collect", + "@via": "Product.suppliers" + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/collect-whole-object/input/meta.demo.json b/fixtures/conformance/collect-whole-object/input/meta.demo.json new file mode 100644 index 000000000..c3ae29579 --- /dev/null +++ b/fixtures/conformance/collect-whole-object/input/meta.demo.json @@ -0,0 +1,142 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Supplier", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "suppliers" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + }, + { + "field.long": { + "name": "productId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "product", + "@references": "Product", + "@fields": [ + "productId" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Product", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "products" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "relationship.association": { + "name": "suppliers", + "@cardinality": "many", + "@objectRef": "Supplier" + } + } + ] + } + }, + { + "object.value": { + "name": "SupplierBrief", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + } + ] + } + }, + { + "object.projection": { + "name": "ProductWithSuppliers", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_product_suppliers" + } + }, + { + "field.long": { + "name": "productId", + "extends": "Product.id" + } + }, + { + "identity.primary": { + "name": "id", + "extends": "Product.id" + } + }, + { + "field.object": { + "name": "supplierBriefs", + "isArray": true, + "@objectRef": "SupplierBrief", + "children": [ + { + "origin.aggregate": { + "@agg": "collect", + "@via": "Product.suppliers" + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/collect-whole-object/providers.json b/fixtures/conformance/collect-whole-object/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/collect-whole-object/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-collect-distinct/expected-errors.json b/fixtures/conformance/error-collect-distinct/expected-errors.json new file mode 100644 index 000000000..f87765e1a --- /dev/null +++ b/fixtures/conformance/error-collect-distinct/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_COLLECT_WHOLE_OBJECT", + "source": { + "format": "json", + "files": [ + "meta.demo.json" + ], + "jsonPath": "$['metadata.root'].children[3]['object.projection'].children[3]['field.object'].children[0]['origin.aggregate']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-collect-distinct/input/meta.demo.json b/fixtures/conformance/error-collect-distinct/input/meta.demo.json new file mode 100644 index 000000000..2bb30a4e6 --- /dev/null +++ b/fixtures/conformance/error-collect-distinct/input/meta.demo.json @@ -0,0 +1,143 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Supplier", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "suppliers" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + }, + { + "field.long": { + "name": "productId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "product", + "@references": "Product", + "@fields": [ + "productId" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Product", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "products" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "relationship.association": { + "name": "suppliers", + "@cardinality": "many", + "@objectRef": "Supplier" + } + } + ] + } + }, + { + "object.value": { + "name": "SupplierBrief", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + } + ] + } + }, + { + "object.projection": { + "name": "ProductWithSuppliers", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_product_suppliers" + } + }, + { + "field.long": { + "name": "productId", + "extends": "Product.id" + } + }, + { + "identity.primary": { + "name": "id", + "extends": "Product.id" + } + }, + { + "field.object": { + "name": "supplierBriefs", + "isArray": true, + "@objectRef": "SupplierBrief", + "children": [ + { + "origin.aggregate": { + "@agg": "collect", + "@via": "Product.suppliers", + "@distinct": true + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-collect-distinct/providers.json b/fixtures/conformance/error-collect-distinct/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-collect-distinct/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-collect-member-type-mismatch/expected-errors.json b/fixtures/conformance/error-collect-member-type-mismatch/expected-errors.json new file mode 100644 index 000000000..f87765e1a --- /dev/null +++ b/fixtures/conformance/error-collect-member-type-mismatch/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_COLLECT_WHOLE_OBJECT", + "source": { + "format": "json", + "files": [ + "meta.demo.json" + ], + "jsonPath": "$['metadata.root'].children[3]['object.projection'].children[3]['field.object'].children[0]['origin.aggregate']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-collect-member-type-mismatch/input/meta.demo.json b/fixtures/conformance/error-collect-member-type-mismatch/input/meta.demo.json new file mode 100644 index 000000000..606ce3aad --- /dev/null +++ b/fixtures/conformance/error-collect-member-type-mismatch/input/meta.demo.json @@ -0,0 +1,142 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Supplier", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "suppliers" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + }, + { + "field.long": { + "name": "productId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "product", + "@references": "Product", + "@fields": [ + "productId" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Product", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "products" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "relationship.association": { + "name": "suppliers", + "@cardinality": "many", + "@objectRef": "Supplier" + } + } + ] + } + }, + { + "object.value": { + "name": "SupplierBrief", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.long": { + "name": "name" + } + } + ] + } + }, + { + "object.projection": { + "name": "ProductWithSuppliers", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_product_suppliers" + } + }, + { + "field.long": { + "name": "productId", + "extends": "Product.id" + } + }, + { + "identity.primary": { + "name": "id", + "extends": "Product.id" + } + }, + { + "field.object": { + "name": "supplierBriefs", + "isArray": true, + "@objectRef": "SupplierBrief", + "children": [ + { + "origin.aggregate": { + "@agg": "collect", + "@via": "Product.suppliers" + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-collect-member-type-mismatch/providers.json b/fixtures/conformance/error-collect-member-type-mismatch/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-collect-member-type-mismatch/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-collect-member-unresolved/expected-errors.json b/fixtures/conformance/error-collect-member-unresolved/expected-errors.json new file mode 100644 index 000000000..937d5ad1a --- /dev/null +++ b/fixtures/conformance/error-collect-member-unresolved/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_COLLECT_MEMBER_UNRESOLVED", + "source": { + "format": "json", + "files": [ + "meta.demo.json" + ], + "jsonPath": "$['metadata.root'].children[3]['object.projection'].children[3]['field.object'].children[0]['origin.aggregate']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-collect-member-unresolved/input/meta.demo.json b/fixtures/conformance/error-collect-member-unresolved/input/meta.demo.json new file mode 100644 index 000000000..11044defd --- /dev/null +++ b/fixtures/conformance/error-collect-member-unresolved/input/meta.demo.json @@ -0,0 +1,142 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Supplier", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "suppliers" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + }, + { + "field.long": { + "name": "productId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "product", + "@references": "Product", + "@fields": [ + "productId" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Product", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "products" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "relationship.association": { + "name": "suppliers", + "@cardinality": "many", + "@objectRef": "Supplier" + } + } + ] + } + }, + { + "object.value": { + "name": "SupplierBrief", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "nickname" + } + } + ] + } + }, + { + "object.projection": { + "name": "ProductWithSuppliers", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_product_suppliers" + } + }, + { + "field.long": { + "name": "productId", + "extends": "Product.id" + } + }, + { + "identity.primary": { + "name": "id", + "extends": "Product.id" + } + }, + { + "field.object": { + "name": "supplierBriefs", + "isArray": true, + "@objectRef": "SupplierBrief", + "children": [ + { + "origin.aggregate": { + "@agg": "collect", + "@via": "Product.suppliers" + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-collect-member-unresolved/providers.json b/fixtures/conformance/error-collect-member-unresolved/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-collect-member-unresolved/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-collect-no-object-ref/expected-errors.json b/fixtures/conformance/error-collect-no-object-ref/expected-errors.json new file mode 100644 index 000000000..f07822cbc --- /dev/null +++ b/fixtures/conformance/error-collect-no-object-ref/expected-errors.json @@ -0,0 +1,25 @@ +{ + "errors": [ + { + "code": "ERR_COLLECT_WHOLE_OBJECT", + "source": { + "format": "json", + "files": [ + "meta.demo.json" + ], + "jsonPath": "$['metadata.root'].children[3]['object.projection'].children[3]['field.object'].children[0]['origin.aggregate']" + } + }, + { + "code": "ERR_OBJECT_FIELD_WITHOUT_OBJECT_REF", + "source": { + "format": "json", + "files": [ + "meta.demo.json" + ], + "jsonPath": "$['metadata.root'].children[3]['object.projection'].children[3]['field.object']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-collect-no-object-ref/input/meta.demo.json b/fixtures/conformance/error-collect-no-object-ref/input/meta.demo.json new file mode 100644 index 000000000..150ee27c2 --- /dev/null +++ b/fixtures/conformance/error-collect-no-object-ref/input/meta.demo.json @@ -0,0 +1,141 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Supplier", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "suppliers" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + }, + { + "field.long": { + "name": "productId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "product", + "@references": "Product", + "@fields": [ + "productId" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Product", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "products" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "relationship.association": { + "name": "suppliers", + "@cardinality": "many", + "@objectRef": "Supplier" + } + } + ] + } + }, + { + "object.value": { + "name": "SupplierBrief", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + } + ] + } + }, + { + "object.projection": { + "name": "ProductWithSuppliers", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_product_suppliers" + } + }, + { + "field.long": { + "name": "productId", + "extends": "Product.id" + } + }, + { + "identity.primary": { + "name": "id", + "extends": "Product.id" + } + }, + { + "field.object": { + "name": "supplierBriefs", + "isArray": true, + "children": [ + { + "origin.aggregate": { + "@agg": "collect", + "@via": "Product.suppliers" + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-collect-no-object-ref/providers.json b/fixtures/conformance/error-collect-no-object-ref/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-collect-no-object-ref/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-collect-no-via/expected-errors.json b/fixtures/conformance/error-collect-no-via/expected-errors.json new file mode 100644 index 000000000..f87765e1a --- /dev/null +++ b/fixtures/conformance/error-collect-no-via/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_COLLECT_WHOLE_OBJECT", + "source": { + "format": "json", + "files": [ + "meta.demo.json" + ], + "jsonPath": "$['metadata.root'].children[3]['object.projection'].children[3]['field.object'].children[0]['origin.aggregate']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-collect-no-via/input/meta.demo.json b/fixtures/conformance/error-collect-no-via/input/meta.demo.json new file mode 100644 index 000000000..c33702111 --- /dev/null +++ b/fixtures/conformance/error-collect-no-via/input/meta.demo.json @@ -0,0 +1,141 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Supplier", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "suppliers" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + }, + { + "field.long": { + "name": "productId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "product", + "@references": "Product", + "@fields": [ + "productId" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Product", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "products" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "relationship.association": { + "name": "suppliers", + "@cardinality": "many", + "@objectRef": "Supplier" + } + } + ] + } + }, + { + "object.value": { + "name": "SupplierBrief", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + } + ] + } + }, + { + "object.projection": { + "name": "ProductWithSuppliers", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_product_suppliers" + } + }, + { + "field.long": { + "name": "productId", + "extends": "Product.id" + } + }, + { + "identity.primary": { + "name": "id", + "extends": "Product.id" + } + }, + { + "field.object": { + "name": "supplierBriefs", + "isArray": true, + "@objectRef": "SupplierBrief", + "children": [ + { + "origin.aggregate": { + "@agg": "collect" + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-collect-no-via/providers.json b/fixtures/conformance/error-collect-no-via/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-collect-no-via/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-collect-orderby-not-terminal/expected-errors.json b/fixtures/conformance/error-collect-orderby-not-terminal/expected-errors.json new file mode 100644 index 000000000..ebe223470 --- /dev/null +++ b/fixtures/conformance/error-collect-orderby-not-terminal/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_COLLECT_WHOLE_OBJECT", + "source": { + "format": "json", + "files": [ + "meta.demo.json" + ], + "jsonPath": "$['metadata.root'].children[4]['object.projection'].children[3]['field.object'].children[0]['origin.aggregate']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-collect-orderby-not-terminal/input/meta.demo.json b/fixtures/conformance/error-collect-orderby-not-terminal/input/meta.demo.json new file mode 100644 index 000000000..209353eb3 --- /dev/null +++ b/fixtures/conformance/error-collect-orderby-not-terminal/input/meta.demo.json @@ -0,0 +1,164 @@ +{ + "metadata.root": { + "package": "acme::chain", + "children": [ + { + "object.entity": { + "name": "C", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "cs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "B", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "bs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + }, + { + "relationship.association": { + "name": "c", + "@cardinality": "many", + "@objectRef": "C" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "A", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "as" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + }, + { + "relationship.association": { + "name": "b", + "@cardinality": "many", + "@objectRef": "B" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.value": { + "name": "CBrief", + "children": [ + { + "field.long": { + "name": "id" + } + } + ] + } + }, + { + "object.projection": { + "name": "AWithCs", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_a_cs" + } + }, + { + "field.long": { + "name": "aId", + "extends": "A.id" + } + }, + { + "identity.primary": { + "name": "id", + "extends": "A.id" + } + }, + { + "field.object": { + "name": "items", + "isArray": true, + "@objectRef": "CBrief", + "children": [ + { + "origin.aggregate": { + "@agg": "collect", + "@via": "A.b.c", + "@orderBy": [ + "name" + ] + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-collect-orderby-not-terminal/providers.json b/fixtures/conformance/error-collect-orderby-not-terminal/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-collect-orderby-not-terminal/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-collect-ref-not-value/expected-errors.json b/fixtures/conformance/error-collect-ref-not-value/expected-errors.json new file mode 100644 index 000000000..833e2e33c --- /dev/null +++ b/fixtures/conformance/error-collect-ref-not-value/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_SUBTYPE_RULE_VIOLATION", + "source": { + "format": "json", + "files": [ + "meta.demo.json" + ], + "jsonPath": "$['metadata.root'].children[3]['object.projection'].children[3]['field.object'].children[0]['origin.aggregate']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-collect-ref-not-value/input/meta.demo.json b/fixtures/conformance/error-collect-ref-not-value/input/meta.demo.json new file mode 100644 index 000000000..5cbf9c32c --- /dev/null +++ b/fixtures/conformance/error-collect-ref-not-value/input/meta.demo.json @@ -0,0 +1,142 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Supplier", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "suppliers" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + }, + { + "field.long": { + "name": "productId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "product", + "@references": "Product", + "@fields": [ + "productId" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Product", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "products" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "relationship.association": { + "name": "suppliers", + "@cardinality": "many", + "@objectRef": "Supplier" + } + } + ] + } + }, + { + "object.value": { + "name": "SupplierBrief", + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + } + ] + } + }, + { + "object.projection": { + "name": "ProductWithSuppliers", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_product_suppliers" + } + }, + { + "field.long": { + "name": "productId", + "extends": "Product.id" + } + }, + { + "identity.primary": { + "name": "id", + "extends": "Product.id" + } + }, + { + "field.object": { + "name": "supplierBriefs", + "isArray": true, + "@objectRef": "Supplier", + "children": [ + { + "origin.aggregate": { + "@agg": "collect", + "@via": "Product.suppliers" + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-collect-ref-not-value/providers.json b/fixtures/conformance/error-collect-ref-not-value/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-collect-ref-not-value/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-collect-via-to-one/expected-errors.json b/fixtures/conformance/error-collect-via-to-one/expected-errors.json new file mode 100644 index 000000000..cc0ee5ac1 --- /dev/null +++ b/fixtures/conformance/error-collect-via-to-one/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_ORIGIN_CARDINALITY", + "source": { + "format": "json", + "files": [ + "meta.demo.json" + ], + "jsonPath": "$['metadata.root'].children[3]['object.projection'].children[3]['field.object'].children[0]['origin.aggregate']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-collect-via-to-one/input/meta.demo.json b/fixtures/conformance/error-collect-via-to-one/input/meta.demo.json new file mode 100644 index 000000000..0e0df1320 --- /dev/null +++ b/fixtures/conformance/error-collect-via-to-one/input/meta.demo.json @@ -0,0 +1,137 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Supplier", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "suppliers" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name" + } + }, + { + "field.long": { + "name": "productId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "product", + "@references": "Product", + "@fields": [ + "productId" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Product", + "children": [ + { + "source.rdb": { + "@kind": "table", + "@table": "products" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "relationship.association": { + "name": "suppliers", + "@cardinality": "many", + "@objectRef": "Supplier" + } + } + ] + } + }, + { + "object.value": { + "name": "SupplierBrief", + "children": [ + { + "field.long": { + "name": "id" + } + } + ] + } + }, + { + "object.projection": { + "name": "ProductWithSuppliers", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_product_suppliers" + } + }, + { + "field.long": { + "name": "productId", + "extends": "Product.id" + } + }, + { + "identity.primary": { + "name": "id", + "extends": "Product.id" + } + }, + { + "field.object": { + "name": "supplierBriefs", + "isArray": true, + "@objectRef": "SupplierBrief", + "children": [ + { + "origin.aggregate": { + "@agg": "collect", + "@via": "Supplier.product" + } + } + ] + } + } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-collect-via-to-one/providers.json b/fixtures/conformance/error-collect-via-to-one/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-collect-via-to-one/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-filterable-array-field/expected-errors.json b/fixtures/conformance/error-filterable-array-field/expected-errors.json new file mode 100644 index 000000000..0bb23ab3d --- /dev/null +++ b/fixtures/conformance/error-filterable-array-field/expected-errors.json @@ -0,0 +1,13 @@ +{ + "errors": [ + { + "code": "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE", + "source": { + "format": "json", + "files": ["meta.demo.json"], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[3]['field.string']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-filterable-array-field/input/meta.demo.json b/fixtures/conformance/error-filterable-array-field/input/meta.demo.json new file mode 100644 index 000000000..17ddc8983 --- /dev/null +++ b/fixtures/conformance/error-filterable-array-field/input/meta.demo.json @@ -0,0 +1,18 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Product", + "children": [ + { "source.rdb": { "@kind": "table", "@table": "products" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "@fields": ["id"] } }, + { "field.string": { "name": "tags", "isArray": true, "@filterable": true, "@db.indexed": true } } + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-sortable-array-field/expected-errors.json b/fixtures/conformance/error-sortable-array-field/expected-errors.json new file mode 100644 index 000000000..a1c16778d --- /dev/null +++ b/fixtures/conformance/error-sortable-array-field/expected-errors.json @@ -0,0 +1,13 @@ +{ + "errors": [ + { + "code": "ERR_SORTABLE_UNSUPPORTED_SUBTYPE", + "source": { + "format": "json", + "files": ["meta.demo.json"], + "jsonPath": "$['metadata.root'].children[0]['object.entity'].children[3]['field.string']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-sortable-array-field/input/meta.demo.json b/fixtures/conformance/error-sortable-array-field/input/meta.demo.json new file mode 100644 index 000000000..f5a385b48 --- /dev/null +++ b/fixtures/conformance/error-sortable-array-field/input/meta.demo.json @@ -0,0 +1,18 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Product", + "children": [ + { "source.rdb": { "@kind": "table", "@table": "products" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "@fields": ["id"] } }, + { "field.string": { "name": "tags", "isArray": true, "@sortable": true } } + ] + } + } + ] + } +} diff --git a/fixtures/metamodel-docs/expected/INDEX.md b/fixtures/metamodel-docs/expected/INDEX.md index 037fc86cf..d36b475f5 100644 --- a/fixtures/metamodel-docs/expected/INDEX.md +++ b/fixtures/metamodel-docs/expected/INDEX.md @@ -55,7 +55,7 @@ children, and cardinality of a subtype. Universal documentation attributes | `object.entity` | An object that owns its data: own identity, writable sources, and lifecycle. The default object subtype — a bare `object:` key resolves to entity. May co-locate templates (template.prompt and friends) with the owning entity. | [types/object.md#objectentity](types/object.md#objectentity) | | `object.projection` | A derived read-only representation of entities. Its fields are extends-bound / origin-derived / self-declared-under-external-assembly, all read-only at the subtype level. Identity is optional and MUST extend an entity identity; sources are restricted to read-only @kinds. The declared field set IS the exposure (inclusive, fail-closed). | [types/object.md#objectprojection](types/object.md#objectprojection) | | `object.value` | A value object — pure shape with NO identity and NO source, ever. Constructed (by caller / embedding), never populated from a store. May `extends` an entity's fields to reuse shape. Equality is by content. | [types/object.md#objectvalue](types/object.md#objectvalue) | -| `origin.aggregate` | A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup of @of). | [types/origin.md#originaggregate](types/origin.md#originaggregate) | +| `origin.aggregate` | A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted). | [types/origin.md#originaggregate](types/origin.md#originaggregate) | | `origin.base` | Abstract base origin — the shared root subtype for field-level provenance. A field carrying any origin.* is derived ⇒ read-only wherever it lives. The base carries no attrs of its own; concrete subtypes add their provenance attrs. | [types/origin.md#originbase](types/origin.md#originbase) | | `origin.computed` | A row-level value computed from the base entity's own fields via a structured expression tree (@expr). No related rows, no @via. Read-only; the expression's inferred type must equal the field's declared subType. | [types/origin.md#origincomputed](types/origin.md#origincomputed) | | `origin.first` | The single related row selected by @orderBy along @via, projecting its @of column (argmax then project). Latest = @orderBy desc. Read-only; empty related set (after @filter) → null, so the field must not be @required. | [types/origin.md#originfirst](types/origin.md#originfirst) | diff --git a/fixtures/metamodel-docs/expected/types/origin.md b/fixtures/metamodel-docs/expected/types/origin.md index c7f2d2d38..7e876c350 100644 --- a/fixtures/metamodel-docs/expected/types/origin.md +++ b/fixtures/metamodel-docs/expected/types/origin.md @@ -12,11 +12,11 @@ documentation attributes are omitted here (see [providers.md](../providers.md)). ### origin.aggregate -A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup of @of). +A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted). **Owning provider:** metaobjects-core-types -**Rules:** @via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max/collect and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only. +**Rules:** @via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only. On @agg:collect @of is OPTIONAL: omitting it declares a WHOLE-OBJECT rollup, which collects each related row as the carrying field's declared @objectRef value object instead of one scalar column. A whole-object rollup requires a field.object carrying @objectRef, requires that @objectRef to name an object.value, requires an explicit @via (there is no @of entity to infer the path from), and refuses @distinct. Its @orderBy keys resolve against the @via TERMINAL entity, not the head or a middle hop. Its value-object members bind to the terminal entity's fields BY NAME — member name == terminal field name, deliberately NOT extends, so one value object stays collectable from two different entities — and every member must match a terminal field agreeing on BOTH field. and array-ness. **When to use:** A projection needs a value derived by reducing related rows — a count/sum/avg/min/max, a 'did any/every related row match' flag, or an array of collected values. Declare it instead of hand-writing the aggregate query — it stays consistent and regenerates. @@ -24,10 +24,10 @@ A value reduced from the related row-set reached along a relationship path (@via | Attribute | Type | Required | Default | Allowed values | Provider | Description | | --- | --- | --- | --- | --- | --- | --- | -| `@agg` | string | yes | | `count`, `sum`, `avg`, `min`, `max`, `any`, `all`, `collect` | metaobjects-core-types | The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup of @of — the field must be isArray). | -| `@distinct` | boolean | no | | | metaobjects-core-types | Set (collect-only) to dedupe collected values (set semantics). | +| `@agg` | string | yes | | `count`, `sum`, `avg`, `min`, `max`, `any`, `all`, `collect` | metaobjects-core-types | The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted; the field must be isArray). | +| `@distinct` | boolean | no | | | metaobjects-core-types | Set (collect-only) to dedupe collected values (set semantics). Not supported on a whole-object collect (@of omitted): it is a guaranteed no-op whenever the value object carries the primary key, and a silent no-op is worse than a refusal. | | `@filter` | filter | no | | | metaobjects-core-types | Optional structured predicate scoping which related rows the aggregate spans (required for any/all, where it is the quantified predicate). A portable attr.filter object (eq/ne/in/isNull with and/or), desugared to canonical { field: { op: value } } at parse time; codegen renders it per target (e.g. SQL FILTER (WHERE ...) or SQLite CASE WHEN for a relational view). | -| `@of` | string | no | | | metaobjects-core-types | Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max/collect; forbidden for any/all (which quantify over rows via @filter, not a column). | +| `@of` | string | no | | | metaobjects-core-types | Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max; OPTIONAL for collect, where absent means a whole-object rollup of the field's declared @objectRef value object; forbidden for any/all (which quantify over rows via @filter, not a column). | | `@orderBy` | string[] | no | | | metaobjects-core-types | Ordering keys as 'field[:asc\|desc]' (default asc) over the related entity's fields; nulls sort last. On @agg:collect sets element order (non-distinct only); on origin.first (required) selects the row. Semantic — carries no SQL syntax. | | `@via` | string | no | | | metaobjects-core-types | Dotted relationship path from the base entity to the aggregated rows (e.g. 'Program.weeks' or 'Program.weeks.workouts'). May be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (FR-024, ADR-0029). | diff --git a/fixtures/registry-conformance/expected-registry.json b/fixtures/registry-conformance/expected-registry.json index 6a8b468b0..5256cf97f 100644 --- a/fixtures/registry-conformance/expected-registry.json +++ b/fixtures/registry-conformance/expected-registry.json @@ -1,5 +1,5 @@ { - "metamodelVersion": "0.11", + "metamodelVersion": "0.12", "types": [ { "type": "attr", @@ -3363,8 +3363,8 @@ { "type": "origin", "subType": "aggregate", - "description": "A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup of @of).", - "rules": "@via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max/collect and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only.", + "description": "A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted).", + "rules": "@via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only. On @agg:collect @of is OPTIONAL: omitting it declares a WHOLE-OBJECT rollup, which collects each related row as the carrying field's declared @objectRef value object instead of one scalar column. A whole-object rollup requires a field.object carrying @objectRef, requires that @objectRef to name an object.value, requires an explicit @via (there is no @of entity to infer the path from), and refuses @distinct. Its @orderBy keys resolve against the @via TERMINAL entity, not the head or a middle hop. Its value-object members bind to the terminal entity's fields BY NAME — member name == terminal field name, deliberately NOT extends, so one value object stays collectable from two different entities — and every member must match a terminal field agreeing on BOTH field. and array-ness.", "whenToUse": "A projection needs a value derived by reducing related rows — a count/sum/avg/min/max, a 'did any/every related row match' flag, or an array of collected values. Declare it instead of hand-writing the aggregate query — it stays consistent and regenerates.", "attrs": [ { @@ -3382,14 +3382,14 @@ "all", "collect" ], - "description": "The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup of @of — the field must be isArray)." + "description": "The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted; the field must be isArray)." }, { "name": "distinct", "valueType": "boolean", "isArray": false, "required": false, - "description": "Set (collect-only) to dedupe collected values (set semantics)." + "description": "Set (collect-only) to dedupe collected values (set semantics). Not supported on a whole-object collect (@of omitted): it is a guaranteed no-op whenever the value object carries the primary key, and a silent no-op is worse than a refusal." }, { "name": "filter", @@ -3403,7 +3403,7 @@ "valueType": "string", "isArray": false, "required": false, - "description": "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max/collect; forbidden for any/all (which quantify over rows via @filter, not a column)." + "description": "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max; OPTIONAL for collect, where absent means a whole-object rollup of the field's declared @objectRef value object; forbidden for any/all (which quantify over rows via @filter, not a column)." }, { "name": "orderBy", diff --git a/server/csharp/MetaObjects/Errors.cs b/server/csharp/MetaObjects/Errors.cs index b804612d3..1f92c4527 100644 --- a/server/csharp/MetaObjects/Errors.cs +++ b/server/csharp/MetaObjects/Errors.cs @@ -173,6 +173,23 @@ public enum ErrorCode // SP-H Unit9 — @filterable: true on a field subtype with no filter-operator // band (e.g. field.object). Would silently generate an empty-ops filter. ERR_FILTERABLE_UNSUPPORTED_SUBTYPE, + // #335 Half B — @sortable: true on an array field or a subtype with no + // filter-operator band (e.g. field.object). Would silently emit a sort + // entry over a column no dialect can ORDER BY. + ERR_SORTABLE_UNSUPPORTED_SUBTYPE, + // #335 Half A — a whole-object @agg:collect (no @of; the carrying field.object + // rolls related rows up as its declared @objectRef value object) is malformed: + // carrier is not a field.object with @objectRef, @via absent, @distinct declared + // (refused — a no-op whenever the value object carries the primary key), an + // @orderBy key not on the @via TERMINAL entity, or a member's declared type + // disagreeing with the matched terminal field's. Distinct from ERR_INVALID_ORIGIN + // so a fixture can tell this arm from a loader that still requires @of. + ERR_COLLECT_WHOLE_OBJECT, + // #335 Half A — a whole-object @agg:collect's value-object member has no + // matching field (by name) on the @via terminal entity. The lowering + // projects exactly the declared members; failing open here is how #270 + // turned a curated value object into the full entity. + ERR_COLLECT_MEMBER_UNRESOLVED, // Index-key resolution for index.lookup AND identity.secondary (#342) — the key is // @fields XOR @expr: neither declared, BOTH declared (@expr is used INSTEAD of // @fields), whichever is declared supplies no key, or a named field does not exist diff --git a/server/csharp/MetaObjects/Loader/MetaDataLoader.cs b/server/csharp/MetaObjects/Loader/MetaDataLoader.cs index 04351ba49..0bc0a90a3 100644 --- a/server/csharp/MetaObjects/Loader/MetaDataLoader.cs +++ b/server/csharp/MetaObjects/Loader/MetaDataLoader.cs @@ -503,9 +503,12 @@ public LoadResult Load(IReadOnlyList sources) // Pass 4: filterable-without-index drift warning warnings.AddRange(ValidationPasses.ValidateFilterableHasIndex(root)); - // Pass 4b: @filterable on a subtype with no operator band → error (SP-H Unit9) + // Pass 4b: @filterable on a subtype with no operator band, or an array → error (SP-H Unit9 / #335) errors.AddRange(ValidationPasses.ValidateFilterableHasSupportedOps(root)); + // Pass 4c: @sortable on a subtype with no operator band, or an array → error (#335 Half B) + errors.AddRange(ValidationPasses.ValidateSortableHasSupportedSubtype(root)); + // Pass 5: origin path validation (FR-024 B5/B6 — @via inference, // cardinality, extends/origin agreement) errors.AddRange(ValidationPasses.ValidateOriginPaths(root)); diff --git a/server/csharp/MetaObjects/Loader/ValidationPasses.cs b/server/csharp/MetaObjects/Loader/ValidationPasses.cs index 9bd42d31a..dc99fb002 100644 --- a/server/csharp/MetaObjects/Loader/ValidationPasses.cs +++ b/server/csharp/MetaObjects/Loader/ValidationPasses.cs @@ -376,6 +376,23 @@ public static IReadOnlyList ValidateFilterableHasSupportedOps(MetaDat { // ADR-0039: resolving — a concrete field may inherit @filterable via extends (TS validation-passes.ts:332). if (field.Attr(FIELD_ATTR_FILTERABLE) is not true) continue; + + // #335 Half B — an ARRAY field has no operator band either. Every FR-009 + // operator (eq/ne/gt/gte/lt/lte/in/like/isNull) is a scalar comparison; + // none applies to a collection column. Same reason as the subtype check + // below, so same code. + // ADR-0039: ResolvedIsArray(), never the own `isArray` flag. + if (field.ResolvedIsArray()) + { + errors.Add(new MetaError( + $"Field \"{obj.Name}.{field.Name}\" has @filterable: true but is an array " + + "(isArray: true). No filter operator applies to a collection column. " + + "Remove @filterable from this field.", + ErrorCode.ERR_FILTERABLE_UNSUPPORTED_SUBTYPE, + Envelope: field.Source)); + continue; + } + if (OpsForSubType(field.SubType).Length > 0) continue; errors.Add(new MetaError( $"Field \"{obj.Name}.{field.Name}\" has @filterable: true but its subtype " + @@ -389,6 +406,48 @@ public static IReadOnlyList ValidateFilterableHasSupportedOps(MetaDat return errors.AsReadOnly(); } + // ========================================================================= + // Pass 4c: ValidateSortableHasSupportedSubtype (#335 Half B) + // - @sortable: true on an ARRAY field, or a field subtype with NO entry in + // OPS_BY_SUBTYPE → error ERR_SORTABLE_UNSUPPORTED_SUBTYPE. @sortable + // defaults FROM @filterable, so it is independently checked only when + // explicit — a @sortable JSON or array column emits a sort entry over a + // column no dialect can ORDER BY meaningfully. + // + // Ported from typescript/packages/metadata/src/loader/validation-passes.ts + // validateSortableHasSupportedSubtype. + // ========================================================================= + + public static IReadOnlyList ValidateSortableHasSupportedSubtype(MetaData root) + { + var errors = new List(); + + foreach (var obj in root.OwnChildren() + .Where(c => c.Type == TYPE_OBJECT)) + { + // Children() (effective) — inherited @sortable fields are visible. + foreach (var field in obj.Children().Where(c => c.Type == TYPE_FIELD)) + { + // ADR-0039: resolving — a concrete field may inherit @sortable via extends. + if (field.Attr(FIELD_ATTR_SORTABLE) is not true) continue; + // ADR-0039: ResolvedIsArray(), never the own `isArray` flag. + bool isArray = field.ResolvedIsArray(); + if (!isArray && OpsForSubType(field.SubType).Length > 0) continue; + + var reason = isArray + ? "is an array (isArray: true) — a collection column has no ordering." + : $"its subtype \"{field.SubType}\" cannot be ordered."; + errors.Add(new MetaError( + $"Field \"{obj.Name}.{field.Name}\" has @sortable: true but {reason} " + + "Remove @sortable from this field.", + ErrorCode.ERR_SORTABLE_UNSUPPORTED_SUBTYPE, + Envelope: field.Source)); + } + } + + return errors.AsReadOnly(); + } + // ========================================================================= // Pass 5: ValidateOriginPaths // - passthrough.@from / aggregate.@of must resolve to existing Entity.field @@ -464,9 +523,9 @@ public static IReadOnlyList ValidateOriginPaths(MetaData root) var viaObj = origin.OwnAttr(ORIGIN_PASSTHROUGH_ATTR_VIA); if (viaObj is string via && via != "") { - var hops = ValidateViaPath(via, root, obj, field.Name, errors, origin.Source); - if (hops is not null) - CheckPassthroughCardinality(hops, obj, field.Name, origin.Source, errors); + var walked = ValidateViaPath(via, root, obj, field.Name, errors, origin.Source); + if (walked is WalkedViaPath wp) + CheckPassthroughCardinality(wp.Hops, obj, field.Name, origin.Source, errors); } else if (fromTarget is ResolvedFromTarget ft2 && !isValueHost) { @@ -547,19 +606,95 @@ public static IReadOnlyList ValidateOriginPaths(MetaData root) ErrorCode.ERR_INVALID_ORIGIN, Envelope: src)); else { - var predHops = ValidateViaPath(predVia, root, obj, field.Name, errors, src); - if (predHops is not null) CheckAggregateCardinality(predHops, obj, field.Name, src, errors); + var predWalked = ValidateViaPath(predVia, root, obj, field.Name, errors, src); + if (predWalked is WalkedViaPath pw) CheckAggregateCardinality(pw.Hops, obj, field.Name, src, errors); } continue; } - // --- count/sum/avg/min/max/collect: @of REQUIRED --- + // --- @of: REQUIRED for count/sum/avg/min/max; OPTIONAL for collect --- + // #335 — an @of-absent collect is a WHOLE-OBJECT rollup: collect the + // related rows as an array of the field's declared @objectRef value + // object rather than an array of one scalar column. if (!ofPresent) { - // Missing-attr — keep origin's own source envelope. - errors.Add(new MetaError( - $"origin.aggregate on {obj.Name}.{field.Name}: missing @of.", - ErrorCode.ERR_INVALID_ORIGIN, Envelope: src)); + if (!isCollect) + { + // Missing-attr — keep origin's own source envelope. + errors.Add(new MetaError( + $"origin.aggregate on {obj.Name}.{field.Name}: missing @of.", + ErrorCode.ERR_INVALID_ORIGIN, Envelope: src)); + continue; + } + // Whole-object rollup. The carrying field must be a field.object + // naming a value object, and @via must be explicit (there is no @of + // entity to infer the single-hop relation from). + // ADR-0039: resolving — @objectRef may be inherited via extends. + var objectRefObj = field.Attr(FIELD_ATTR_OBJECT_REF); + if (field.SubType != FIELD_SUBTYPE_OBJECT + || objectRefObj is not string objectRef || objectRef == "") + { + errors.Add(new MetaError( + $"origin.aggregate @agg:collect on {obj.Name}.{field.Name}: @of is omitted, so this is a " + + "whole-object rollup — the carrying field must be a field.object declaring @objectRef " + + "(add @of to collect a single column instead).", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, Envelope: src)); + continue; + } + // #210's value-only rule is PAYLOAD-scoped and never reaches a + // projection-hosted field, so this branch enforces it itself. + // Without it an @objectRef to an entity silently rolls up the FULL + // entity — the #270 shape, this time baked into DDL. + // ADR-0042 — a bare @objectRef resolves in the DECLARING owner's + // package (an inherited field resolves in the package that declared it). + var refTarget = FindObject(root, objectRef, + NamingRefs.EffectivePackage(field.Parent ?? obj)); + if (refTarget is not null && refTarget.SubType != OBJECT_SUBTYPE_VALUE) + { + errors.Add(new MetaError( + $"origin.aggregate @agg:collect on {obj.Name}.{field.Name}: @objectRef '{objectRef}' " + + $"resolves to {TYPE_OBJECT}.{refTarget.SubType} — a whole-object rollup must target an " + + "object.value (#210, ADR-0028).", + ErrorCode.ERR_SUBTYPE_RULE_VIOLATION, Envelope: src)); + continue; + } + // ADR-0039: own — origin.* never inherits (ADR-0029). + if (origin.OwnAttr(ORIGIN_AGGREGATE_ATTR_VIA) is not string woVia || woVia == "") + { + errors.Add(new MetaError( + $"origin.aggregate @agg:collect on {obj.Name}.{field.Name}: @via is required on a " + + "whole-object rollup — there is no @of entity to infer the relationship from.", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, Envelope: src)); + continue; + } + // @distinct is refused on the object form. It is NOT an engine limit + // (both engines dedupe JSON objects); it is a guaranteed no-op + // whenever the value object carries the entity's primary key, which + // is the common case, and a silent no-op is worse than a refusal. + if (hasDistinct) + { + errors.Add(new MetaError( + $"origin.aggregate @agg:collect on {obj.Name}.{field.Name}: @distinct is not supported on a " + + "whole-object rollup (it is a no-op whenever the value object carries the primary key).", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, Envelope: src)); + continue; + } + // One walk yields both the hops (cardinality) and the terminal entity + // (@orderBy keys, member resolution). An invalid @via (e.g. a single- + // segment "A") returns null having already pushed its own error, so + // everything downstream is skipped and no second, misleadingly-scoped + // error is emitted. + var woWalked = ValidateViaPath(woVia, root, obj, field.Name, errors, src); + if (woWalked is WalkedViaPath ww) + { + CheckAggregateCardinality(ww.Hops, obj, field.Name, src, errors); + // @orderBy keys resolve against the @via TERMINAL entity, not @of. + ValidateOrderByKeys(orderBy, ww.Terminal, obj, field.Name, + "origin.aggregate @agg:collect", src, errors, + ErrorCode.ERR_COLLECT_WHOLE_OBJECT); + if (refTarget is not null) + CheckCollectMembers(refTarget, ww.Terminal, obj, field, src, errors); + } continue; } string of = (string)ofObj!; @@ -581,9 +716,9 @@ public static IReadOnlyList ValidateOriginPaths(MetaData root) var viaObj = origin.OwnAttr(ORIGIN_AGGREGATE_ATTR_VIA); if (viaObj is string via && via != "") { - var hops = ValidateViaPath(via, root, obj, field.Name, errors, src); - if (hops is not null) - CheckAggregateCardinality(hops, obj, field.Name, src, errors); + var walked = ValidateViaPath(via, root, obj, field.Name, errors, src); + if (walked is WalkedViaPath aw) + CheckAggregateCardinality(aw.Hops, obj, field.Name, src, errors); continue; } // FR-024 §6 — no @via on an aggregate: inference applies only @@ -677,8 +812,8 @@ public static IReadOnlyList ValidateOriginPaths(MetaData root) var viaObj = origin.OwnAttr(ORIGIN_FIRST_ATTR_VIA); if (viaObj is string via && via != "") { - var hops = ValidateViaPath(via, root, obj, field.Name, errors, src); - if (hops is not null) CheckAggregateCardinality(hops, obj, field.Name, src, errors); + var walked = ValidateViaPath(via, root, obj, field.Name, errors, src); + if (walked is WalkedViaPath fw) CheckAggregateCardinality(fw.Hops, obj, field.Name, src, errors); } else if (ofTarget is ResolvedFromTarget inferTarget) { @@ -716,9 +851,67 @@ public static IReadOnlyList ValidateOriginPaths(MetaData root) /// order) and origin.first (row selection). A null related entity means a /// prior error already fired — skip silently. Mirrors TS _validateOrderByKeys. /// + /// A field's declared type on BOTH axes — field.<subType> plus [] when it + /// is an array. ADR-0039: ResolvedIsArray(), so array-ness inherited via extends counts. + /// Mirrors the TS _typeLabel. + private static string TypeLabel(MetaData field) + => $"field.{field.SubType}{(field.ResolvedIsArray() ? "[]" : "")}"; + + /// #335 — a whole-object @agg:collect projects EXACTLY the declared value object's + /// members, matched BY NAME against the @via terminal entity's fields: + /// + /// an unmatched member is an error, never a silent drop. Failing open here is how + /// #270 turned a curated value object into the full entity, invisible in a diff because + /// the metadata still read as curated. + /// a matched member must agree on BOTH type axes (#185 type-preserving doctrine), + /// so a scalar member cannot bind an array field or vice versa. + /// + /// Both refusals carry a whole-object-specific code — ERR_COLLECT_MEMBER_UNRESOLVED for the + /// unmatched member, ERR_COLLECT_WHOLE_OBJECT for the type disagreement. The latter is + /// deliberately NOT the scalar arm's ERR_INVALID_ORIGIN: a loader that still requires @of + /// rejects this metadata with ERR_INVALID_ORIGIN too, so sharing the code would make a + /// corpus fixture pass on a port that implements nothing. + /// Mirrors the TS _checkCollectMembers. + private static void CheckCollectMembers( + MetaData refTarget, MetaData terminal, MetaData obj, MetaData field, + ErrorSource src, List errors) + { + // ADR-0039: resolving — a value object may inherit members via extends, and the + // terminal entity may inherit fields; own-only would silently skip inherited + // members, which is exactly the #270 bug class this guards. + var terminalFields = terminal.Children().Where(c => c.Type == TYPE_FIELD).ToList(); + foreach (var member in refTarget.Children().Where(c => c.Type == TYPE_FIELD)) + { + var match = terminalFields.FirstOrDefault(f => f.Name == member.Name); + if (match is null) + { + errors.Add(new MetaError( + $"origin.aggregate @agg:collect on {obj.Name}.{field.Name}: value-object member " + + $"'{member.Name}' has no matching field on '{terminal.Name}' — a whole-object " + + "rollup projects exactly the declared members.", + ErrorCode.ERR_COLLECT_MEMBER_UNRESOLVED, Envelope: src)); + continue; + } + string memberLabel = TypeLabel(member); + string matchLabel = TypeLabel(match); + if (memberLabel != matchLabel) + { + errors.Add(new MetaError( + $"origin.aggregate @agg:collect on {obj.Name}.{field.Name}: value-object member " + + $"'{member.Name}' is {memberLabel} but '{terminal.Name}.{match.Name}' is " + + $"{matchLabel} — a whole-object rollup preserves each member's type.", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, Envelope: src)); + } + } + } + + /// lets #335's whole-object @agg:collect arm report + /// ERR_COLLECT_WHOLE_OBJECT instead; it defaults to ERR_INVALID_ORIGIN so the scalar + /// @of and origin.first call sites keep their existing envelope byte-for-byte. private static void ValidateOrderByKeys( object? orderBy, MetaData? relatedEntity, MetaData obj, string fieldName, - string label, ErrorSource originSource, List errors) + string label, ErrorSource originSource, List errors, + ErrorCode code = ErrorCode.ERR_INVALID_ORIGIN) { // @orderBy is a declared string[] attr → stored as IReadOnlyList. if (orderBy is not IReadOnlyList keys || relatedEntity is null) return; @@ -734,13 +927,13 @@ private static void ValidateOrderByKeys( { errors.Add(new MetaError( $"{label} on {obj.Name}.{fieldName}: @orderBy key \"{raw}\" — no such field \"{key}\" on {relatedEntity.Name}.", - ErrorCode.ERR_INVALID_ORIGIN, Envelope: originSource)); + code, Envelope: originSource)); } else if (dir is not null && !SORT_ORDER_VALUES.Contains(dir)) { errors.Add(new MetaError( $"{label} on {obj.Name}.{fieldName}: @orderBy key \"{raw}\" — direction must be one of {string.Join("|", SORT_ORDER_VALUES)}.", - ErrorCode.ERR_INVALID_ORIGIN, Envelope: originSource)); + code, Envelope: originSource)); } } } @@ -892,7 +1085,13 @@ private static bool IsReferenceHop(MetaData hop) /// Validate an explicit `@via` "Entity.rel[.rel...]" path. Returns the walked /// relationship hop nodes (in path order) on full success — FR-024 B5 runs /// the cardinality checks over them — or null when any error was pushed. - private static List? ValidateViaPath( + /// A fully-walked @via path: the relationship hop nodes in path order, plus the + /// entity they terminate at. Returned as a pair (rather than re-walking for the terminal) + /// because a second walk means a second copy of the ADR-0042 package-resolution rule — + /// mirrors the TS WalkedViaPath. + private readonly record struct WalkedViaPath(List Hops, MetaData Terminal); + + private static WalkedViaPath? ValidateViaPath( string viaAttr, MetaData root, MetaData projection, @@ -992,7 +1191,8 @@ private static bool IsReferenceHop(MetaData hop) currentObj = nextObj; } - return hops; + // currentObj is the terminal: every earlier exit returned null. + return new WalkedViaPath(hops, currentObj); } // ------------------------------------------------------------------------- diff --git a/server/csharp/MetaObjects/Persistence/Origin/OriginSchema.cs b/server/csharp/MetaObjects/Persistence/Origin/OriginSchema.cs index 4fed267f6..5bff8120c 100644 --- a/server/csharp/MetaObjects/Persistence/Origin/OriginSchema.cs +++ b/server/csharp/MetaObjects/Persistence/Origin/OriginSchema.cs @@ -39,7 +39,7 @@ public static class OriginSchema Required: true, // #195 — the full @agg vocabulary (numeric reduces + any/all quantifiers + collect). AllowedValues: [.. OriginConstants.ORIGIN_AGG_VALUES], - Description: "The reducing function applied over the related row-set: count/sum/avg/min/max, any/all (predicate quantifiers over @filter), or collect (array rollup of @of)."), + Description: "The reducing function applied over the related row-set: count/sum/avg/min/max, any/all (predicate quantifiers over @filter), or collect (array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted)."), // #195 — @of relaxed to optional (any/all forbid it, the rest require it); // presence is enforced per-@agg in the origin-path validation pass. @@ -47,7 +47,7 @@ public static class OriginSchema Name: OriginConstants.ORIGIN_AGGREGATE_ATTR_OF, ValueType: AttrConstants.ATTR_SUBTYPE_STRING, Required: false, - Description: "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max/collect; forbidden for any/all."), + Description: "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max; OPTIONAL for collect, where absent means a whole-object rollup of the field's declared @objectRef value object; forbidden for any/all."), new AttrSchema( Name: OriginConstants.ORIGIN_AGGREGATE_ATTR_VIA, @@ -66,7 +66,7 @@ public static class OriginSchema Name: OriginConstants.ORIGIN_ATTR_DISTINCT, ValueType: AttrConstants.ATTR_SUBTYPE_BOOLEAN, Required: false, - Description: "Set (collect-only) to dedupe collected values (set semantics)."), + Description: "Set (collect-only) to dedupe collected values (set semantics). Not supported on a whole-object collect (@of omitted): it is a guaranteed no-op whenever the value object carries the primary key."), new AttrSchema( Name: OriginConstants.ORIGIN_ATTR_ORDER_BY, diff --git a/server/csharp/MetaObjects/RegistryManifest.cs b/server/csharp/MetaObjects/RegistryManifest.cs index 2728360dd..36e93443a 100644 --- a/server/csharp/MetaObjects/RegistryManifest.cs +++ b/server/csharp/MetaObjects/RegistryManifest.cs @@ -374,7 +374,7 @@ private static ManifestType ToManifestType(TypeDefinition def) => /// unstable (semver major-0). The 1.0 cut flips this to "1.0". /// Emitted as the FIRST top-level key in the registry manifest (C4). /// - public const string MetamodelVersion = "0.11"; + public const string MetamodelVersion = "0.12"; /// /// Emit the canonical registry manifest as a byte-stable JSON string. diff --git a/server/csharp/MetaObjects/SpecMetamodel/origin.json b/server/csharp/MetaObjects/SpecMetamodel/origin.json index e6be6d33b..6ddd1dbb7 100644 --- a/server/csharp/MetaObjects/SpecMetamodel/origin.json +++ b/server/csharp/MetaObjects/SpecMetamodel/origin.json @@ -20,15 +20,15 @@ { "type": "origin", "subType": "aggregate", - "description": "A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup of @of).", + "description": "A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted).", "whenToUse": "A projection needs a value derived by reducing related rows — a count/sum/avg/min/max, a 'did any/every related row match' flag, or an array of collected values. Declare it instead of hand-writing the aggregate query — it stays consistent and regenerates.", - "rules": "@via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max/collect and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only.", + "rules": "@via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only. On @agg:collect @of is OPTIONAL: omitting it declares a WHOLE-OBJECT rollup, which collects each related row as the carrying field's declared @objectRef value object instead of one scalar column. A whole-object rollup requires a field.object carrying @objectRef, requires that @objectRef to name an object.value, requires an explicit @via (there is no @of entity to infer the path from), and refuses @distinct. Its @orderBy keys resolve against the @via TERMINAL entity, not the head or a middle hop. Its value-object members bind to the terminal entity's fields BY NAME — member name == terminal field name, deliberately NOT extends, so one value object stays collectable from two different entities — and every member must match a terminal field agreeing on BOTH field. and array-ness.", "children": [ - { "type": "attr", "subType": "string", "name": "agg", "min": 1, "max": 1, "allowedValues": ["count", "sum", "avg", "min", "max", "any", "all", "collect"], "description": "The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup of @of — the field must be isArray)." }, - { "type": "attr", "subType": "string", "name": "of", "min": 0, "max": 1, "description": "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max/collect; forbidden for any/all (which quantify over rows via @filter, not a column)." }, + { "type": "attr", "subType": "string", "name": "agg", "min": 1, "max": 1, "allowedValues": ["count", "sum", "avg", "min", "max", "any", "all", "collect"], "description": "The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted; the field must be isArray)." }, + { "type": "attr", "subType": "string", "name": "of", "min": 0, "max": 1, "description": "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max; OPTIONAL for collect, where absent means a whole-object rollup of the field's declared @objectRef value object; forbidden for any/all (which quantify over rows via @filter, not a column)." }, { "type": "attr", "subType": "string", "name": "via", "min": 0, "max": 1, "description": "Dotted relationship path from the base entity to the aggregated rows (e.g. 'Program.weeks' or 'Program.weeks.workouts'). May be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (FR-024, ADR-0029)." }, { "type": "attr", "subType": "filter", "name": "filter", "min": 0, "max": 1, "description": "Optional structured predicate scoping which related rows the aggregate spans (required for any/all, where it is the quantified predicate). A portable attr.filter object (eq/ne/in/isNull with and/or), desugared to canonical { field: { op: value } } at parse time; codegen renders it per target (e.g. SQL FILTER (WHERE ...) or SQLite CASE WHEN for a relational view)." }, - { "type": "attr", "subType": "boolean", "name": "distinct", "min": 0, "max": 1, "description": "Set (collect-only) to dedupe collected values (set semantics)." }, + { "type": "attr", "subType": "boolean", "name": "distinct", "min": 0, "max": 1, "description": "Set (collect-only) to dedupe collected values (set semantics). Not supported on a whole-object collect (@of omitted): it is a guaranteed no-op whenever the value object carries the primary key, and a silent no-op is worse than a refusal." }, { "type": "attr", "subType": "string", "name": "orderBy", "isArray": true, "min": 0, "max": 1, "description": "Ordering keys as 'field[:asc|desc]' (default asc) over the related entity's fields; nulls sort last. On @agg:collect sets element order (non-distinct only); on origin.first (required) selects the row. Semantic — carries no SQL syntax." } ] }, diff --git a/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java b/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java index 7fe2cb0cb..3bdb034a1 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java +++ b/server/java/metadata/src/main/java/com/metaobjects/ErrorCode.java @@ -366,6 +366,34 @@ public enum ErrorCode { */ ERR_FILTERABLE_UNSUPPORTED_SUBTYPE, + /** + * #335 Half B — a field carries {@code @sortable: true} but is an array or + * its subtype has no filter-operator band (e.g. {@code field.object}), the + * same subtype-support signal {@code @filterable} uses. Generating a sort + * entry for it would emit an ORDER BY no dialect can execute. + */ + ERR_SORTABLE_UNSUPPORTED_SUBTYPE, + + /** + * #335 Half A — a whole-object {@code @agg:collect} (no {@code @of}; the carrying + * {@code field.object} rolls related rows up as its declared {@code @objectRef} value + * object) is malformed: the carrier is not a {@code field.object} with {@code @objectRef}, + * {@code @via} is absent, {@code @distinct} is declared (refused — a no-op whenever the + * value object carries the primary key), an {@code @orderBy} key does not resolve against + * the {@code @via} TERMINAL entity, or a member's declared type disagrees with the matched + * terminal field's. Distinct from {@code ERR_INVALID_ORIGIN} so a conformance fixture can + * tell this arm from a loader that still requires {@code @of}. + */ + ERR_COLLECT_WHOLE_OBJECT, + + /** + * #335 Half A — a whole-object {@code @agg:collect}'s value-object member has no + * matching field (by name) on the {@code @via} terminal entity. The lowering + * projects exactly the declared members; failing open here is how #270 turned a + * curated value object into the full entity. + */ + ERR_COLLECT_MEMBER_UNRESOLVED, + /** * ADR-0023 Decision 2: a registration ({@code register}/{@code extendType}/ * {@code registerCommonAttribute}/{@code addConstraint}/{@code registerType}/ diff --git a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java index c2bc56998..0ea9f4759 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java +++ b/server/java/metadata/src/main/java/com/metaobjects/loader/ValidationPhase.java @@ -230,6 +230,7 @@ public static void run(MetaRoot root, MetaDataLoader loader) { pass(collected, () -> validateTemplates(root)); pass(collected, () -> validateEntityHasPrimaryIdentity(root, loader)); pass(collected, () -> validateFilterableHasSupportedOps(root)); + pass(collected, () -> validateSortableHasSupportedSubtype(root)); pass(collected, () -> validateIndexLookupFields(root)); // The capability ledger's closed status enum (requirements-as-metadata // ruling, Amendment 3) — the loader owns what is UNCONDITIONAL. @@ -2132,7 +2133,7 @@ private static void validateOriginNode(MetaRoot root, MetaObject obj, String via = origin.getVia(); if (via != null && !via.isEmpty()) { java.util.List hops = - validateViaPath(via, root, obj, field.getName(), origin.getSource()); + validateViaPath(via, root, obj, field.getName(), origin.getSource()).hops(); checkPassthroughCardinality(hops, obj, field.getName(), origin.getSource()); } else if (!isValueHost) { // FR-024 §6 — no @via: derive the base entity; a @from targeting the @@ -2248,18 +2249,94 @@ private static void validateOriginNode(MetaRoot root, MetaObject obj, ErrorCode.ERR_INVALID_ORIGIN, src); } java.util.List hops = - validateViaPath(via, root, obj, field.getName(), src); + validateViaPath(via, root, obj, field.getName(), src).hops(); checkAggregateCardinality(hops, obj, field.getName(), src); return; } - // --- count/sum/avg/min/max/collect: @of REQUIRED --- + // --- @of: REQUIRED for count/sum/avg/min/max; OPTIONAL for collect --- + // #335 — an @of-absent collect is a WHOLE-OBJECT rollup: collect the related + // rows as an array of the field's declared @objectRef value object rather than + // an array of one scalar column. if (!ofPresent) { - throw new MetaDataException( - ErrorMessageConstants.ERR_INVALID_ORIGIN - + ": origin.aggregate on " + obj.getName() + "." + field.getName() - + ": missing @of.", - ErrorCode.ERR_INVALID_ORIGIN, src); + if (!isCollect) { + throw new MetaDataException( + ErrorMessageConstants.ERR_INVALID_ORIGIN + + ": origin.aggregate on " + obj.getName() + "." + field.getName() + + ": missing @of.", + ErrorCode.ERR_INVALID_ORIGIN, src); + } + // Whole-object rollup. The carrying field must be a field.object naming a + // value object, and @via must be explicit (there is no @of entity to infer + // the single-hop relation from). + // ADR-0039: resolving (hasMetaAttr/getMetaAttr default to includeParentData) + // — @objectRef may be inherited via extends. + Object objectRefVal = field.hasMetaAttr(ObjectField.ATTR_OBJECTREF) + ? field.getMetaAttr(ObjectField.ATTR_OBJECTREF).getValue() + : null; + String objectRef = objectRefVal instanceof String ? (String) objectRefVal : null; + if (!ObjectField.SUBTYPE_OBJECT.equals(field.getSubType()) + || objectRef == null || objectRef.isEmpty()) { + throw new MetaDataException( + ErrorMessageConstants.ERR_COLLECT_WHOLE_OBJECT + + ": origin.aggregate @agg:collect on " + obj.getName() + "." + field.getName() + + ": @of is omitted, so this is a whole-object rollup — the carrying field" + + " must be a field.object declaring @objectRef (add @of to collect a" + + " single column instead).", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, src); + } + // #210's value-only rule is PAYLOAD-scoped and never reaches a projection-hosted + // field, so this branch enforces it itself. Without it an @objectRef to an entity + // silently rolls up the FULL entity — the #270 shape, this time baked into DDL. + // ADR-0042 — a bare @objectRef resolves in the DECLARING owner's package. + String refPkg = obj.getPackage() == null ? "" : obj.getPackage(); + MetaObject refTarget = resolveRootObject(root, objectRef, refPkg); + if (refTarget != null && !MetaObject.SUBTYPE_VALUE.equals(refTarget.getSubType())) { + throw new MetaDataException( + "ERR_SUBTYPE_RULE_VIOLATION" + + ": origin.aggregate @agg:collect on " + obj.getName() + "." + field.getName() + + ": @objectRef '" + objectRef + "' resolves to " + + MetaObject.TYPE_OBJECT + "." + refTarget.getSubType() + + " — a whole-object rollup must target an object.value (#210, ADR-0028).", + ErrorCode.ERR_SUBTYPE_RULE_VIOLATION, src); + } + // ADR-0039: own — origin.* never inherits (ADR-0029); getVia() reads the + // origin's own @via, as every other arm of this method does. + String woVia = origin.getVia(); + if (woVia == null || woVia.isEmpty()) { + throw new MetaDataException( + ErrorMessageConstants.ERR_COLLECT_WHOLE_OBJECT + + ": origin.aggregate @agg:collect on " + obj.getName() + "." + field.getName() + + ": @via is required on a whole-object rollup — there is no @of entity" + + " to infer the relationship from.", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, src); + } + // @distinct is refused on the object form. It is NOT an engine limit (both + // engines dedupe JSON objects); it is a guaranteed no-op whenever the value + // object carries the entity's primary key, which is the common case, and a + // silent no-op is worse than a refusal. + if (hasDistinct) { + throw new MetaDataException( + ErrorMessageConstants.ERR_COLLECT_WHOLE_OBJECT + + ": origin.aggregate @agg:collect on " + obj.getName() + "." + field.getName() + + ": @distinct is not supported on a whole-object rollup (it is a no-op" + + " whenever the value object carries the primary key).", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, src); + } + // One walk yields both the hops (cardinality) and the terminal entity + // (@orderBy keys, member resolution). An invalid @via throws out of + // validateViaPath with its own error, so nothing downstream runs and no + // second, misleadingly-scoped error is produced. + WalkedViaPath woWalked = validateViaPath(woVia, root, obj, field.getName(), src); + checkAggregateCardinality(woWalked.hops(), obj, field.getName(), src); + // @orderBy keys resolve against the @via TERMINAL entity, not @of. + validateOrderByKeys(originOrderBy(origin), woWalked.terminal(), obj, field.getName(), + "origin.aggregate @agg:collect", src, + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, ErrorMessageConstants.ERR_COLLECT_WHOLE_OBJECT); + if (refTarget != null) { + checkCollectMembers(refTarget, woWalked.terminal(), obj, field, src); + } + return; } OriginTarget ofTarget = validateFromOrOfPath(of, root, obj, field.getName(), "origin.aggregate.@of", src); @@ -2283,7 +2360,7 @@ private static void validateOriginNode(MetaRoot root, MetaObject obj, String via = origin.getVia(); if (via != null && !via.isEmpty()) { java.util.List hops = - validateViaPath(via, root, obj, field.getName(), src); + validateViaPath(via, root, obj, field.getName(), src).hops(); checkAggregateCardinality(hops, obj, field.getName(), src); return; } @@ -2392,7 +2469,7 @@ private static void validateOriginNode(MetaRoot root, MetaObject obj, String via = origin.getVia(); if (via != null && !via.isEmpty()) { java.util.List hops = - validateViaPath(via, root, obj, field.getName(), src); + validateViaPath(via, root, obj, field.getName(), src).hops(); checkAggregateCardinality(hops, obj, field.getName(), src); } else { // (A value host never reaches here — the #210 assembly-origin @@ -2474,6 +2551,72 @@ private static List originOrderBy(MetaOrigin origin) { return java.util.Collections.emptyList(); } + /** + * A field's declared type on BOTH axes — {@code field.} plus {@code []} when it is + * an array. ADR-0039: {@code isArrayType()} (resolving), so array-ness inherited via + * {@code extends} counts. Mirrors the TS {@code _typeLabel}. + */ + private static String typeLabel(MetaField field) { + return "field." + field.getSubType() + (field.isArrayType() ? "[]" : ""); + } + + /** + * #335 — a whole-object {@code @agg:collect} projects EXACTLY the declared value object's + * members, matched BY NAME against the {@code @via} terminal entity's fields: + *
    + *
  • an unmatched member is an error, never a silent drop. Failing open here is how #270 + * turned a curated value object into the full entity, invisible in a diff because the + * metadata still read as curated.
    • + *
  • a matched member must agree on BOTH type axes (#185 type-preserving doctrine), so a + * scalar member cannot bind an array field or vice versa.
    • + *
+ * Both refusals carry a whole-object-specific code — ERR_COLLECT_MEMBER_UNRESOLVED for the + * unmatched member, ERR_COLLECT_WHOLE_OBJECT for the type disagreement. The latter is + * deliberately NOT the scalar arm's ERR_INVALID_ORIGIN: a loader that still requires + * {@code @of} rejects this metadata with ERR_INVALID_ORIGIN too, so sharing the code would + * make a corpus fixture pass on a port that implements nothing. + * Mirrors the TS {@code _checkCollectMembers}. + */ + private static void checkCollectMembers(MetaObject refTarget, MetaObject terminal, + MetaObject obj, MetaField field, + com.metaobjects.source.ErrorSource src) { + // ADR-0039: resolving (includeParentData=true) — a value object may inherit members via + // extends, and the terminal entity may inherit fields; own-only would silently skip + // inherited members, which is exactly the #270 bug class this guards. + java.util.List> terminalFields = new java.util.ArrayList<>(); + for (MetaData c : terminal.getChildren(MetaData.class, true)) { + if (c instanceof MetaField) terminalFields.add((MetaField) c); + } + for (MetaData c : refTarget.getChildren(MetaData.class, true)) { + if (!(c instanceof MetaField)) continue; + MetaField member = (MetaField) c; + MetaField match = null; + for (MetaField f : terminalFields) { + if (nameMatches(f, member.getName())) { match = f; break; } + } + if (match == null) { + throw new MetaDataException( + ErrorMessageConstants.ERR_COLLECT_MEMBER_UNRESOLVED + + ": origin.aggregate @agg:collect on " + obj.getName() + "." + field.getName() + + ": value-object member '" + member.getName() + + "' has no matching field on '" + terminal.getName() + + "' — a whole-object rollup projects exactly the declared members.", + ErrorCode.ERR_COLLECT_MEMBER_UNRESOLVED, src); + } + String memberLabel = typeLabel(member); + String matchLabel = typeLabel(match); + if (!memberLabel.equals(matchLabel)) { + throw new MetaDataException( + ErrorMessageConstants.ERR_COLLECT_WHOLE_OBJECT + + ": origin.aggregate @agg:collect on " + obj.getName() + "." + field.getName() + + ": value-object member '" + member.getName() + "' is " + memberLabel + + " but '" + terminal.getName() + "." + match.getName() + "' is " + matchLabel + + " — a whole-object rollup preserves each member's type.", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, src); + } + } + } + /** * #195 — validate {@code @orderBy} keys ({@code 'field[:asc|desc]'}) resolve * against the RELATED entity's effective fields (the entity reached via @@ -2482,10 +2625,22 @@ private static List originOrderBy(MetaOrigin origin) { * {@code @agg:collect} (element order) and {@code origin.first} (row selection). * A {@code null} related entity means a prior error already fired — skip silently. * Mirrors the TS {@code _validateOrderByKeys}. + * + *

{@code code}/{@code codeName} let #335's whole-object {@code @agg:collect} arm report + * ERR_COLLECT_WHOLE_OBJECT instead. The 6-arg overload keeps ERR_INVALID_ORIGIN, so the + * scalar {@code @of} and {@code origin.first} call sites are byte-identical.

*/ private static void validateOrderByKeys(List orderBy, MetaObject relatedEntity, MetaObject obj, String fieldName, String label, com.metaobjects.source.ErrorSource originSource) { + validateOrderByKeys(orderBy, relatedEntity, obj, fieldName, label, originSource, + ErrorCode.ERR_INVALID_ORIGIN, ErrorMessageConstants.ERR_INVALID_ORIGIN); + } + + private static void validateOrderByKeys(List orderBy, MetaObject relatedEntity, + MetaObject obj, String fieldName, String label, + com.metaobjects.source.ErrorSource originSource, + ErrorCode code, String codeName) { if (orderBy == null || relatedEntity == null) return; for (String raw : orderBy) { if (raw == null) continue; @@ -2502,18 +2657,18 @@ private static void validateOrderByKeys(List orderBy, MetaObject related } if (target == null) { throw new MetaDataException( - ErrorMessageConstants.ERR_INVALID_ORIGIN + codeName + ": " + label + " on " + obj.getName() + "." + fieldName + ": @orderBy key \"" + raw + "\" — no such field \"" + key + "\" on " + relatedEntity.getName() + ".", - ErrorCode.ERR_INVALID_ORIGIN, originSource); + code, originSource); } if (dir != null && !SORT_ORDER_VALUES.contains(dir)) { throw new MetaDataException( - ErrorMessageConstants.ERR_INVALID_ORIGIN + codeName + ": " + label + " on " + obj.getName() + "." + fieldName + ": @orderBy key \"" + raw + "\" — direction must be one of asc|desc.", - ErrorCode.ERR_INVALID_ORIGIN, originSource); + code, originSource); } } } @@ -3017,8 +3172,24 @@ private static void validateFilterableHasSupportedOps(MetaRoot root) { : (v instanceof String) ? "true".equalsIgnoreCase((String) v) : false; if (!filterable) continue; - if (subtypeSupportsFiltering(field.getSubType())) continue; String objName = obj.getShortName() != null ? obj.getShortName() : obj.getName(); + + // #335 Half B — an ARRAY field has no operator band either. Every + // FR-009 operator (eq/ne/gt/gte/lt/lte/in/like/isNull) is a scalar + // comparison; none applies to a collection column. Same reason as + // the subtype check below, so same code. + // ADR-0039: isArrayType(), the resolving accessor, never isArray(). + if (field.isArrayType()) { + throw new MetaDataException( + ErrorMessageConstants.ERR_FILTERABLE_UNSUPPORTED_SUBTYPE + + ": field \"" + objName + "." + field.getShortName() + + "\" has @filterable: true but is an array (isArray: true)." + + " No filter operator applies to a collection column." + + " Remove @filterable from this field.", + ErrorCode.ERR_FILTERABLE_UNSUPPORTED_SUBTYPE, field.getSource()); + } + + if (subtypeSupportsFiltering(field.getSubType())) continue; throw new MetaDataException( ErrorMessageConstants.ERR_FILTERABLE_UNSUPPORTED_SUBTYPE + ": field \"" + objName + "." + field.getShortName() @@ -3031,6 +3202,47 @@ private static void validateFilterableHasSupportedOps(MetaRoot root) { } } + // ========================================================================= + // @sortable on an array field or unsupported subtype — error pass (#335 Half B) + // + // @sortable defaults FROM @filterable, so it is checked independently only + // when explicit — nothing validated it before, while @filterable has had a + // hard error since SP-H Unit9. A @sortable array/unsupported-subtype column + // emits a sort entry over a column no dialect can ORDER BY meaningfully. + // → ERR_SORTABLE_UNSUPPORTED_SUBTYPE. + // ========================================================================= + + private static void validateSortableHasSupportedSubtype(MetaRoot root) { + for (MetaData rootChild : root.getChildren(MetaData.class, false)) { + if (!(rootChild instanceof MetaObject)) continue; + MetaObject obj = (MetaObject) rootChild; + // Effective fields (includes inherited via extends:/super:). + for (MetaField field : obj.getChildren(MetaField.class, true)) { + if (!field.hasMetaAttr(MetaField.ATTR_SORTABLE, true)) continue; + Object v = field.getMetaAttr(MetaField.ATTR_SORTABLE, true).getValue(); + boolean sortable = + (v instanceof Boolean) ? (Boolean) v + : (v instanceof String) ? "true".equalsIgnoreCase((String) v) + : false; + if (!sortable) continue; + // ADR-0039: isArrayType(), the resolving accessor, never isArray(). + boolean isArray = field.isArrayType(); + if (!isArray && subtypeSupportsFiltering(field.getSubType())) continue; + + String objName = obj.getShortName() != null ? obj.getShortName() : obj.getName(); + String reason = isArray + ? "is an array (isArray: true) — a collection column has no ordering." + : "its subtype \"" + field.getSubType() + "\" cannot be ordered."; + throw new MetaDataException( + ErrorMessageConstants.ERR_SORTABLE_UNSUPPORTED_SUBTYPE + + ": field \"" + objName + "." + field.getShortName() + + "\" has @sortable: true but " + reason + + " Remove @sortable from this field.", + ErrorCode.ERR_SORTABLE_UNSUPPORTED_SUBTYPE, field.getSource()); + } + } + } + /** True iff {@code subType} has a canonical filter-operator band. */ private static boolean subtypeSupportsFiltering(String st) { return com.metaobjects.query.FilterOps.supportsFiltering(st); @@ -3448,7 +3660,15 @@ private static OriginTarget validateFromOrOfPath(String pathAttr, MetaRoot root, * {@code @references} (FR-024). The resolved target must be an entity at root, * which becomes the next hop's current entity. */ - private static java.util.List validateViaPath(String viaAttr, MetaRoot root, + /** + * A fully-walked {@code @via} path: the relationship hop nodes in path order, plus the + * entity they terminate at. Returned as a pair (rather than re-walking for the terminal) + * because a second walk means a second copy of the ADR-0042 package-resolution rule — + * mirrors the TS {@code WalkedViaPath}. + */ + private record WalkedViaPath(java.util.List hops, MetaObject terminal) { } + + private static WalkedViaPath validateViaPath(String viaAttr, MetaRoot root, MetaObject projection, String fieldName, com.metaobjects.source.ErrorSource envelope) { // FR5d — referrer is `::` (matches @@ -3541,7 +3761,8 @@ private static java.util.List validateViaPath(String viaAttr, MetaRoot hops.add(hop); currentObj = nextObj; } - return hops; + // currentObj is the terminal: every earlier exit threw. + return new WalkedViaPath(hops, currentObj); } // ========================================================================= diff --git a/server/java/metadata/src/main/java/com/metaobjects/registry/RegistryManifest.java b/server/java/metadata/src/main/java/com/metaobjects/registry/RegistryManifest.java index 6488804d7..48ca69dc1 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/registry/RegistryManifest.java +++ b/server/java/metadata/src/main/java/com/metaobjects/registry/RegistryManifest.java @@ -274,7 +274,7 @@ public static List metamodelProviders() { * the 1.0 cut will flip it to {@code "1.0"}. Mirrors the TS reference's * {@code METAMODEL_VERSION} constant. */ - public static final String METAMODEL_VERSION = "0.11"; + public static final String METAMODEL_VERSION = "0.12"; // ------------------------------------------------------------------ // SP-G Phase1 Units2-3 — manifest emitter exclusions (documented, uniform diff --git a/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java b/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java index 71065f859..20aa9f624 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java +++ b/server/java/metadata/src/main/java/com/metaobjects/util/ErrorMessageConstants.java @@ -209,6 +209,37 @@ private ErrorMessageConstants() { */ public static final String ERR_FILTERABLE_UNSUPPORTED_SUBTYPE = "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE"; + /** + * #335 Half B — a field carries {@code @sortable: true} but is an array, or its + * subtype has no filter-operator band (the same band {@code @sortable} borrows + * its ordering test from). A sort entry over such a column has no dialect that + * can order it meaningfully. + * Cross-language contract: {@code ERR_SORTABLE_UNSUPPORTED_SUBTYPE}. + */ + public static final String ERR_SORTABLE_UNSUPPORTED_SUBTYPE = "ERR_SORTABLE_UNSUPPORTED_SUBTYPE"; + + /** + * #335 Half A — a whole-object {@code @agg:collect} (no {@code @of}; the carrying + * {@code field.object} rolls related rows up as its declared {@code @objectRef} value + * object) is malformed: the carrier is not a {@code field.object} with {@code @objectRef}, + * {@code @via} is absent, {@code @distinct} is declared (refused — a no-op whenever the + * value object carries the primary key), an {@code @orderBy} key does not resolve against + * the {@code @via} TERMINAL entity, or a member's declared type disagrees with the matched + * terminal field's. Distinct from {@code ERR_INVALID_ORIGIN} so a conformance fixture can + * tell this arm from a loader that still requires {@code @of}. + * Cross-language contract: {@code ERR_COLLECT_WHOLE_OBJECT}. + */ + public static final String ERR_COLLECT_WHOLE_OBJECT = "ERR_COLLECT_WHOLE_OBJECT"; + + /** + * #335 Half A — a whole-object {@code @agg:collect}'s value-object member has no + * matching field (by name) on the {@code @via} terminal entity. The lowering + * projects exactly the declared members; failing open here is how #270 turned a + * curated value object into the full entity. + * Cross-language contract: {@code ERR_COLLECT_MEMBER_UNRESOLVED}. + */ + public static final String ERR_COLLECT_MEMBER_UNRESOLVED = "ERR_COLLECT_MEMBER_UNRESOLVED"; + /** * Error code emitted when a {@code layout.dataGrid @defaultSortField} value * does not name a real field on the owning entity. diff --git a/server/python/src/metaobjects/errors.py b/server/python/src/metaobjects/errors.py index 10d88774a..ed6b4459f 100644 --- a/server/python/src/metaobjects/errors.py +++ b/server/python/src/metaobjects/errors.py @@ -156,6 +156,23 @@ class ErrorCode(str, Enum): # SP-H Unit9 — @filterable: true on a field subtype with no filter-operator # band (e.g. field.object). Would silently generate an empty-ops filter. ERR_FILTERABLE_UNSUPPORTED_SUBTYPE = "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE" + # #335 Half B — @sortable: true on an array field or a subtype with no + # filter-operator band (e.g. field.object). Would silently emit a sort + # entry over a column no dialect can ORDER BY. + ERR_SORTABLE_UNSUPPORTED_SUBTYPE = "ERR_SORTABLE_UNSUPPORTED_SUBTYPE" + # #335 Half A — a whole-object @agg:collect (no @of; the carrying field.object + # rolls related rows up as its declared @objectRef value object) is malformed: + # carrier is not a field.object with @objectRef, @via absent, @distinct declared + # (refused — a no-op whenever the value object carries the primary key), an + # @orderBy key not on the @via TERMINAL entity, or a member's declared type + # disagreeing with the matched terminal field's. Distinct from ERR_INVALID_ORIGIN + # so a fixture can tell this arm from a loader that still requires @of. + ERR_COLLECT_WHOLE_OBJECT = "ERR_COLLECT_WHOLE_OBJECT" + # #335 Half A — a whole-object @agg:collect's value-object member has no + # matching field (by name) on the @via terminal entity. The lowering + # projects exactly the declared members; failing open here is how #270 + # turned a curated value object into the full entity. + ERR_COLLECT_MEMBER_UNRESOLVED = "ERR_COLLECT_MEMBER_UNRESOLVED" # ADR-0023 — a registration was attempted against a registry sealed after its # agreed metamodel-provider bootstrap. Codegen cannot invent metamodel attrs. ERR_REGISTRY_SEALED = "ERR_REGISTRY_SEALED" diff --git a/server/python/src/metaobjects/loader/validation_passes.py b/server/python/src/metaobjects/loader/validation_passes.py index 4b49a5686..f3b99a28c 100644 --- a/server/python/src/metaobjects/loader/validation_passes.py +++ b/server/python/src/metaobjects/loader/validation_passes.py @@ -26,6 +26,7 @@ FIELD_ATTR_INT_VALUE_MAP, FIELD_ATTR_OBJECT_REF, FIELD_ATTR_REQUIRED, + FIELD_ATTR_SORTABLE, FIELD_ATTR_STORAGE, FIELD_ATTR_VALUE_TYPE, FIELD_ATTR_VALUES, @@ -230,8 +231,10 @@ def run_validations( _validate_identity_passthrough(root, errors) _validate_max_occurs(root, registry, errors) _validate_filterable_has_index(root, warnings) - # SP-H Unit9 — @filterable on a subtype with no operator band → error. + # SP-H Unit9 — @filterable on a subtype with no operator band, or an array → error. _validate_filterable_has_supported_ops(root, errors) + # #335 Half B — @sortable on an array field, or a subtype with no operator band → error. + _validate_sortable_has_supported_subtype(root, errors) _validate_index_lookup_fields(root, errors) @@ -1581,6 +1584,16 @@ def _validate_entity_field_ref( return (entity, field_node) +class WalkedViaPath(NamedTuple): + """A fully-walked ``@via`` path: the relationship hop nodes in path order, + plus the entity they terminate at. Returned as a pair (rather than re-walking + for the terminal) because a second walk means a second copy of the ADR-0042 + package-resolution rule — mirrors the TS ``WalkedViaPath``.""" + + hops: list[MetaData] + terminal: MetaData + + def _validate_via_path( via: str, context: str, @@ -1589,12 +1602,15 @@ def _validate_via_path( errors: list[MetaError], origin_node: MetaData, referrer: str, -) -> list[MetaData] | None: +) -> WalkedViaPath | None: """Validate a dotted relationship path 'Entity.rel1[.rel2...]'. - Returns the walked relationship hop nodes (in path order) on full success - (FR-024 B5 runs the cardinality checks over them); appends ERR_INVALID_ORIGIN - and returns None if not. + Returns the walked relationship hop nodes (in path order) together with the + TERMINAL entity node on full success (FR-024 B5 runs the cardinality checks + over the hops; #335's whole-object @agg:collect has no @of entity, so its + @orderBy keys and value-object members resolve against the terminal); + appends ERR_INVALID_ORIGIN and returns None if not. ``terminal`` is defined + exactly when ``hops`` is — every early exit returns None. *origin_node* carries the parse-time envelope (files/json_path); *referrer* is the canonical referrer FQN (``::``) attached @@ -1692,7 +1708,8 @@ def _validate_via_path( hops.append(rel_node) current_entity = next_entity - return hops + # current_entity is the terminal: every earlier exit returned None. + return WalkedViaPath(hops, current_entity) # --------------------------------------------------------------------------- @@ -2001,6 +2018,61 @@ def _check_passthrough_type( ) +def _type_label(field: MetaData) -> str: + """A field's declared type on BOTH axes — ``field.`` plus ``[]`` when it is + an array. ADR-0039: resolved_is_array(), so array-ness inherited via extends counts. + Mirrors the TS ``_typeLabel``.""" + return f"field.{field.sub_type}{'[]' if field.resolved_is_array() else ''}" + + +def _check_collect_members( + ref_target: MetaData, + terminal: MetaData, + obj: MetaData, + field: MetaData, + src: object, + errors: list[MetaError], +) -> None: + """#335 — a whole-object @agg:collect projects EXACTLY the declared value object's + members, matched BY NAME against the @via terminal entity's fields: + + - an unmatched member is an error, never a silent drop. Failing open here is how + #270 turned a curated value object into the full entity, invisible in a diff + because the metadata still read as curated. + - a matched member must agree on BOTH type axes (#185 type-preserving doctrine), + so a scalar member cannot bind an array field or vice versa. + + Both refusals carry a whole-object-specific code — ERR_COLLECT_MEMBER_UNRESOLVED for + the unmatched member, ERR_COLLECT_WHOLE_OBJECT for the type disagreement. The latter + is deliberately NOT the scalar arm's ERR_INVALID_ORIGIN: a loader that still requires + @of rejects this metadata with ERR_INVALID_ORIGIN too, so sharing the code would make + a corpus fixture pass on a port that implements nothing. + + Mirrors the TS ``_checkCollectMembers``.""" + # ADR-0039: resolving — a value object may inherit members via extends, and the + # terminal entity may inherit fields; own-only would silently skip inherited + # members, which is exactly the #270 bug class this guards. + terminal_fields = [c for c in terminal.children() if c.type == TYPE_FIELD] + for member in (c for c in ref_target.children() if c.type == TYPE_FIELD): + match = next((f for f in terminal_fields if f.name == member.name), None) + if match is None: + errors.append(MetaError( + f"origin.aggregate @agg:collect on {obj.name}.{field.name}: value-object " + f"member '{member.name}' has no matching field on '{terminal.name}' — a " + f"whole-object rollup projects exactly the declared members.", + ErrorCode.ERR_COLLECT_MEMBER_UNRESOLVED, envelope=src)) + continue + member_label = _type_label(member) + match_label = _type_label(match) + if member_label != match_label: + errors.append(MetaError( + f"origin.aggregate @agg:collect on {obj.name}.{field.name}: value-object " + f"member '{member.name}' is {member_label} but " + f"'{terminal.name}.{match.name}' is {match_label} — a whole-object rollup " + f"preserves each member's type.", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, envelope=src)) + + def _validate_order_by_keys( order_by: object, related_entity: MetaData | None, @@ -2009,12 +2081,17 @@ def _validate_order_by_keys( label: str, origin_source: object, errors: list[MetaError], + code: ErrorCode = ErrorCode.ERR_INVALID_ORIGIN, ) -> None: """#195 — validate that ``@orderBy`` keys ('field[:asc|desc]') resolve against the RELATED entity's effective fields (the entity reached via @via/@of), and that any direction suffix is asc/desc. Shared by @agg:collect (element order) and origin.first (row selection). A missing related entity means a prior error already fired — skip. - Mirrors the TS _validateOrderByKeys.""" + Mirrors the TS _validateOrderByKeys. + + *code* lets #335's whole-object @agg:collect arm report ERR_COLLECT_WHOLE_OBJECT + instead; it defaults to ERR_INVALID_ORIGIN so the scalar @of and origin.first call + sites keep their existing envelope byte-for-byte.""" if not isinstance(order_by, (list, tuple)) or related_entity is None: return for raw in order_by: @@ -2033,7 +2110,7 @@ def _validate_order_by_keys( MetaError( f'{label} on {obj.name}.{field_name}: @orderBy key "{raw}" — no such ' f'field "{key}" on {related_entity.name}.', - ErrorCode.ERR_INVALID_ORIGIN, + code, envelope=origin_source, ) ) @@ -2042,7 +2119,7 @@ def _validate_order_by_keys( MetaError( f'{label} on {obj.name}.{field_name}: @orderBy key "{raw}" — direction ' f"must be one of {'|'.join(SORT_ORDER_VALUES)}.", - ErrorCode.ERR_INVALID_ORIGIN, + code, envelope=origin_source, ) ) @@ -2136,7 +2213,8 @@ def _validate_origin_paths( ) via = origin.attr(ORIGIN_ATTR_VIA) if isinstance(via, str) and via: - hops = _validate_via_path(via, ctx, root, host_pkg, errors, origin, referrer) + walked = _validate_via_path(via, ctx, root, host_pkg, errors, origin, referrer) + hops = walked.hops if walked is not None else None if hops is not None: _check_passthrough_cardinality(hops, node.name, origin.source, errors) elif from_target is not None and not is_value_host: @@ -2232,20 +2310,94 @@ def _validate_origin_paths( f"explicit @via (a quantifier has no @of to infer the path from).", ErrorCode.ERR_INVALID_ORIGIN, envelope=src)) else: - hops = _validate_via_path(via, ctx, root, host_pkg, errors, origin, referrer) + walked = _validate_via_path(via, ctx, root, host_pkg, errors, origin, referrer) + hops = walked.hops if walked is not None else None if hops is not None: _check_aggregate_cardinality(hops, node.name, src, errors) continue - # --- count/sum/avg/min/max/collect: @of REQUIRED --- + # --- @of: REQUIRED for count/sum/avg/min/max; OPTIONAL for collect --- + # #335 — an @of-absent collect is a WHOLE-OBJECT rollup: collect the + # related rows as an array of the field's declared @objectRef value + # object rather than an array of one scalar column. if not of_present: - errors.append( - MetaError( - f"{ctx} is missing required attribute '@{ORIGIN_ATTR_OF}'", - ErrorCode.ERR_INVALID_ORIGIN, - envelope=src, + if not is_collect: + errors.append( + MetaError( + f"{ctx} is missing required attribute '@{ORIGIN_ATTR_OF}'", + ErrorCode.ERR_INVALID_ORIGIN, + envelope=src, + ) ) - ) + continue + # Whole-object rollup. The carrying field must be a field.object + # naming a value object, and @via must be explicit (there is no @of + # entity to infer the single-hop relation from). + # ADR-0039: resolving — @objectRef may be inherited via extends. + object_ref = node.get_meta_attr(FIELD_ATTR_OBJECT_REF) + if node.sub_type != FIELD_SUBTYPE_OBJECT or not isinstance(object_ref, str) \ + or not object_ref: + errors.append(MetaError( + f"origin.aggregate @agg:collect on {obj.name}.{node.name}: @of is " + f"omitted, so this is a whole-object rollup — the carrying field " + f"must be a field.object declaring @objectRef (add @of to collect a " + f"single column instead).", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, envelope=src)) + continue + # #210's value-only rule is PAYLOAD-scoped and never reaches a + # projection-hosted field, so this branch enforces it itself. + # Without it an @objectRef to an entity silently rolls up the FULL + # entity — the #270 shape, this time baked into DDL. + # ADR-0042 — a bare @objectRef resolves in the DECLARING owner's + # package (an inherited field resolves in the package that declared it). + ref_owner = node.parent or obj + ref_pkg = ref_owner.package or ref_owner.file_default_package or "" + ref_target = resolve_object_ref(root, object_ref, ref_pkg) + if ref_target is not None and ref_target.sub_type != OBJECT_SUBTYPE_VALUE: + errors.append(MetaError( + f"origin.aggregate @agg:collect on {obj.name}.{node.name}: " + f"@objectRef '{object_ref}' resolves to " + f"{TYPE_OBJECT}.{ref_target.sub_type} — a whole-object rollup must " + f"target an object.value (#210, ADR-0028).", + ErrorCode.ERR_SUBTYPE_RULE_VIOLATION, envelope=src)) + continue + # ADR-0039: own — origin.* never inherits (ADR-0029). + via_attr = origin.attr(ORIGIN_ATTR_VIA) + if not isinstance(via_attr, str) or not via_attr: + errors.append(MetaError( + f"origin.aggregate @agg:collect on {obj.name}.{node.name}: @via is " + f"required on a whole-object rollup — there is no @of entity to " + f"infer the relationship from.", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, envelope=src)) + continue + # @distinct is refused on the object form. It is NOT an engine limit + # (both engines dedupe JSON objects); it is a guaranteed no-op + # whenever the value object carries the entity's primary key, which + # is the common case, and a silent no-op is worse than a refusal. + if has_distinct: + errors.append(MetaError( + f"origin.aggregate @agg:collect on {obj.name}.{node.name}: @distinct " + f"is not supported on a whole-object rollup (it is a no-op whenever " + f"the value object carries the primary key).", + ErrorCode.ERR_COLLECT_WHOLE_OBJECT, envelope=src)) + continue + # One walk yields both the hops (cardinality) and the terminal entity + # (@orderBy keys, member resolution). An invalid @via (e.g. a single- + # segment "A") returns None having already pushed its own error, so + # everything downstream is skipped and no second, misleadingly-scoped + # error is emitted. + walked = _validate_via_path( + via_attr, ctx, root, host_pkg, errors, origin, referrer) + if walked is not None: + _check_aggregate_cardinality(walked.hops, node.name, src, errors) + # @orderBy keys resolve against the @via TERMINAL entity, not @of. + _validate_order_by_keys( + order_by, walked.terminal, obj, node.name, + "origin.aggregate @agg:collect", src, errors, + ErrorCode.ERR_COLLECT_WHOLE_OBJECT) + if ref_target is not None: + _check_collect_members( + ref_target, walked.terminal, obj, node, src, errors) continue # NOTE (FR-024 B6): NO extends/origin agreement on aggregates — # an aggregate computes something new (spec §4 is passthrough-only). @@ -2267,7 +2419,8 @@ def _validate_origin_paths( obj, node.name, "origin.aggregate @agg:collect", src, errors) via = origin.attr(ORIGIN_ATTR_VIA) if isinstance(via, str) and via: - hops = _validate_via_path(via, ctx, root, host_pkg, errors, origin, referrer) + walked = _validate_via_path(via, ctx, root, host_pkg, errors, origin, referrer) + hops = walked.hops if walked is not None else None if hops is not None: _check_aggregate_cardinality(hops, node.name, src, errors) continue @@ -2378,7 +2531,8 @@ def _resolve_field(name: str, _base: MetaData = base) -> str | None: # @via — explicit (validated + cardinality) or single-hop-unique inferred. via = origin.attr(ORIGIN_ATTR_VIA) if isinstance(via, str) and via: - hops = _validate_via_path(via, ctx, root, host_pkg, errors, origin, referrer) + walked = _validate_via_path(via, ctx, root, host_pkg, errors, origin, referrer) + hops = walked.hops if walked is not None else None if hops is not None: _check_aggregate_cardinality(hops, node.name, src, errors) elif of_target is not None: @@ -3129,6 +3283,24 @@ def _validate_filterable_has_supported_ops( for field in node.fields(): if field.attrs().get("filterable") is not True: continue + + # #335 Half B — an ARRAY field has no operator band either. Every + # FR-009 operator (eq/ne/gt/gte/lt/lte/in/like/isNull) is a scalar + # comparison; none applies to a collection column. Same reason as + # the subtype check below, so same code. + # ADR-0039: resolved_is_array(), never the own `is_array` flag. + if field.resolved_is_array(): + errors.append( + MetaError( + f'Field "{node.name}.{field.name}" has @filterable: true but is an array ' + f"(isArray: true). No filter operator applies to a collection column. " + f"Remove @filterable from this field.", + ErrorCode.ERR_FILTERABLE_UNSUPPORTED_SUBTYPE, + envelope=field.source, + ) + ) + continue + if ops_for_subtype(field.sub_type): continue errors.append( @@ -3143,6 +3315,46 @@ def _validate_filterable_has_supported_ops( ) +# --------------------------------------------------------------------------- +# Pass: @sortable on an array field or unsupported subtype (#335 Half B) +# --------------------------------------------------------------------------- +# @sortable defaults FROM @filterable, so it is checked independently only +# when explicit — nothing validated it before, while @filterable has had a +# hard error since SP-H Unit9. A @sortable JSON or array column emits a sort +# entry over a column no dialect can ORDER BY meaningfully. +# → ERR_SORTABLE_UNSUPPORTED_SUBTYPE. + + +def _validate_sortable_has_supported_subtype( + root: MetaData, + errors: list[MetaError], +) -> None: + for node in _walk(root): + if node.type != TYPE_OBJECT or not isinstance(node, MetaObject): + continue + for field in node.fields(): + if field.attrs().get(FIELD_ATTR_SORTABLE) is not True: + continue + # ADR-0039: resolved_is_array(), never the own `is_array` flag. + is_array = field.resolved_is_array() + if not is_array and ops_for_subtype(field.sub_type): + continue + + reason = ( + "is an array (isArray: true) — a collection column has no ordering." + if is_array + else f'its subtype "{field.sub_type}" cannot be ordered.' + ) + errors.append( + MetaError( + f'Field "{node.name}.{field.name}" has @sortable: true but {reason} ' + f"Remove @sortable from this field.", + ErrorCode.ERR_SORTABLE_UNSUPPORTED_SUBTYPE, + envelope=field.source, + ) + ) + + # --------------------------------------------------------------------------- # Pass: field.object @storage validation # --------------------------------------------------------------------------- diff --git a/server/python/src/metaobjects/registry_manifest.py b/server/python/src/metaobjects/registry_manifest.py index 741efafd0..905b2a3fb 100644 --- a/server/python/src/metaobjects/registry_manifest.py +++ b/server/python/src/metaobjects/registry_manifest.py @@ -255,7 +255,7 @@ def _sorted_children(definition: TypeDefinition) -> list[dict[str, object]]: return children -METAMODEL_VERSION = "0.11" +METAMODEL_VERSION = "0.12" """Rolled-up spec-version for the cross-port registry manifest. ``"0"`` = pre-1.0 / unstable (semver major-0). Flips to ``"1.0"`` at the diff --git a/server/python/src/metaobjects/spec_metamodel/origin.json b/server/python/src/metaobjects/spec_metamodel/origin.json index e6be6d33b..6ddd1dbb7 100644 --- a/server/python/src/metaobjects/spec_metamodel/origin.json +++ b/server/python/src/metaobjects/spec_metamodel/origin.json @@ -20,15 +20,15 @@ { "type": "origin", "subType": "aggregate", - "description": "A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup of @of).", + "description": "A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted).", "whenToUse": "A projection needs a value derived by reducing related rows — a count/sum/avg/min/max, a 'did any/every related row match' flag, or an array of collected values. Declare it instead of hand-writing the aggregate query — it stays consistent and regenerates.", - "rules": "@via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max/collect and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only.", + "rules": "@via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only. On @agg:collect @of is OPTIONAL: omitting it declares a WHOLE-OBJECT rollup, which collects each related row as the carrying field's declared @objectRef value object instead of one scalar column. A whole-object rollup requires a field.object carrying @objectRef, requires that @objectRef to name an object.value, requires an explicit @via (there is no @of entity to infer the path from), and refuses @distinct. Its @orderBy keys resolve against the @via TERMINAL entity, not the head or a middle hop. Its value-object members bind to the terminal entity's fields BY NAME — member name == terminal field name, deliberately NOT extends, so one value object stays collectable from two different entities — and every member must match a terminal field agreeing on BOTH field. and array-ness.", "children": [ - { "type": "attr", "subType": "string", "name": "agg", "min": 1, "max": 1, "allowedValues": ["count", "sum", "avg", "min", "max", "any", "all", "collect"], "description": "The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup of @of — the field must be isArray)." }, - { "type": "attr", "subType": "string", "name": "of", "min": 0, "max": 1, "description": "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max/collect; forbidden for any/all (which quantify over rows via @filter, not a column)." }, + { "type": "attr", "subType": "string", "name": "agg", "min": 1, "max": 1, "allowedValues": ["count", "sum", "avg", "min", "max", "any", "all", "collect"], "description": "The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted; the field must be isArray)." }, + { "type": "attr", "subType": "string", "name": "of", "min": 0, "max": 1, "description": "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max; OPTIONAL for collect, where absent means a whole-object rollup of the field's declared @objectRef value object; forbidden for any/all (which quantify over rows via @filter, not a column)." }, { "type": "attr", "subType": "string", "name": "via", "min": 0, "max": 1, "description": "Dotted relationship path from the base entity to the aggregated rows (e.g. 'Program.weeks' or 'Program.weeks.workouts'). May be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (FR-024, ADR-0029)." }, { "type": "attr", "subType": "filter", "name": "filter", "min": 0, "max": 1, "description": "Optional structured predicate scoping which related rows the aggregate spans (required for any/all, where it is the quantified predicate). A portable attr.filter object (eq/ne/in/isNull with and/or), desugared to canonical { field: { op: value } } at parse time; codegen renders it per target (e.g. SQL FILTER (WHERE ...) or SQLite CASE WHEN for a relational view)." }, - { "type": "attr", "subType": "boolean", "name": "distinct", "min": 0, "max": 1, "description": "Set (collect-only) to dedupe collected values (set semantics)." }, + { "type": "attr", "subType": "boolean", "name": "distinct", "min": 0, "max": 1, "description": "Set (collect-only) to dedupe collected values (set semantics). Not supported on a whole-object collect (@of omitted): it is a guaranteed no-op whenever the value object carries the primary key, and a silent no-op is worse than a refusal." }, { "type": "attr", "subType": "string", "name": "orderBy", "isArray": true, "min": 0, "max": 1, "description": "Ordering keys as 'field[:asc|desc]' (default asc) over the related entity's fields; nulls sort last. On @agg:collect sets element order (non-distinct only); on origin.first (required) selects the row. Semantic — carries no SQL syntax." } ] }, diff --git a/server/typescript/packages/codegen-ts/src/projection/build-projection-views.ts b/server/typescript/packages/codegen-ts/src/projection/build-projection-views.ts index 91de41102..cd8a850d9 100644 --- a/server/typescript/packages/codegen-ts/src/projection/build-projection-views.ts +++ b/server/typescript/packages/codegen-ts/src/projection/build-projection-views.ts @@ -315,10 +315,11 @@ function collectViewColumns( const out: ExpectedViewColumn[] = []; for (const c of spec.selectSpec.columns) { - // #195: the four new origin column kinds (predicateAgg/collectAgg/computed/first) do - // not resolve to a single (table, column) SqlType via the prefix rule — computed is - // an expression, first is a correlated subquery, and the array/boolean aggregate - // result types are richer than the OR-REPLACE prefix check models. Per this module's + // #195/#335: the five non-scalar origin column kinds (predicateAgg / collectAgg / + // collectObjectAgg / computed / first) do not resolve to a single (table, column) + // SqlType via the prefix rule — computed is an expression, first is a correlated + // subquery, and the array/jsonb/boolean aggregate result types are richer than the + // OR-REPLACE prefix check models. Per this module's // fail-safe doctrine (unknown → drop+create, never a wrong-but-confident replace), // an unknown column drops the whole list so migrate routes through a gated // drop+create. Precise native typing is a later phase. diff --git a/server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts b/server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts index 3654d3a52..94e9d7497 100644 --- a/server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts +++ b/server/typescript/packages/codegen-ts/src/projection/extract-view-spec.ts @@ -1,5 +1,6 @@ import { TYPE_FIELD, + FIELD_ATTR_OBJECT_REF, TYPE_IDENTITY, TYPE_ORIGIN, TYPE_RELATIONSHIP, @@ -886,6 +887,59 @@ function buildSelectSpec( continue; } + // #335 — @of ABSENT on collect is a WHOLE-OBJECT rollup. This arm sits ABOVE + // the `if (!of_) continue;` guard on purpose: that guard used to swallow this + // shape, so the metadata LOADED and the column was silently dropped from the + // CREATE VIEW while the generated type still declared the field. + // The related entity comes from @via's terminal hop, not from @of — exactly as + // the any/all arm above resolves it. + const wholeObjectCollect = + agg === AGG_COLLECT && origin.ownAttr(ORIGIN_AGGREGATE_ATTR_OF) === undefined; + if (wholeObjectCollect) { + // Every `continue` below is a loader-guaranteed impossibility (the load fails + // with ERR_COLLECT_WHOLE_OBJECT / ERR_SUBTYPE_RULE_VIOLATION / + // ERR_COLLECT_MEMBER_UNRESOLVED before codegen runs); they are defence for a + // caller that reached codegen without loading, never a silent-drop path. + const via = origin.ownAttr(ORIGIN_AGGREGATE_ATTR_VIA) as string | undefined; + if (!via) continue; + // ADR-0039: resolving — @objectRef may be inherited via extends. + const objectRef = field.attr(FIELD_ATTR_OBJECT_REF) as string | undefined; + if (!objectRef) continue; + const relatedName = viaTerminalEntity(via, root, projPkg); + if (!relatedName) continue; + const relatedEntity = resolveEntityRef(root, relatedName, projPkg); + if (!relatedEntity) continue; + const sourceAlias = findAliasInTree(joinTree, relatedEntity.resolutionKey()); + if (sourceAlias === undefined) continue; + const joinedPk = primaryKeyColumn(relatedEntity, ctx); + if (joinedPk === undefined) continue; + // ADR-0042 — a bare @objectRef resolves in the projection's package. + const vo = resolveObjectRef(root, objectRef, projPkg).node; + if (!vo) continue; + // ADR-0039: resolving — a value object may inherit members via extends, and the + // terminal entity may inherit fields. The VO's member list IS the exposure: a + // field the terminal has but the VO omits is deliberately not projected (#270). + const members: { memberName: string; sourceColumn: string }[] = []; + for (const m of vo.children().filter((c): c is MetaField => c.type === TYPE_FIELD)) { + const target = relatedEntity.fields().find((f) => f.name === m.name); + if (target === undefined) continue; // loader: ERR_COLLECT_MEMBER_UNRESOLVED + members.push({ memberName: m.name, sourceColumn: sourceColumnNameFor(target, ctx) }); + } + if (members.length === 0) continue; + columns.push({ + kind: "collectObjectAgg", + fieldName: field.name, + dbColAlias: dbCol, + sourceAlias, + joinedPkColumn: joinedPk, + members, + // @orderBy resolves against the @via TERMINAL entity, not @of — there is no + // @of entity. Empty ⇒ the emitter's default of related-PK ascending. + orderBy: resolveOrderByKeys(origin.ownAttr(ORIGIN_ATTR_ORDER_BY), relatedEntity, ctx), + }); + continue; + } + // collect + the scalar reduces (count/sum/avg/min/max) all name @of. const of_ = origin.ownAttr(ORIGIN_AGGREGATE_ATTR_OF) as string | undefined; if (!of_) continue; @@ -1003,6 +1057,9 @@ function buildSelectSpec( function isInflationSensitive(c: SelectColumn): boolean { if (c.kind === "aggregate") return c.agg === "sum" || c.agg === "avg"; if (c.kind === "collectAgg") return !c.distinct; + // #335 — a whole-object collect is ALWAYS non-distinct (the loader refuses @distinct + // on this form), so it is unconditionally inflation-sensitive. + if (c.kind === "collectObjectAgg") return true; return false; } @@ -1015,10 +1072,12 @@ function countManyBranches(joinTree: JoinTree): number { } function buildGroupBy(spec: SelectSpec): string[] { - // predicateAgg (bool_or/bool_and) and collectAgg (array_agg) are real aggregates and - // force GROUP BY too; computed/first are scalar-per-row and never grouped. + // predicateAgg (bool_or/bool_and), collectAgg (array_agg) and collectObjectAgg + // (jsonb_agg) are real aggregates and force GROUP BY too; computed/first are + // scalar-per-row and never grouped. const hasAgg = spec.columns.some( - (c) => c.kind === "aggregate" || c.kind === "predicateAgg" || c.kind === "collectAgg", + (c) => c.kind === "aggregate" || c.kind === "predicateAgg" + || c.kind === "collectAgg" || c.kind === "collectObjectAgg", ); if (!hasAgg) return []; return spec.columns diff --git a/server/typescript/packages/codegen-ts/src/projection/view-ddl-emit.ts b/server/typescript/packages/codegen-ts/src/projection/view-ddl-emit.ts index 91c2c1e69..3e4a36c49 100644 --- a/server/typescript/packages/codegen-ts/src/projection/view-ddl-emit.ts +++ b/server/typescript/packages/codegen-ts/src/projection/view-ddl-emit.ts @@ -68,6 +68,11 @@ function renderFilterCond(clause: ViewFilterClause, dialect: EmitOptions["dialec return `${lhs} ${op} ${sqlLiteral(clause.value, dialect)}`; } +/** Subquery alias for the json_each re-wrap in the SQLite whole-object collect. Scoped to + * its own scalar subquery, so it cannot collide with a JOIN-tree alias; named distinctly + * anyway so it is obvious in emitted DDL where it came from. */ +const JSON_EACH_ALIAS = "mo_je"; + /** * Render resolved ordering keys, applying the #195 nulls-last pin (`NULLS LAST` in * both directions — PG + SQLite ≥ 3.30). `alias` qualifies each key's column. @@ -200,6 +205,55 @@ function renderColumn(c: SelectColumn, options: EmitOptions, baseAlias: string): return `COALESCE(array_agg(${distinctKw}${src} ${orderClause}) FILTER (WHERE ${guard}), '{}') AS ${alias}`; } + if (c.kind === "collectObjectAgg") { + // #335 whole-object rollup. jsonb, not json: PG's `json` has neither an equality + // nor an ordering operator, so `json_agg(json_build_object(…) ORDER BY …)` does not + // run — verified against a real PG 15. + const guard = `${c.sourceAlias}.${quoteIfNeeded(c.joinedPkColumn)} IS NOT NULL`; + const pk = `${c.sourceAlias}.${quoteIfNeeded(c.joinedPkColumn)}`; + // Element order: the related entity's PK ascending by default — "value ascending" + // is meaningless for an object (and does not parse on PG json). An explicit + // @orderBy leads, with the PK appended as a tie-break so equal-order rows stay + // byte-deterministic. The SCALAR arm above deliberately keeps its no-tie-break + // behaviour: adding one there would alter emitted SQL for every existing project. + const orderClause = c.orderBy.length > 0 + ? `ORDER BY ${renderOrderKeys(c.orderBy, c.sourceAlias)}, ${pk} ASC` + : `ORDER BY ${pk} ASC`; + // The JSON key is the VO MEMBER name; the value reads the TERMINAL entity's + // physical column. Those two differ whenever a field carries @column. + const pairs = c.members + .map((m) => `'${m.memberName}', ${c.sourceAlias}.${quoteIfNeeded(m.sourceColumn)}`) + .join(", "); + // In-aggregate ORDER BY needs SQLite >= 3.44 — not a new constraint: the scalar + // collect above already emits it, and D1's baseline is pinned at 3.44.0. + if (dialect === "sqlite") { + // SQLite cannot do BOTH in-aggregate ORDER BY and JSON nesting in one call. + // Measured on SQLite 3.44.0, D1's pinned baseline: + // + // json_group_array(json_object(…)) -> nests correctly + // json_group_array(json_object(…) ORDER BY …) -> array of QUOTED STRINGS + // json_group_array(json(json_object(…)) ORDER BY …) -> array of QUOTED STRINGS + // + // The ORDER BY clause itself destroys the JSON subtype, and a json() wrapper on the + // argument does not survive it. Dropping ORDER BY is not an option: element order + // would stop being deterministic and an author's @orderBy would silently do nothing. + // + // So build the ordered array first (elements quoted), then re-wrap element by + // element through json_each — which iterates in ARRAY ORDER, so the ordering + // survives while json(value) restores each element's JSON subtype. Still a grouped + // LEFT JOIN: unlike origin.first this needs no correlation info, so a multi-hop + // @via lowers here exactly as a single-hop one does. + // + // Found by the real-engine probe in integration-tests. Emitted SQL text cannot show + // this, which is exactly why golden SQL is not evidence for new DDL. + const ordered = `COALESCE(json_group_array(json_object(${pairs}) ${orderClause}) FILTER (WHERE ${guard}), json_array())`; + return `(SELECT json_group_array(json(${JSON_EACH_ALIAS}.value)) FROM json_each(${ordered}) ${JSON_EACH_ALIAS}) AS ${alias}`; + } + // PG's jsonb_build_object already yields real jsonb, so jsonb_agg nests it correctly + // with no wrapper — verified against a real engine, not assumed by symmetry. + return `COALESCE(jsonb_agg(jsonb_build_object(${pairs}) ${orderClause}) FILTER (WHERE ${guard}), '[]'::jsonb) AS ${alias}`; + } + if (c.kind === "computed") { return `${renderExpr(c.expr, dialect)} AS ${alias}`; } diff --git a/server/typescript/packages/codegen-ts/src/projection/view-spec.ts b/server/typescript/packages/codegen-ts/src/projection/view-spec.ts index 40fadc264..295674823 100644 --- a/server/typescript/packages/codegen-ts/src/projection/view-spec.ts +++ b/server/typescript/packages/codegen-ts/src/projection/view-spec.ts @@ -124,6 +124,31 @@ export type SelectColumn = /** Element ordering over the @of entity's columns; empty ⇒ value-ascending default. */ readonly orderBy: readonly ViewOrderKey[]; } + | { + // #335 — origin.aggregate @agg:collect with NO @of — a WHOLE-OBJECT rollup: each + // related row is collected as the carrying field.object's declared @objectRef value + // object rather than as one scalar column. Lowered to + // COALESCE(jsonb_agg(jsonb_build_object(...) ORDER BY ASC) FILTER (WHERE + // joined.pk IS NOT NULL), '[]'::jsonb) on PG; json_group_array(json_object(...)) on + // SQLite. Empty set → []. Default element order is the RELATED entity's PK ascending + // — ordering rows by a serialized object is meaningless, and on PG `json` it does not + // even parse (no ordering operator), which is also why the column is jsonb not json. + // + // A separate kind rather than an arm of collectAgg: the payloads differ (a member + // list vs one source column), so a union would force every consumer to re-narrow. + // @distinct never appears — the loader refuses it on this form. + readonly kind: "collectObjectAgg"; + readonly fieldName: string; + readonly dbColAlias: string; + readonly sourceAlias: string; + readonly joinedPkColumn: string; // related entity's PK column — the LEFT-JOIN phantom guard + /** The declared value object's members, in declaration order. `memberName` is the + * emitted JSON key; `sourceColumn` is the TERMINAL entity's physical column it reads. + * The loader guarantees every member resolves (ERR_COLLECT_MEMBER_UNRESOLVED). */ + readonly members: readonly { readonly memberName: string; readonly sourceColumn: string }[]; + /** Element ordering over the @via terminal entity's columns; empty ⇒ PK ascending. */ + readonly orderBy: readonly ViewOrderKey[]; + } | { // #195 — origin.computed — a row-level value from the base entity's own fields via // a structured @expr tree (no related rows). Lowered by a tree-walk to a SQL scalar diff --git a/server/typescript/packages/codegen-ts/test/projection/collect-whole-object-ddl.test.ts b/server/typescript/packages/codegen-ts/test/projection/collect-whole-object-ddl.test.ts new file mode 100644 index 000000000..9beb42098 --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/projection/collect-whole-object-ddl.test.ts @@ -0,0 +1,132 @@ +// #335 — SQL lowering for a WHOLE-OBJECT `origin.aggregate @agg:collect` column. +// +// jsonb, not json, on Postgres: verified against PG 15 — `json` has neither an +// equality nor an ordering operator, so `json_agg(json_build_object(…) ORDER BY …)` +// does not run at all. `field.object` is already jsonb elsewhere in codegen. +// +// Element order defaults to the RELATED entity's PK ascending, not "value +// ascending": ordering rows by a serialized object is meaningless, and on PG json +// it does not even parse. An explicit @orderBy leads with the PK appended as a +// tie-break so equal-order rows stay byte-deterministic. + +import { describe, test, expect } from "bun:test"; +import type { ViewSpec, ViewOrderKey } from "../../src/projection/view-spec.js"; +import { emitViewDdl } from "../../src/projection/view-ddl-emit.js"; + +const OPTS = { baseTableName: "products", joinTables: { Supplier: "suppliers" } } as const; + +function wholeObject(orderBy: ViewOrderKey[] = []): ViewSpec { + return { + viewName: "v_product_summary", + joinTree: { + baseEntity: "Product", + baseAlias: "p", + joins: [ + { relationship: "suppliers", targetEntity: "Supplier", alias: "s", cardinality: "many", + fkColumn: "product_id", pkColumn: "id", referenceHolder: "target", joinType: "left", children: [] }, + ], + }, + selectSpec: { + columns: [ + { kind: "passthrough", fieldName: "id", dbColAlias: "id", sourceAlias: "p", sourceColumn: "id" }, + { + kind: "collectObjectAgg", + fieldName: "supplierBriefs", + dbColAlias: "supplier_briefs", + sourceAlias: "s", + joinedPkColumn: "id", + members: [ + { memberName: "id", sourceColumn: "id" }, + { memberName: "name", sourceColumn: "supplier_name" }, + ], + orderBy, + }, + ], + }, + groupBy: ["p.id"], + }; +} + +describe("emitViewDdl — #335 whole-object collect", () => { + test("postgres: jsonb_agg of jsonb_build_object, related-PK ascending, empty-set guarded", () => { + const sql = emitViewDdl(wholeObject(), { dialect: "postgres", ...OPTS }); + expect(sql).toContain( + "COALESCE(jsonb_agg(jsonb_build_object('id', s.id, 'name', s.supplier_name) ORDER BY s.id ASC) " + + "FILTER (WHERE s.id IS NOT NULL), '[]'::jsonb) AS supplier_briefs", + ); + }); + + // SQLite needs the json_each re-wrap. Measured on 3.44.0: the in-aggregate ORDER BY + // destroys the JSON subtype, so json_group_array(json_object(...) ORDER BY ...) returns + // an array of QUOTED STRINGS — and a json() wrapper on the argument does not survive it + // either. json_each iterates in array order, so re-wrapping element-by-element restores + // the objects while keeping the ordering. Found by the real-engine probe. + test("sqlite: the ordered array is re-wrapped through json_each so elements stay OBJECTS", () => { + const sql = emitViewDdl(wholeObject(), { dialect: "sqlite", ...OPTS }); + expect(sql).toContain( + "(SELECT json_group_array(json(mo_je.value)) FROM json_each(" + + "COALESCE(json_group_array(json_object('id', s.id, 'name', s.supplier_name) ORDER BY s.id ASC) " + + "FILTER (WHERE s.id IS NOT NULL), json_array())) mo_je) AS supplier_briefs", + ); + }); + + test("an explicit @orderBy leads and the related PK is appended as tie-break", () => { + const sql = emitViewDdl(wholeObject([{ column: "supplier_name", dir: "desc" }]), + { dialect: "postgres", ...OPTS }); + expect(sql).toContain("ORDER BY s.supplier_name DESC NULLS LAST, s.id ASC)"); + }); + + test("the JSON key is the VO MEMBER name; the column read is the terminal entity's", () => { + // `name` -> s.supplier_name. If these two ever collapse into one identifier the + // rollup starts emitting the physical column name into the JSON payload, which the + // generated type would not match. + const sql = emitViewDdl(wholeObject(), { dialect: "postgres", ...OPTS }); + expect(sql).toContain("'name', s.supplier_name"); + expect(sql).not.toContain("'supplier_name', s.supplier_name"); + }); +}); + +// The scalar @of arm shares this branch neighbourhood. Its emitted SQL is in every +// existing project's committed migrations, so it must not move by so much as a byte. +describe("emitViewDdl — #335 no-churn pin on the SCALAR collect arm", () => { + const scalar = (distinct: boolean, orderBy: ViewOrderKey[]): ViewSpec => ({ + viewName: "v_order_summary", + joinTree: { + baseEntity: "Order", + baseAlias: "o", + joins: [ + { relationship: "items", targetEntity: "Item", alias: "i", cardinality: "many", + fkColumn: "order_id", pkColumn: "id", referenceHolder: "target", joinType: "left", children: [] }, + ], + }, + selectSpec: { + columns: [ + { kind: "passthrough", fieldName: "id", dbColAlias: "id", sourceAlias: "o", sourceColumn: "id" }, + { kind: "collectAgg", fieldName: "categories", dbColAlias: "categories", + sourceAlias: "i", sourceColumn: "category", joinedPkColumn: "id", distinct, orderBy }, + ], + }, + groupBy: ["o.id"], + }); + const opts = { baseTableName: "orders", joinTables: { Item: "items" } } as const; + + test("postgres: array_agg, value-ascending default, '{}' empty — unchanged", () => { + expect(emitViewDdl(scalar(false, []), { dialect: "postgres", ...opts })).toContain( + "COALESCE(array_agg(i.category ORDER BY i.category ASC) FILTER (WHERE i.id IS NOT NULL), '{}') AS categories", + ); + }); + + test("postgres: an explicit @orderBy still gets NO PK tie-break — unchanged", () => { + // Deliberately asymmetric with the whole-object arm above: adding a tie-break here + // would change the emitted SQL of every existing project that uses @orderBy. + expect(emitViewDdl(scalar(false, [{ column: "category", dir: "desc" }]), { dialect: "postgres", ...opts })).toContain( + "COALESCE(array_agg(i.category ORDER BY i.category DESC NULLS LAST) FILTER (WHERE i.id IS NOT NULL), '{}') AS categories", + ); + }); + + test("sqlite: json_group_array / json_array() — unchanged", () => { + expect(emitViewDdl(scalar(false, []), { dialect: "sqlite", ...opts })).toContain( + "COALESCE(json_group_array(i.category ORDER BY i.category ASC) FILTER (WHERE i.id IS NOT NULL), json_array()) AS categories", + ); + }); +}); diff --git a/server/typescript/packages/codegen-ts/test/projection/collect-whole-object-spec.test.ts b/server/typescript/packages/codegen-ts/test/projection/collect-whole-object-spec.test.ts new file mode 100644 index 000000000..21eb6c230 --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/projection/collect-whole-object-spec.test.ts @@ -0,0 +1,144 @@ +// #335 — extractViewSpec for a WHOLE-OBJECT `origin.aggregate @agg:collect` +// (no `@of`): the related rows roll up as the carrying field.object's declared +// `@objectRef` value object rather than as one scalar column. +// +// The pre-#335 extractor bailed on `if (!of_) continue;` ABOVE the collect +// branch, so this metadata LOADED and codegen silently dropped the column — a +// CREATE VIEW with no such column while the generated type still declared the +// field. That is the failure this file exists to prevent, so the "column is +// absent" assertions below are load-bearing, not incidental. + +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader, InMemoryStringSource } from "@metaobjectsdev/metadata"; +import { extractViewSpec } from "../../src/projection/extract-view-spec.js"; + +async function load(children: unknown[]) { + const json = JSON.stringify({ "metadata.root": { package: "test", children } }); + const result = await new MetaDataLoader().load([new InMemoryStringSource(json)]); + if (result.errors.length > 0) { + throw new Error(`Loader errors:\n${result.errors.map((e) => e.message).join("\n")}`); + } + return result.root; +} + +/** Product 1—* Supplier; SupplierBrief is the curated value object collected per product. + * `voFields` and `orderBy` vary per test; everything else is fixed. */ +function model(opts: { voFields?: unknown[]; orderBy?: string[] } = {}) { + const voFields = opts.voFields ?? [ + { "field.int": { name: "id" } }, + { "field.string": { name: "name" } }, + ]; + const origin: Record = { "@agg": "collect", "@via": "Product.suppliers" }; + if (opts.orderBy !== undefined) origin["@orderBy"] = opts.orderBy; + return [ + { + "object.entity": { + name: "Product", + children: [ + { "source.rdb": { "@table": "products" } }, + { "field.int": { name: "id" } }, + { "field.string": { name: "title" } }, + { "identity.primary": { name: "id", "@fields": "id" } }, + { "relationship.association": { name: "suppliers", "@objectRef": "Supplier", "@cardinality": "many" } }, + ], + }, + }, + { + "object.entity": { + name: "Supplier", + children: [ + { "source.rdb": { "@table": "suppliers" } }, + { "field.int": { name: "id" } }, + { "field.string": { name: "name" } }, + { "field.string": { name: "region" } }, + { "field.int": { name: "productId" } }, + { "identity.primary": { name: "id", "@fields": "id" } }, + { "identity.reference": { name: "ref_product", "@fields": "productId", "@references": "Product" } }, + ], + }, + }, + { "object.value": { name: "SupplierBrief", children: voFields } }, + { + "object.projection": { + name: "ProductSummary", + children: [ + { "source.rdb": { "@kind": "view", "@table": "v_product_summary" } }, + { "field.int": { name: "id", extends: "Product.id" } }, + { "identity.primary": { name: "id", extends: "Product.id" } }, + { + "field.object": { + name: "supplierBriefs", + isArray: true, + "@objectRef": "SupplierBrief", + children: [{ "origin.aggregate": origin }], + }, + }, + ], + }, + }, + ]; +} + +async function specFor(opts: Parameters[0] = {}) { + const root = await load(model(opts)); + const projection = root.objects().find((o) => o.name === "ProductSummary")!; + return extractViewSpec(projection, root, { columnNamingStrategy: "snake_case" }); +} + +describe("extractViewSpec — #335 whole-object collect", () => { + test("an @of-less collect resolves a collectObjectAgg column carrying the VO's members", async () => { + const spec = await specFor(); + const col = spec.selectSpec.columns.find((c) => c.fieldName === "supplierBriefs"); + expect(col).toBeDefined(); + expect(col!.kind).toBe("collectObjectAgg"); + if (col!.kind !== "collectObjectAgg") return; + expect(col!.members).toEqual([ + { memberName: "id", sourceColumn: "id" }, + { memberName: "name", sourceColumn: "name" }, + ]); + expect(col!.orderBy).toEqual([]); // default = related PK ascending, applied at emit + expect(col!.joinedPkColumn).toBe("id"); + expect(col!.dbColAlias).toBe("supplier_briefs"); + }); + + test("the VO's member list is the exposure — a member the VO omits is not projected", async () => { + // Supplier declares `region`; SupplierBrief does not. The column must carry two + // members, not three: the declared value object IS the projection, which is the + // #270 guarantee this rollup has to keep. + const spec = await specFor(); + const col = spec.selectSpec.columns.find((c) => c.fieldName === "supplierBriefs")!; + if (col.kind !== "collectObjectAgg") throw new Error("expected collectObjectAgg"); + expect(col.members.map((m) => m.memberName)).toEqual(["id", "name"]); + expect(col.members.map((m) => m.memberName)).not.toContain("region"); + }); + + test("a member's sourceColumn is the TERMINAL entity's physical column, not the member name", async () => { + // Supplier.name carries @column "supplier_name" — the emitted SQL must read that + // column while the JSON key stays the VO member name. + const children = model(); + const supplier = children[1] as { "object.entity": { children: Record[] } }; + supplier["object.entity"].children[2] = { "field.string": { name: "name", "@column": "supplier_name" } }; + const root = await load(children); + const projection = root.objects().find((o) => o.name === "ProductSummary")!; + const spec = extractViewSpec(projection, root, { columnNamingStrategy: "snake_case" }); + const col = spec.selectSpec.columns.find((c) => c.fieldName === "supplierBriefs")!; + if (col.kind !== "collectObjectAgg") throw new Error("expected collectObjectAgg"); + expect(col.members).toEqual([ + { memberName: "id", sourceColumn: "id" }, + { memberName: "name", sourceColumn: "supplier_name" }, + ]); + }); + + test("@orderBy keys resolve against the @via terminal entity", async () => { + const spec = await specFor({ orderBy: ["name:desc"] }); + const col = spec.selectSpec.columns.find((c) => c.fieldName === "supplierBriefs")!; + if (col.kind !== "collectObjectAgg") throw new Error("expected collectObjectAgg"); + expect(col.orderBy).toEqual([{ column: "name", dir: "desc" }]); + }); + + test("it is a real aggregate — the base passthrough columns are GROUPed BY", async () => { + const spec = await specFor(); + expect(spec.groupBy.length).toBeGreaterThan(0); + expect(spec.groupBy).toContain(`${spec.joinTree.baseAlias}.id`); + }); +}); diff --git a/server/typescript/packages/integration-tests/test/view-lifecycle-pg.test.ts b/server/typescript/packages/integration-tests/test/view-lifecycle-pg.test.ts index 89bd3ceeb..bec18dd73 100644 --- a/server/typescript/packages/integration-tests/test/view-lifecycle-pg.test.ts +++ b/server/typescript/packages/integration-tests/test/view-lifecycle-pg.test.ts @@ -432,6 +432,13 @@ describe("view lifecycle — real Postgres", () => { { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, { "identity.reference": { name: "ref_program", "@fields": "programId", "@references": "Program" } }, ] } }, + // #335 — the curated value object a WHOLE-OBJECT collect rolls Week rows up into. + // Week also declares programId / durationMinutes / createdAt; the brief omits them, + // and the emitted jsonb must contain ONLY the two declared members. + { "object.value": { name: "WeekBrief", children: [ + { "field.long": { name: "id" } }, + { "field.string": { name: "label" } }, + ] } }, { "object.projection": { name: "ProgramSummary", children: [ { "source.rdb": { "@kind": "view", "@table": "v_program_summary" } }, { "identity.primary": { name: "id", extends: "Program.id", "@fields": "id" } }, @@ -445,6 +452,10 @@ describe("view lifecycle — real Postgres", () => { // collectAgg — the week labels as a native array, ordered for determinism. { "field.string": { name: "weekLabels", isArray: true, children: [ { "origin.aggregate": { "@agg": "collect", "@of": "Week.label", "@via": "Program.weeks", "@orderBy": ["label:asc"] } } ] } }, + // #335 collectObjectAgg — the whole WeekBrief per related row, beside the scalar + // collect above, so both collect arms are exercised in ONE view on one engine. + { "field.object": { name: "weekBriefs", isArray: true, "@objectRef": "WeekBrief", children: [ + { "origin.aggregate": { "@agg": "collect", "@via": "Program.weeks" } } ] } }, // first — the most-recent week's label (null over an empty related set). { "field.string": { name: "latestWeekLabel", children: [ { "origin.first": { "@of": "Week.label", "@via": "Program.weeks", "@orderBy": ["createdAt:desc"] } } ] } }, @@ -459,6 +470,11 @@ describe("view lifecycle — real Postgres", () => { expect(up).toMatch(/COALESCE\(bool_or\([^)]*\) FILTER \(WHERE [^)]*\), FALSE\) AS "anyLongWeek"/); expect(up).toMatch(/COALESCE\(bool_and\([^)]*\) FILTER \(WHERE [^)]*\), TRUE\) AS "allLongWeeks"/); expect(up).toMatch(/COALESCE\(array_agg\([^)]*\) FILTER \(WHERE [^)]*\), '\{\}'\) AS "weekLabels"/); + // #335 — jsonb, not json: PG's `json` has no equality or ordering operator, so the + // json_agg(json_build_object(…) ORDER BY …) form does not even run. + expect(up).toContain( + `COALESCE(jsonb_agg(jsonb_build_object('id', w.id, 'label', w.label) ORDER BY w.id ASC) FILTER (WHERE w.id IS NOT NULL), '[]'::jsonb) AS "weekBriefs"`, + ); // THE gate: the view (aggregates + correlated-subquery + computed) round-trips. await assertConverged(expected); @@ -471,12 +487,13 @@ describe("view lifecycle — real Postgres", () => { ).execute(k); const rows = await sql.raw( - `SELECT "id","anyLongWeek","allLongWeeks","weekLabels","latestWeekLabel","isPublished" + `SELECT "id","anyLongWeek","allLongWeeks","weekLabels","weekBriefs","latestWeekLabel","isPublished" FROM "v_program_summary" ORDER BY "id"`, ).execute(k); type Row = { id: string; anyLongWeek: boolean; allLongWeeks: boolean; - weekLabels: string[]; latestWeekLabel: string | null; isPublished: boolean; + weekLabels: string[]; weekBriefs: { id: number; label: string }[]; + latestWeekLabel: string | null; isPublished: boolean; }; const byId = new Map((rows.rows as Row[]).map((r) => [String(r.id), r])); @@ -485,6 +502,22 @@ describe("view lifecycle — real Postgres", () => { expect(full.anyLongWeek).toBe(true); // 90 > 60 expect(full.allLongWeeks).toBe(false); // 30 is NOT > 60 expect(full.weekLabels).toEqual(["A", "B"]); // native PG array, ordered by label + // #335 — the node-postgres driver parses jsonb, so this is a real array of OBJECTS + // (not a string). Members are the DECLARED two, in related-PK order; if programId / + // durationMinutes / createdAt appear, "the declared VO IS the exposure" (#270) has + // broken at the SQL tier. + // + // NOTE the id types, which are deliberately asserted rather than normalised away: + // `Week.id` is a field.long -> BIGINT, and a TOP-LEVEL bigint column arrives as the + // STRING "1" (node-postgres stringifies bigint to avoid precision loss) — see the + // `id: string` row type above. The SAME value inside the rollup arrives as the NUMBER + // 1, because jsonb_build_object serialises it as a JSON number and JSON has no bigint. + // That asymmetry is inherent to a JSON-valued rollup, not a codegen bug, and it is + // lossy above 2^53. Pinned here so a future change to either tier has to face it. + expect(full.weekBriefs).toEqual([ + { id: 1, label: "A" }, + { id: 2, label: "B" }, + ]); expect(full.latestWeekLabel).toBe("B"); // most recent by createdAt expect(full.isPublished).toBe(true); // status = 'PUBLISHED' @@ -493,6 +526,7 @@ describe("view lifecycle — real Postgres", () => { expect(empty.anyLongWeek).toBe(false); // any over ∅ = false expect(empty.allLongWeeks).toBe(true); // all over ∅ = true (vacuous) expect(empty.weekLabels).toEqual([]); // collect over ∅ = [] (NOT null) + expect(empty.weekBriefs).toEqual([]); // whole-object collect over ∅ = [] too expect(empty.latestWeekLabel).toBeNull(); // first over ∅ = null expect(empty.isPublished).toBe(false); // status = 'DRAFT' }); diff --git a/server/typescript/packages/integration-tests/test/view-lifecycle-sqlite.test.ts b/server/typescript/packages/integration-tests/test/view-lifecycle-sqlite.test.ts index cec96ac13..fd7db1828 100644 --- a/server/typescript/packages/integration-tests/test/view-lifecycle-sqlite.test.ts +++ b/server/typescript/packages/integration-tests/test/view-lifecycle-sqlite.test.ts @@ -45,6 +45,13 @@ const META = JSON.stringify({ "metadata.root": { package: "acme", children: [ { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, { "identity.reference": { name: "ref_program", "@fields": "programId", "@references": "Program" } }, ] } }, + // #335 — the curated value object a WHOLE-OBJECT collect rolls Week rows up into. + // Week declares `programId`, `durationMinutes` and `createdAt` too; the brief omits + // them, and the emitted JSON must contain ONLY the two declared members. + { "object.value": { name: "WeekBrief", children: [ + { "field.long": { name: "id" } }, + { "field.string": { name: "label" } }, + ] } }, { "object.projection": { name: "ProgramSummary", children: [ { "source.rdb": { "@kind": "view", "@table": "v_program_summary" } }, { "identity.primary": { name: "id", extends: "Program.id", "@fields": "id" } }, @@ -56,6 +63,8 @@ const META = JSON.stringify({ "metadata.root": { package: "acme", children: [ { "origin.aggregate": { "@agg": "all", "@via": "Program.weeks", "@filter": { durationMinutes: { gt: 60 } } } } ] } }, { "field.string": { name: "weekLabels", isArray: true, children: [ { "origin.aggregate": { "@agg": "collect", "@of": "Week.label", "@via": "Program.weeks", "@orderBy": ["label:asc"] } } ] } }, + { "field.object": { name: "weekBriefs", isArray: true, "@objectRef": "WeekBrief", children: [ + { "origin.aggregate": { "@agg": "collect", "@via": "Program.weeks" } } ] } }, { "field.string": { name: "latestWeekLabel", children: [ { "origin.first": { "@of": "Week.label", "@via": "Program.weeks", "@orderBy": ["createdAt:desc"] } } ] } }, { "field.boolean": { name: "isPublished", children: [ @@ -103,6 +112,11 @@ describe("view value-probe — real SQLite (#195 origins)", () => { // SQLite lowers collect to json_group_array (a JSON string), any/all to MAX/MIN over 1/0. expect(up).toContain(`CREATE VIEW "v_program_summary" AS`); expect(up).toContain("json_group_array"); + // #335 whole-object rollup: json_object per row, related-PK ascending, [] on empty. + // #335 whole-object rollup: an ordered json_object array re-wrapped through json_each + // (SQLite's in-aggregate ORDER BY destroys the JSON subtype — see view-ddl-emit). + expect(up).toContain("json_group_array(json_object('id', w.id, 'label', w.label) ORDER BY w.id ASC)"); + expect(up).toContain("SELECT json_group_array(json(mo_je.value)) FROM json_each("); expect(up).toMatch(/COALESCE\(MAX\(CASE WHEN[^)]*\)[^,]*, 0\) AS "anyLongWeek"/); expect(up).toMatch(/COALESCE\(MIN\(CASE WHEN[^)]*\)[^,]*, 1\) AS "allLongWeeks"/); await applyRaw(up); @@ -124,12 +138,12 @@ describe("view value-probe — real SQLite (#195 origins)", () => { ).execute(k); const rows = await sql.raw( - `SELECT "id","anyLongWeek","allLongWeeks","weekLabels","latestWeekLabel","isPublished" + `SELECT "id","anyLongWeek","allLongWeeks","weekLabels","weekBriefs","latestWeekLabel","isPublished" FROM "v_program_summary" ORDER BY "id"`, ).execute(k); type Row = { id: number; anyLongWeek: number; allLongWeeks: number; - weekLabels: string; latestWeekLabel: string | null; isPublished: number; + weekLabels: string; weekBriefs: string; latestWeekLabel: string | null; isPublished: number; }; const byId = new Map((rows.rows as Row[]).map((r) => [String(r.id), r])); @@ -139,6 +153,13 @@ describe("view value-probe — real SQLite (#195 origins)", () => { expect(full.anyLongWeek).toBe(1); // 90 > 60 expect(full.allLongWeeks).toBe(0); // 30 is NOT > 60 expect(JSON.parse(full.weekLabels)).toEqual(["A", "B"]); + // #335 — the DECLARED members only, in related-PK order. Week also has programId / + // durationMinutes / createdAt; if any of those appear the "declared VO IS the + // exposure" guarantee (#270) has broken at the SQL tier. + expect(JSON.parse(full.weekBriefs)).toEqual([ + { id: 1, label: "A" }, + { id: 2, label: "B" }, + ]); expect(full.latestWeekLabel).toBe("B"); // most recent by createdAt expect(full.isPublished).toBe(1); // status = 'PUBLISHED' @@ -147,6 +168,7 @@ describe("view value-probe — real SQLite (#195 origins)", () => { expect(empty.anyLongWeek).toBe(0); // any over ∅ = false (0) expect(empty.allLongWeeks).toBe(1); // all over ∅ = true (1, vacuous) expect(JSON.parse(empty.weekLabels)).toEqual([]); // collect over ∅ = [] (NOT null) + expect(JSON.parse(empty.weekBriefs)).toEqual([]); // whole-object collect over ∅ = [] too expect(empty.latestWeekLabel).toBeNull(); // first over ∅ = null expect(empty.isPublished).toBe(0); // status = 'DRAFT' }); diff --git a/server/typescript/packages/metadata/src/errors.ts b/server/typescript/packages/metadata/src/errors.ts index 15a5a099b..ac3d934fb 100644 --- a/server/typescript/packages/metadata/src/errors.ts +++ b/server/typescript/packages/metadata/src/errors.ts @@ -159,6 +159,23 @@ export const ERROR_CODES = [ // SP-H Unit9 — @filterable: true on a field subtype with no filter-operator // band (e.g. field.object). Would silently generate an empty-ops filter. "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE", + // #335 Half B — @sortable: true on an array field or a subtype with no + // filter-operator band (e.g. field.object). Would silently emit a sort + // entry over a column no dialect can ORDER BY. + "ERR_SORTABLE_UNSUPPORTED_SUBTYPE", + // #335 Half A — a whole-object @agg:collect (no @of; the carrying field.object + // rolls related rows up as its declared @objectRef value object) is malformed: + // carrier is not a field.object with @objectRef, @via absent, @distinct declared + // (refused — a no-op whenever the value object carries the primary key), an + // @orderBy key not on the @via TERMINAL entity, or a member's declared type + // disagreeing with the matched terminal field's. Distinct from ERR_INVALID_ORIGIN + // so a fixture can tell this arm from a loader that still requires @of. + "ERR_COLLECT_WHOLE_OBJECT", + // #335 Half A — a whole-object @agg:collect's value-object member has no + // matching field (by name) on the @via terminal entity. The lowering + // projects exactly the declared members; failing open here is how #270 + // turned a curated value object into the full entity. + "ERR_COLLECT_MEMBER_UNRESOLVED", // ADR-0023 — a registration was attempted against a registry sealed after its // agreed metamodel-provider bootstrap. Codegen cannot invent metamodel attrs. "ERR_REGISTRY_SEALED", diff --git a/server/typescript/packages/metadata/src/loader/meta-data-loader.ts b/server/typescript/packages/metadata/src/loader/meta-data-loader.ts index e91cefd7a..386cacbf1 100644 --- a/server/typescript/packages/metadata/src/loader/meta-data-loader.ts +++ b/server/typescript/packages/metadata/src/loader/meta-data-loader.ts @@ -18,7 +18,7 @@ import { ParseError } from "../errors.js"; import type { LoaderWarning } from "../source.js"; import { codeSource, resolvedSource } from "../source.js"; import { parseJson } from "../parser-json.js"; -import { validateDataGridSortFields, validateFilterableHasIndex, validateFilterableHasSupportedOps, validateOriginPaths, validateDerivedFieldProvidability, validateDataGridFilterValues, validateFieldObjectStorage, validateFieldMap, validateTemplatePayloadRefs, validateFieldDefaults, validateRelationships, validateIndexLookupFields, validateProjectionFilter } from "./validation-passes.js"; +import { validateDataGridSortFields, validateFilterableHasIndex, validateFilterableHasSupportedOps, validateSortableHasSupportedSubtype, validateOriginPaths, validateDerivedFieldProvidability, validateDataGridFilterValues, validateFieldObjectStorage, validateFieldMap, validateTemplatePayloadRefs, validateFieldDefaults, validateRelationships, validateIndexLookupFields, validateProjectionFilter } from "./validation-passes.js"; import { runRegisteredValidation } from "./validation-registry.js"; import { validateSourceRoles } from "../persistence/source/validate-source-roles.js"; import { validateSourceEscapes } from "../persistence/source/validate-source-escapes.js"; @@ -578,6 +578,10 @@ export class MetaDataLoader { // (would silently generate a filter that rejects every request). errors.push(...validateFilterableHasSupportedOps(root)); + // #335 Half B — @sortable on an array field or a subtype with no operator + // band → error (would silently emit a sort entry no dialect can execute). + errors.push(...validateSortableHasSupportedSubtype(root)); + // Sixth pass: origin path validation — validates passthrough.@from, // aggregate.@of, and .@via relationship chains. errors.push(...validateOriginPaths(root)); diff --git a/server/typescript/packages/metadata/src/loader/validation-passes.ts b/server/typescript/packages/metadata/src/loader/validation-passes.ts index aced951e7..3f4c57777 100644 --- a/server/typescript/packages/metadata/src/loader/validation-passes.ts +++ b/server/typescript/packages/metadata/src/loader/validation-passes.ts @@ -13,7 +13,7 @@ import type { MetaData } from "../shared/meta-data.js"; import type { MetaObject } from "../core/object/meta-object.js"; import type { MetaReferenceIdentity } from "../core/identity/meta-identity.js"; -import { ParseError } from "../errors.js"; +import { ParseError, type ErrorCode } from "../errors.js"; import { resolveObjectRef, didYouMeanHint } from "../naming-refs.js"; import { PACKAGE_SEPARATOR, CHILD_REF_SEPARATOR } from "../shared/structural.js"; import { resolvedSource, type ErrorSource } from "../source.js"; @@ -61,6 +61,7 @@ import { } from "../presentation/layout/layout-constants.js"; import { FIELD_ATTR_FILTERABLE, + FIELD_ATTR_SORTABLE, FIELD_ATTR_OBJECT_REF, FIELD_ATTR_STORAGE, STORAGE_FLATTENED, @@ -443,6 +444,26 @@ export function validateFilterableHasSupportedOps(root: MetaData): ParseError[] for (const field of obj.children().filter((c) => c.type === TYPE_FIELD)) { // ADR-0039: resolving — a concrete field may inherit @filterable via extends. if (field.attr(FIELD_ATTR_FILTERABLE) !== true) continue; + + // #335 Half B — an ARRAY field has no operator band either. Every FR-009 + // operator (eq/ne/gt/gte/lt/lte/in/like/isNull) is a scalar comparison; + // none applies to a collection column. The allowlist template does not + // consult isArray and falls through to the "string" band, so this + // previously emitted a `like` rule against a text[] column — SQL that + // cannot execute. Same reason as the subtype check below, so same code. + // ADR-0039: resolvedIsArray(), never the own `isArray` flag. + if (field.resolvedIsArray()) { + errors.push( + new ParseError( + `Field "${obj.name}.${field.name}" has @filterable: true but is an array ` + + `(isArray: true). No filter operator applies to a collection column. ` + + `Remove @filterable from this field.`, + { code: "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE", source: field.source }, + ), + ); + continue; + } + if (opsForSubType(field.subType).length > 0) continue; errors.push( new ParseError( @@ -457,6 +478,40 @@ export function validateFilterableHasSupportedOps(root: MetaData): ParseError[] return errors; } +// --------------------------------------------------------------------------- +// @sortable on a subtype or shape that cannot be ordered (#335 Half B) +// --------------------------------------------------------------------------- +// @sortable defaults FROM @filterable, so it is independently set only when +// explicit — and nothing validated it, while @filterable has had a hard error +// since SP-H Unit9. A @sortable JSON or array column emits a sort entry over a +// column no dialect can ORDER BY meaningfully. → ERR_SORTABLE_UNSUPPORTED_SUBTYPE. + +export function validateSortableHasSupportedSubtype(root: MetaData): ParseError[] { + const errors: ParseError[] = []; + // ADR-0039: root has no super; children()==ownChildren() but resolving is the default. + for (const obj of root.children().filter((c) => c.type === TYPE_OBJECT)) { + // children() — inherited @sortable fields (via extends:/super:) are visible. + for (const field of obj.children().filter((c) => c.type === TYPE_FIELD)) { + // ADR-0039: resolving — a concrete field may inherit @sortable via extends. + if (field.attr(FIELD_ATTR_SORTABLE) !== true) continue; + // ADR-0039: resolvedIsArray(), never the own `isArray` flag. + const isArray = field.resolvedIsArray(); + if (!isArray && opsForSubType(field.subType).length > 0) continue; + errors.push( + new ParseError( + `Field "${obj.name}.${field.name}" has @sortable: true but ` + + (isArray + ? `is an array (isArray: true) — a collection column has no ordering.` + : `its subtype "${field.subType}" cannot be ordered.`) + + ` Remove @sortable from this field.`, + { code: "ERR_SORTABLE_UNSUPPORTED_SUBTYPE", source: field.source }, + ), + ); + } + } + return errors; +} + // --------------------------------------------------------------------------- // Origin path validation // @@ -594,10 +649,25 @@ function _validateFromPath( return { entity: sourceObj, field: sourceField }; } +/** A fully-walked `@via` path: the relationship hop nodes in path order, and + * the TERMINAL entity reached after the last hop. Both fall out of one walk, + * so they are returned together — recovering the terminal with a second walk + * means maintaining a second copy of the ADR-0042 package-resolution rule. + * Shape mirrors `_validateFromPath`'s `ResolvedFromTarget`, which returns a + * pair for the same reason. */ +interface WalkedViaPath { + hops: MetaData[]; + terminal: MetaData; +} + /** - * Validate an explicit `@via` "Entity.rel[.rel...]" path. Returns the walked - * relationship hop nodes (in path order) on full success — FR-024 B5 runs the - * cardinality checks over them — or undefined when any error was pushed. + * Validate an explicit `@via` "Entity.rel[.rel...]" path. On full success + * returns the walked relationship hop nodes (in path order) — FR-024 B5 runs + * the cardinality checks over them — together with the terminal entity node + * (#335: a whole-object `@agg:collect` has no `@of` entity, so `@orderBy` keys + * and value-object members resolve against the terminal instead). Returns + * undefined when any error was pushed, so `terminal` is defined exactly when + * `hops` is. */ function _validateViaPath( viaAttr: string, @@ -606,7 +676,7 @@ function _validateViaPath( fieldName: string, originSource: ErrorSource, errors: ParseError[], -): MetaData[] | undefined { +): WalkedViaPath | undefined { const projectionName = projection.name; // FR5d — referrer is `::`. const referrer = `${projection.fqn()}::${fieldName}`; @@ -708,7 +778,8 @@ function _validateViaPath( hops.push(rel); currentObj = nextObj; } - return hops; + // currentObj is the terminal: every earlier exit returned undefined. + return { hops, terminal: currentObj }; } // --------------------------------------------------------------------------- @@ -958,6 +1029,58 @@ function _checkAggregateCardinality( } } +/** + * #335 — a whole-object `@agg:collect` projects EXACTLY the declared value + * object's members, each matched by NAME against the `@via` terminal entity. + * + * Two rules, both fail-closed: + * - a member with no matching field on the terminal is unresolvable. Failing + * OPEN here is how #270 turned a curated value object into the full entity, + * invisible in a diff because the metadata still read as curated. + * - a matched member must agree on BOTH type axes (#185 type-preserving + * doctrine), so a scalar member cannot bind an array field or vice versa. + * + * Both refusals carry a whole-object-specific code — ERR_COLLECT_MEMBER_UNRESOLVED + * for the unmatched member, ERR_COLLECT_WHOLE_OBJECT for the type disagreement. + * The latter is deliberately NOT the scalar arm's ERR_INVALID_ORIGIN: a loader + * that still requires @of rejects this metadata with ERR_INVALID_ORIGIN too, so + * sharing the code would make a corpus fixture pass on a port that implements + * nothing (the corpus compares only code + source, never message text). + */ +function _checkCollectMembers( + refTarget: MetaData, + terminal: MetaData, + obj: MetaData, + field: MetaData, + src: ErrorSource, + errors: ParseError[], +): void { + // ADR-0039: resolving — a value object may inherit members via extends, and + // the terminal entity may inherit fields; own-only would silently skip + // inherited members, which is exactly the #270 bug class this guards. + const terminalFields = terminal.children().filter((c) => c.type === TYPE_FIELD); + for (const member of refTarget.children().filter((c) => c.type === TYPE_FIELD)) { + const match = terminalFields.find((f) => f.name === member.name); + if (match === undefined) { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: value-object member ` + + `'${member.name}' has no matching field on '${terminal.name}' — a whole-object ` + + `rollup projects exactly the declared members.`, + { code: "ERR_COLLECT_MEMBER_UNRESOLVED", source: src })); + continue; + } + const memberLabel = _typeLabel(member); + const matchLabel = _typeLabel(match); + if (memberLabel !== matchLabel) { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: value-object member ` + + `'${member.name}' is ${memberLabel} but '${terminal.name}.${match.name}' ` + + `is ${matchLabel} — a whole-object rollup preserves each member's type.`, + { code: "ERR_COLLECT_WHOLE_OBJECT", source: src })); + } + } +} + /** * FR-024 B6 (spec §4; ADR-0029 decision 7) — extends/origin agreement. * @@ -1023,6 +1146,17 @@ function _checkExtendsOriginAgreement( * FR-015 stored-proc parameter refs the retired ERR_PARAMETER_REF_PASSTHROUGH_ * TYPE_MISMATCH used to cover). */ +/** + * Both type axes in one comparable token. Subtype names never contain "[]", so + * equal labels ⇔ same subType AND same array-ness. Nullability is deliberately + * NOT judged — an outer-join view legitimately widens NOT NULL. + * ADR-0039: resolvedIsArray(), never the own `isArray` flag — a field may + * inherit its array-ness via extends. + */ +function _typeLabel(field: MetaData): string { + return `field.${field.subType}${field.resolvedIsArray() ? "[]" : ""}`; +} + function _checkPassthroughType( field: MetaData, fromField: MetaData, @@ -1033,11 +1167,8 @@ function _checkPassthroughType( errors: ParseError[], ): void { if (convert) return; // deliberate type change acknowledged - // Compare both axes at once via the type-label: subtype names never contain - // "[]", so equal labels ⇔ same subType AND same array-ness (nullability is - // deliberately not judged — an outer-join view legitimately widens NOT NULL). - const declared = `field.${field.subType}${field.resolvedIsArray() ? "[]" : ""}`; - const source = `field.${fromField.subType}${fromField.resolvedIsArray() ? "[]" : ""}`; + const declared = _typeLabel(field); + const source = _typeLabel(fromField); if (declared === source) return; errors.push( new ParseError( @@ -1059,6 +1190,10 @@ function _checkPassthroughType( * and carries no vocabulary. Shared by `@agg:collect` (element order) and * `origin.first` (row selection). A missing related entity means a prior error * already fired — skip silently. + * + * `code` lets the whole-object `@agg:collect` arm report ERR_COLLECT_WHOLE_OBJECT + * instead; it defaults to ERR_INVALID_ORIGIN so the scalar `@of` and `origin.first` + * call sites keep their existing envelope byte-for-byte. */ function _validateOrderByKeys( orderBy: unknown, @@ -1068,6 +1203,7 @@ function _validateOrderByKeys( label: string, originSource: ErrorSource, errors: ParseError[], + code: ErrorCode = "ERR_INVALID_ORIGIN", ): void { if (!Array.isArray(orderBy) || relatedEntity === undefined) return; for (const raw of orderBy) { @@ -1081,14 +1217,14 @@ function _validateOrderByKeys( errors.push( new ParseError( `${label} on ${obj.name}.${fieldName}: @orderBy key "${raw}" — no such field "${key}" on ${relatedEntity.name}.`, - { code: "ERR_INVALID_ORIGIN", source: originSource }, + { code, source: originSource }, ), ); } else if (dir !== undefined && !(SORT_ORDER_VALUES as readonly string[]).includes(dir)) { errors.push( new ParseError( `${label} on ${obj.name}.${fieldName}: @orderBy key "${raw}" — direction must be one of ${SORT_ORDER_VALUES.join("|")}.`, - { code: "ERR_INVALID_ORIGIN", source: originSource }, + { code, source: originSource }, ), ); } @@ -1158,9 +1294,9 @@ export function validateOriginPaths(root: MetaData): ParseError[] { // ADR-0039: own — origin.* never inherits (ADR-0029). const via = origin.ownAttr(ORIGIN_PASSTHROUGH_ATTR_VIA); if (typeof via === "string" && via !== "") { - const hops = _validateViaPath(via, root, obj, field.name, origin.source, errors); - if (hops !== undefined) { - _checkPassthroughCardinality(hops, obj, field.name, origin.source, errors); + const walked = _validateViaPath(via, root, obj, field.name, origin.source, errors); + if (walked !== undefined) { + _checkPassthroughCardinality(walked.hops, obj, field.name, origin.source, errors); } } else if (fromTarget !== undefined && !isValueHost) { // FR-024 §6 — no @via: derive the base entity; a @from targeting @@ -1245,17 +1381,89 @@ export function validateOriginPaths(root: MetaData): ParseError[] { `origin.aggregate @agg:${String(agg)} on ${obj.name}.${field.name}: requires an explicit @via (a quantifier has no @of to infer the path from).`, { code: "ERR_INVALID_ORIGIN", source: src })); } else { - const hops = _validateViaPath(via, root, obj, field.name, src, errors); - if (hops !== undefined) _checkAggregateCardinality(hops, obj, field.name, src, errors); + const walked = _validateViaPath(via, root, obj, field.name, src, errors); + if (walked !== undefined) _checkAggregateCardinality(walked.hops, obj, field.name, src, errors); } continue; } - // --- count/sum/avg/min/max/collect: @of REQUIRED --- + // --- @of: REQUIRED for count/sum/avg/min/max; OPTIONAL for collect --- + // #335 — an @of-absent collect is a WHOLE-OBJECT rollup: collect the + // related rows as an array of the field's declared @objectRef value + // object rather than an array of one scalar column. if (!ofPresent) { - errors.push(new ParseError( - `origin.aggregate on ${obj.name}.${field.name}: missing @of.`, - { code: "ERR_INVALID_ORIGIN", source: src })); + if (!isCollect) { + errors.push(new ParseError( + `origin.aggregate on ${obj.name}.${field.name}: missing @of.`, + { code: "ERR_INVALID_ORIGIN", source: src })); + continue; + } + // Whole-object rollup. The carrying field must be a field.object + // naming a value object, and @via must be explicit (there is no @of + // entity to infer the single-hop relation from). + // ADR-0039: resolving — @objectRef may be inherited via extends. + const objectRef = field.attr(FIELD_ATTR_OBJECT_REF); + if (field.subType !== FIELD_SUBTYPE_OBJECT || typeof objectRef !== "string" || objectRef === "") { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: @of is omitted, so this is a ` + + `whole-object rollup — the carrying field must be a field.object declaring @objectRef ` + + `(add @of to collect a single column instead).`, + { code: "ERR_COLLECT_WHOLE_OBJECT", source: src })); + continue; + } + // #210's value-only rule is PAYLOAD-scoped and never reaches a + // projection-hosted field, so this branch enforces it itself. + // Without it an @objectRef to an entity silently rolls up the FULL + // entity — the #270 shape, this time baked into DDL. + // ADR-0042 — a bare @objectRef resolves in the DECLARING owner's + // package (an inherited field resolves in the package that + // declared it) — same rule _checkNestedPayloadRefsValueOnly uses. + const refOwner = field.parent ?? obj; + const refPkg = refOwner.package ?? refOwner.fileDefaultPackage ?? ""; + const refTarget = resolveObjectRef(root, objectRef, refPkg).node; + if (refTarget !== undefined && refTarget.subType !== OBJECT_SUBTYPE_VALUE) { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: @objectRef '${objectRef}' ` + + `resolves to ${TYPE_OBJECT}.${refTarget.subType} — a whole-object rollup must target an ` + + `object.value (#210, ADR-0028).`, + { code: "ERR_SUBTYPE_RULE_VIOLATION", source: src })); + continue; + } + // ADR-0039: own — origin.* never inherits (ADR-0029). + const viaAttr = origin.ownAttr(ORIGIN_AGGREGATE_ATTR_VIA); + if (typeof viaAttr !== "string" || viaAttr === "") { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: @via is required on a ` + + `whole-object rollup — there is no @of entity to infer the relationship from.`, + { code: "ERR_COLLECT_WHOLE_OBJECT", source: src })); + continue; + } + // @distinct is refused on the object form. It is NOT an engine limit + // (both engines dedupe JSON objects); it is a guaranteed no-op + // whenever the value object carries the entity's primary key, which + // is the common case, and a silent no-op is worse than a refusal. + if (hasDistinct) { + errors.push(new ParseError( + `origin.aggregate @agg:collect on ${obj.name}.${field.name}: @distinct is not supported on a ` + + `whole-object rollup (it is a no-op whenever the value object carries the primary key).`, + { code: "ERR_COLLECT_WHOLE_OBJECT", source: src })); + continue; + } + // One walk yields both the hops (cardinality) and the terminal + // entity (@orderBy keys, member resolution). An invalid @via + // (e.g. single-segment "A") returns undefined having already + // pushed its own error, so everything downstream is skipped and + // no second, misleadingly-scoped error is emitted. + const via = _validateViaPath(viaAttr, root, obj, field.name, src, errors); + if (via !== undefined) { + _checkAggregateCardinality(via.hops, obj, field.name, src, errors); + // @orderBy keys resolve against the @via TERMINAL entity, not @of. + _validateOrderByKeys(orderBy, via.terminal, obj, field.name, "origin.aggregate @agg:collect", src, errors, + "ERR_COLLECT_WHOLE_OBJECT"); + if (refTarget !== undefined) { + _checkCollectMembers(refTarget, via.terminal, obj, field, src, errors); + } + } continue; } // NOTE (FR-024 B6): NO extends/origin agreement check on aggregates — @@ -1276,9 +1484,9 @@ export function validateOriginPaths(root: MetaData): ParseError[] { // ADR-0039: own — origin.* never inherits (ADR-0029). const via = origin.ownAttr(ORIGIN_AGGREGATE_ATTR_VIA); if (typeof via === "string" && via !== "") { - const hops = _validateViaPath(via, root, obj, field.name, src, errors); - if (hops !== undefined) { - _checkAggregateCardinality(hops, obj, field.name, src, errors); + const walked = _validateViaPath(via, root, obj, field.name, src, errors); + if (walked !== undefined) { + _checkAggregateCardinality(walked.hops, obj, field.name, src, errors); } continue; } @@ -1371,8 +1579,8 @@ export function validateOriginPaths(root: MetaData): ParseError[] { // @via — explicit (validated + cardinality) or single-hop-unique inferred. const via = origin.ownAttr(ORIGIN_FIRST_ATTR_VIA); if (typeof via === "string" && via !== "") { - const hops = _validateViaPath(via, root, obj, field.name, src, errors); - if (hops !== undefined) _checkAggregateCardinality(hops, obj, field.name, src, errors); + const walked = _validateViaPath(via, root, obj, field.name, src, errors); + if (walked !== undefined) _checkAggregateCardinality(walked.hops, obj, field.name, src, errors); } else if (ofTarget !== undefined) { // (A value host never reaches here — the #210 assembly-origin // check above already rejected origin.first on a value.) diff --git a/server/typescript/packages/metadata/src/persistence/origin/origin-definition.embedded.ts b/server/typescript/packages/metadata/src/persistence/origin/origin-definition.embedded.ts index 14d14588d..f01ff4160 100644 --- a/server/typescript/packages/metadata/src/persistence/origin/origin-definition.embedded.ts +++ b/server/typescript/packages/metadata/src/persistence/origin/origin-definition.embedded.ts @@ -49,9 +49,9 @@ export const ORIGIN_DEFINITION: ProviderDefinition = { { "type": "origin", "subType": "aggregate", - "description": "A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup of @of).", + "description": "A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted).", "whenToUse": "A projection needs a value derived by reducing related rows — a count/sum/avg/min/max, a 'did any/every related row match' flag, or an array of collected values. Declare it instead of hand-writing the aggregate query — it stays consistent and regenerates.", - "rules": "@via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max/collect and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only.", + "rules": "@via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only. On @agg:collect @of is OPTIONAL: omitting it declares a WHOLE-OBJECT rollup, which collects each related row as the carrying field's declared @objectRef value object instead of one scalar column. A whole-object rollup requires a field.object carrying @objectRef, requires that @objectRef to name an object.value, requires an explicit @via (there is no @of entity to infer the path from), and refuses @distinct. Its @orderBy keys resolve against the @via TERMINAL entity, not the head or a middle hop. Its value-object members bind to the terminal entity's fields BY NAME — member name == terminal field name, deliberately NOT extends, so one value object stays collectable from two different entities — and every member must match a terminal field agreeing on BOTH field. and array-ness.", "children": [ { "type": "attr", @@ -69,7 +69,7 @@ export const ORIGIN_DEFINITION: ProviderDefinition = { "all", "collect" ], - "description": "The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup of @of — the field must be isArray)." + "description": "The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted; the field must be isArray)." }, { "type": "attr", @@ -77,7 +77,7 @@ export const ORIGIN_DEFINITION: ProviderDefinition = { "name": "of", "min": 0, "max": 1, - "description": "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max/collect; forbidden for any/all (which quantify over rows via @filter, not a column)." + "description": "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max; OPTIONAL for collect, where absent means a whole-object rollup of the field's declared @objectRef value object; forbidden for any/all (which quantify over rows via @filter, not a column)." }, { "type": "attr", @@ -101,7 +101,7 @@ export const ORIGIN_DEFINITION: ProviderDefinition = { "name": "distinct", "min": 0, "max": 1, - "description": "Set (collect-only) to dedupe collected values (set semantics)." + "description": "Set (collect-only) to dedupe collected values (set semantics). Not supported on a whole-object collect (@of omitted): it is a guaranteed no-op whenever the value object carries the primary key, and a silent no-op is worse than a refusal." }, { "type": "attr", diff --git a/server/typescript/packages/metadata/src/registry-manifest.ts b/server/typescript/packages/metadata/src/registry-manifest.ts index c346cb734..dbbf6fade 100644 --- a/server/typescript/packages/metadata/src/registry-manifest.ts +++ b/server/typescript/packages/metadata/src/registry-manifest.ts @@ -112,7 +112,7 @@ interface ManifestType { * constant read `"0.10"`). Bump with that script — never by hand — so the manifest and * all four port constants move together. */ -export const METAMODEL_VERSION = "0.11"; +export const METAMODEL_VERSION = "0.12"; /** The full canonical manifest. All collections are sorted for byte-stability. */ interface RegistryManifest { diff --git a/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts b/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts new file mode 100644 index 000000000..835dec71b --- /dev/null +++ b/server/typescript/packages/metadata/test/validation-collect-whole-object.test.ts @@ -0,0 +1,282 @@ +// #335 Half A — @of is optional on origin.aggregate @agg:collect. +// +// Today collect always requires @of (an "Entity.field" naming ONE scalar +// column). When @of is absent, this is instead a WHOLE-OBJECT rollup: the +// carrying field must be a field.object (isArray, @objectRef) and the +// collect assembles the related rows as an array of that value object +// rather than an array of one scalar column. +// +// Model shape verified by execution against the corrected model in the +// task-5 brief (the model originally drafted in the brief's Step 1 does +// NOT load — it declares an object-level `object.projection extends +// "Product"`, an entity, which is ERR_SUBTYPE_RULE_VIOLATION, and an +// identity.primary with fresh @fields instead of extends, which is +// ERR_PROJECTION_IDENTITY_NOT_EXTENDED). This model instead mirrors the +// corpus's own proven-loading shape (fixtures/conformance/ +// error-origin-aggregate-no-to-many/input/meta.demo.json, CustomerSummary): +// no object-level extends on the projection; a plain field mirroring the +// base entity's PK via field-level extends; identity.primary named and +// extending the base entity's (also-named) identity, never declaring its +// own @fields. +// +// MetaDataLoader.load() is async and returns errors on the LoadResult — it +// does not throw. Harness copied from the sibling +// validation-filterable-array.test.ts. +// +// Fix round 1: the brief's six must-enforce rules and its own six test +// arms did not correspond 1:1 — cardinality and @orderBy-resolves-against- +// the-terminal-entity had no arm at all, so a review that deleted the +// _validateViaPath/_checkAggregateCardinality call AND the entire +// `_viaTerminalEntityNode` call site (every caller of the helper) still +// passed all six original tests. Two arms added below close that gap, and +// the three arms that previously discriminated only by a shared code now +// also assert each rule's distinctive message fragment. +// +// The five arms that once shared ERR_INVALID_ORIGIN now carry +// ERR_COLLECT_WHOLE_OBJECT. ERR_INVALID_ORIGIN is what a loader still +// REQUIRING @of emits for this same metadata, so the shared code could not +// tell "the whole-object arm rejected this" from "nothing is implemented" — +// which is precisely how five of the shared conformance fixtures passed +// against three unported ports. + +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader } from "../src/loader/meta-data-loader.js"; +import { InMemoryStringSource } from "../src/loader/meta-data-source.js"; + +/** SupplierBrief's default member set — mirrors Supplier.id + Supplier.name exactly. */ +const DEFAULT_SUPPLIER_BRIEF_FIELDS = + `{ "field.long": { "name": "id" } }, { "field.string": { "name": "name" } }`; + +/** + * Product 1:N Supplier, plus a projection rolling suppliers up as objects. + * `voFields` parameterises SupplierBrief's member list (#335 Half A member + * resolution) — defaults to the field set every pre-existing arm in this file + * assumes, so no existing call site needs updating. + */ +const model = (collectField: string, voFields: string = DEFAULT_SUPPLIER_BRIEF_FIELDS) => `{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { "name": "Supplier", "children": [ + { "source.rdb": { "@kind": "table", "@table": "suppliers" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "name" } }, + { "field.long": { "name": "productId" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "identity.reference": { "name": "product", "@references": "Product", "@fields": ["productId"] } } + ]}}, + { "object.entity": { "name": "Product", "children": [ + { "source.rdb": { "@kind": "table", "@table": "products" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "relationship.association": { "name": "suppliers", "@cardinality": "many", "@objectRef": "Supplier" } } + ]}}, + { "object.value": { "name": "SupplierBrief", "children": [ ${voFields} ] }}, + { "object.projection": { "name": "ProductWithSuppliers", "children": [ + { "source.rdb": { "@kind": "view", "@view": "v_product_suppliers" } }, + { "field.long": { "name": "productId", "extends": "Product.id" } }, + { "identity.primary": { "name": "id", "extends": "Product.id" } }, + ${collectField} + ]}} + ] + } +}`; + +/** + * A.b.c three-entity chain: A -> (relationship b, many) -> B -> (relationship + * c, many) -> C. B declares field "name"; C does not. A whole-object collect + * on a projection of A rolls up C (via "A.b.c", 2 hops) — @orderBy must + * resolve against C (the TERMINAL entity reached after BOTH hops), not A + * (the @via head) or B (the middle hop). Both relationships are @cardinality + * "many" so no ERR_ORIGIN_CARDINALITY noise competes with the assertion. + */ +const CHAIN_MODEL = `{ + "metadata.root": { + "package": "acme::chain", + "children": [ + { "object.entity": { "name": "C", "children": [ + { "source.rdb": { "@kind": "table", "@table": "cs" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } } + ]}}, + { "object.entity": { "name": "B", "children": [ + { "source.rdb": { "@kind": "table", "@table": "bs" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "name" } }, + { "relationship.association": { "name": "c", "@cardinality": "many", "@objectRef": "C" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } } + ]}}, + { "object.entity": { "name": "A", "children": [ + { "source.rdb": { "@kind": "table", "@table": "as" } }, + { "field.long": { "name": "id" } }, + { "relationship.association": { "name": "b", "@cardinality": "many", "@objectRef": "B" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } } + ]}}, + { "object.value": { "name": "CBrief", "children": [ + { "field.long": { "name": "id" } } + ]}}, + { "object.projection": { "name": "AWithCs", "children": [ + { "source.rdb": { "@kind": "view", "@view": "v_a_cs" } }, + { "field.long": { "name": "aId", "extends": "A.id" } }, + { "identity.primary": { "name": "id", "extends": "A.id" } }, + { "field.object": { "name": "items", "isArray": true, "@objectRef": "CBrief", "children": [ + { "origin.aggregate": { "@agg": "collect", "@via": "A.b.c", "@orderBy": ["name"] } } + ]}} + ]}} + ] + } +}`; + +const WHOLE_OBJECT = `{ "field.object": { + "name": "supplierBriefs", "isArray": true, "@objectRef": "SupplierBrief", + "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers" } } ] +}}`; + +async function loadErrors(src: string): Promise<{ code: string; message: string }[]> { + const result = await new MetaDataLoader().load([ + new InMemoryStringSource(src, { id: "meta.demo.json" }), + ]); + return result.errors as unknown as { code: string; message: string }[]; +} + +describe("@of-absent collect (whole-object rollup)", () => { + test("loads on a field.object @objectRef isArray with @via", async () => { + const errors = await loadErrors(model(WHOLE_OBJECT)); + expect(errors).toEqual([]); + }); + + test("fails without @objectRef", async () => { + const src = model(`{ "field.object": { + "name": "supplierBriefs", "isArray": true, + "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers" } } ] + }}`); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_COLLECT_WHOLE_OBJECT"); + expect(hit).toBeDefined(); + expect(hit?.message).toContain("supplierBriefs"); + expect(hit?.message).toContain("must be a field.object"); + }); + + test("fails when @objectRef targets an entity, not a value", async () => { + const src = model(`{ "field.object": { + "name": "supplierBriefs", "isArray": true, "@objectRef": "Supplier", + "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers" } } ] + }}`); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_SUBTYPE_RULE_VIOLATION"); + expect(hit).toBeDefined(); + expect(hit?.message).toContain("supplierBriefs"); + }); + + test("fails without @via (nothing to infer the relation from)", async () => { + const src = model(`{ "field.object": { + "name": "supplierBriefs", "isArray": true, "@objectRef": "SupplierBrief", + "children": [ { "origin.aggregate": { "@agg": "collect" } } ] + }}`); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_COLLECT_WHOLE_OBJECT"); + expect(hit).toBeDefined(); + expect(hit?.message).toContain("supplierBriefs"); + expect(hit?.message).toContain("@via is required"); + }); + + test("fails when @distinct is declared", async () => { + const src = model(`{ "field.object": { + "name": "supplierBriefs", "isArray": true, "@objectRef": "SupplierBrief", + "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers", "@distinct": true } } ] + }}`); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_COLLECT_WHOLE_OBJECT"); + expect(hit).toBeDefined(); + expect(hit?.message).toContain("supplierBriefs"); + expect(hit?.message).toContain("@distinct is not supported"); + }); + + test("fails when @via is provably to-one (cardinality)", async () => { + // "Supplier.product" is a single reference hop — inherently to-one + // (_hopCardinality treats every identity.reference hop as CARDINALITY_ONE) + // — so aggregating over it is the passthrough-not-aggregate mistake + // ADR-0029 decision 6 rejects. This is the arm that pins the + // `_validateViaPath` + `_checkAggregateCardinality` call in the + // whole-object branch: deleting that call leaves this model loading + // clean. + const src = model(`{ "field.object": { + "name": "supplierBriefs", "isArray": true, "@objectRef": "SupplierBrief", + "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Supplier.product" } } ] + }}`); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_ORIGIN_CARDINALITY"); + expect(hit).toBeDefined(); + }); + + test("@orderBy resolves against the @via TERMINAL entity, not the head or a middle hop", async () => { + // CHAIN_MODEL's "name" field exists on B (the middle hop) but not on C + // (the terminal entity two hops from A). This is the arm that pins + // `_viaTerminalEntityNode` actually walking to the END of the path: a + // regression to head-only resolution would name "A" in the error, a + // regression that stops at the first hop would name "B", and deleting + // the `if (hasOrderBy...)` call site entirely emits no such error at all. + const errors = await loadErrors(CHAIN_MODEL); + const hit = errors.find((e) => e.code === "ERR_COLLECT_WHOLE_OBJECT" && e.message.includes("@orderBy")); + expect(hit).toBeDefined(); + expect(hit?.message).toContain('no such field "name" on C'); + }); + + test("a collect WITH @of is unaffected", async () => { + const src = model(`{ "field.string": { + "name": "supplierNames", "isArray": true, + "children": [ { "origin.aggregate": { "@agg": "collect", "@of": "Supplier.name", "@via": "Product.suppliers" } } ] + }}`); + const errors = await loadErrors(src); + expect(errors).toEqual([]); + }); + + // #335 Half A — member resolution. The lowering projects EXACTLY the + // declared value object's members, matched by NAME against the @via + // TERMINAL entity (Supplier, here). Failing open on an unresolved member is + // how #270 turned a curated value object into the full entity. + + test("a VO member with no matching field on the @via terminal fails", async () => { + // SupplierBrief declares "nickname"; Supplier has no such field. + const src = model( + WHOLE_OBJECT, + `{ "field.long": { "name": "id" } }, { "field.string": { "name": "nickname" } }`, + ); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_COLLECT_MEMBER_UNRESOLVED"); + expect(hit).toBeDefined(); + expect(hit?.message).toContain("nickname"); + expect(hit?.message).toContain("Supplier"); + }); + + test("a VO member whose type differs from the matched field fails", async () => { + // SupplierBrief declares "name" as a field.long; Supplier.name is a field.string. + const src = model( + WHOLE_OBJECT, + `{ "field.long": { "name": "id" } }, { "field.long": { "name": "name" } }`, + ); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_COLLECT_WHOLE_OBJECT" && e.message.includes("value-object member")); + expect(hit).toBeDefined(); + expect(hit?.message).toContain("'name' is field.long"); + expect(hit?.message).toContain("Supplier.name' is field.string"); + }); + + test("a VO member whose ARRAY-NESS differs from the matched field fails", async () => { + // SupplierBrief declares "name" as field.string isArray; Supplier.name is a + // scalar field.string. Same subType, different array-ness — the #185 + // type-preserving doctrine this rule cites judges BOTH axes (the sibling + // _checkPassthroughType compares the `field.[]` label for exactly + // this reason). Comparing subType alone would project an array into a + // scalar member. + const src = model( + WHOLE_OBJECT, + `{ "field.long": { "name": "id" } }, { "field.string": { "name": "name", "isArray": true } }`, + ); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_COLLECT_WHOLE_OBJECT" && e.message.includes("value-object member")); + expect(hit).toBeDefined(); + expect(hit?.message).toContain("'name' is field.string[]"); + expect(hit?.message).toContain("Supplier.name' is field.string"); + }); +}); diff --git a/server/typescript/packages/metadata/test/validation-filterable-array.test.ts b/server/typescript/packages/metadata/test/validation-filterable-array.test.ts new file mode 100644 index 000000000..f78814276 --- /dev/null +++ b/server/typescript/packages/metadata/test/validation-filterable-array.test.ts @@ -0,0 +1,65 @@ +// #335 Half B — an array field has no filter-operator band either. +// +// filterSubTypeFor (codegen-ts/src/templates/filter-allowlist.ts) falls +// through to "string" for anything unrecognised and never consults isArray, +// so a `field.string isArray: true @filterable: true` previously emitted a +// like/eq rule against a text[] column — SQL that cannot execute. No FR-009 +// operator applies to a collection column, the same reason field.object is +// already rejected, so this reuses ERR_FILTERABLE_UNSUPPORTED_SUBTYPE. +// +// Uses the same MetaDataLoader().load([new InMemoryStringSource(...)]) API +// as the neighbouring filterable-attrs.test.ts — MetaDataLoader has no +// throwing loadFromString entry point; load() collects errors on the +// returned LoadResult instead of throwing. + +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader } from "../src/loader/meta-data-loader.js"; +import { InMemoryStringSource } from "../src/loader/meta-data-source.js"; + +const model = (fieldJson: string) => `{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { + "name": "Product", + "children": [ + { "source.rdb": { "@kind": "table", "@table": "products" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "@fields": ["id"] } }, + ${fieldJson} + ] + }} + ] + } +}`; + +async function loadErrors(src: string) { + const result = await new MetaDataLoader().load([ + new InMemoryStringSource(src, { id: "meta.demo.json" }), + ]); + return result.errors as unknown as { code: string; message: string }[]; +} + +describe("@filterable on an array field", () => { + test("an array field marked @filterable fails to load", async () => { + const src = model( + `{ "field.string": { "name": "tags", "isArray": true, "@filterable": true } }`, + ); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_FILTERABLE_UNSUPPORTED_SUBTYPE"); + expect(hit).toBeDefined(); + expect(hit?.message).toContain("Product.tags"); + }); + + test("the same field WITHOUT isArray still loads", async () => { + const src = model(`{ "field.string": { "name": "tags", "@filterable": true } }`); + const errors = await loadErrors(src); + expect(errors).toEqual([]); + }); + + test("an array field NOT marked @filterable still loads", async () => { + const src = model(`{ "field.string": { "name": "tags", "isArray": true } }`); + const errors = await loadErrors(src); + expect(errors).toEqual([]); + }); +}); diff --git a/server/typescript/packages/metadata/test/validation-sortable-array.test.ts b/server/typescript/packages/metadata/test/validation-sortable-array.test.ts new file mode 100644 index 000000000..aaeac5648 --- /dev/null +++ b/server/typescript/packages/metadata/test/validation-sortable-array.test.ts @@ -0,0 +1,76 @@ +// #335 Half B — @sortable gets the same subtype/array validation @filterable +// already had. +// +// @sortable defaults FROM @filterable, so it is independently set only when +// explicit — and nothing validated it: a @sortable JSON or array column +// passed the loader and emitted a sort entry over a column no dialect can +// ORDER BY meaningfully. +// +// Uses the same MetaDataLoader().load([new InMemoryStringSource(...)]) API +// as the neighbouring validation-filterable-array.test.ts — MetaDataLoader +// has no throwing loadFromString entry point; load() collects errors on the +// returned LoadResult instead of throwing. + +import { describe, test, expect } from "bun:test"; +import { MetaDataLoader } from "../src/loader/meta-data-loader.js"; +import { InMemoryStringSource } from "../src/loader/meta-data-source.js"; + +const model = (fieldJson: string) => `{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { + "name": "Spec", + "children": [ + { "source.rdb": { "@kind": "table", "@table": "specs" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "@fields": ["id"] } } + ] + }}, + { "object.entity": { + "name": "Product", + "children": [ + { "source.rdb": { "@kind": "table", "@table": "products" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "@fields": ["id"] } }, + ${fieldJson} + ] + }} + ] + } +}`; + +async function loadErrors(src: string) { + const result = await new MetaDataLoader().load([ + new InMemoryStringSource(src, { id: "meta.demo.json" }), + ]); + return result.errors as unknown as { code: string; message: string }[]; +} + +describe("@sortable subtype validation", () => { + test("an array field marked @sortable fails to load", async () => { + const src = model( + `{ "field.string": { "name": "tags", "isArray": true, "@sortable": true } }`, + ); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_SORTABLE_UNSUPPORTED_SUBTYPE"); + expect(hit).toBeDefined(); + expect(hit?.message).toContain("Product.tags"); + }); + + test("a field.object marked @sortable fails to load", async () => { + const src = model( + `{ "field.object": { "name": "spec", "@objectRef": "acme::shop::Spec", "@storage": "jsonb", "@sortable": true } }`, + ); + const errors = await loadErrors(src); + const hit = errors.find((e) => e.code === "ERR_SORTABLE_UNSUPPORTED_SUBTYPE"); + expect(hit).toBeDefined(); + expect(hit?.message).toContain("Product.spec"); + }); + + test("a plain scalar marked @sortable still loads", async () => { + const src = model(`{ "field.string": { "name": "sku", "@sortable": true } }`); + const errors = await loadErrors(src); + expect(errors).toEqual([]); + }); +}); diff --git a/spec/metamodel/origin.json b/spec/metamodel/origin.json index e6be6d33b..6ddd1dbb7 100644 --- a/spec/metamodel/origin.json +++ b/spec/metamodel/origin.json @@ -20,15 +20,15 @@ { "type": "origin", "subType": "aggregate", - "description": "A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup of @of).", + "description": "A value reduced from the related row-set reached along a relationship path (@via) from the base entity: count/sum/avg/min/max over a column (@of); any/all predicate quantifiers over a @filter; or collect (an array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted).", "whenToUse": "A projection needs a value derived by reducing related rows — a count/sum/avg/min/max, a 'did any/every related row match' flag, or an array of collected values. Declare it instead of hand-writing the aggregate query — it stays consistent and regenerates.", - "rules": "@via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max/collect and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only.", + "rules": "@via may be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (single-hop-unique inference; FR-024, ADR-0029). Multi-hop paths must always be stated explicitly. @of is required for count/sum/avg/min/max and forbidden for any/all (which quantify over rows via @filter, not a column). @filter is required for any/all. The field must be isArray:true for collect and isArray:false for every other @agg. @distinct and @orderBy are collect-only. On @agg:collect @of is OPTIONAL: omitting it declares a WHOLE-OBJECT rollup, which collects each related row as the carrying field's declared @objectRef value object instead of one scalar column. A whole-object rollup requires a field.object carrying @objectRef, requires that @objectRef to name an object.value, requires an explicit @via (there is no @of entity to infer the path from), and refuses @distinct. Its @orderBy keys resolve against the @via TERMINAL entity, not the head or a middle hop. Its value-object members bind to the terminal entity's fields BY NAME — member name == terminal field name, deliberately NOT extends, so one value object stays collectable from two different entities — and every member must match a terminal field agreeing on BOTH field. and array-ness.", "children": [ - { "type": "attr", "subType": "string", "name": "agg", "min": 1, "max": 1, "allowedValues": ["count", "sum", "avg", "min", "max", "any", "all", "collect"], "description": "The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup of @of — the field must be isArray)." }, - { "type": "attr", "subType": "string", "name": "of", "min": 0, "max": 1, "description": "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max/collect; forbidden for any/all (which quantify over rows via @filter, not a column)." }, + { "type": "attr", "subType": "string", "name": "agg", "min": 1, "max": 1, "allowedValues": ["count", "sum", "avg", "min", "max", "any", "all", "collect"], "description": "The reducing function applied over the related row-set: count/sum/avg/min/max (numeric/ordinal reduces over @of); any/all (predicate quantifiers over @filter — @of forbidden; empty set → any=false, all=true); collect (array rollup — of the @of column, or of the carrying field.object's declared @objectRef value object when @of is omitted; the field must be isArray)." }, + { "type": "attr", "subType": "string", "name": "of", "min": 0, "max": 1, "description": "Dotted Entity.field reference identifying the column being aggregated (e.g. 'Week.durationMinutes'). Required for count/sum/avg/min/max; OPTIONAL for collect, where absent means a whole-object rollup of the field's declared @objectRef value object; forbidden for any/all (which quantify over rows via @filter, not a column)." }, { "type": "attr", "subType": "string", "name": "via", "min": 0, "max": 1, "description": "Dotted relationship path from the base entity to the aggregated rows (e.g. 'Program.weeks' or 'Program.weeks.workouts'). May be omitted only when exactly one single-hop relationship leads from the base entity to the @of entity (FR-024, ADR-0029)." }, { "type": "attr", "subType": "filter", "name": "filter", "min": 0, "max": 1, "description": "Optional structured predicate scoping which related rows the aggregate spans (required for any/all, where it is the quantified predicate). A portable attr.filter object (eq/ne/in/isNull with and/or), desugared to canonical { field: { op: value } } at parse time; codegen renders it per target (e.g. SQL FILTER (WHERE ...) or SQLite CASE WHEN for a relational view)." }, - { "type": "attr", "subType": "boolean", "name": "distinct", "min": 0, "max": 1, "description": "Set (collect-only) to dedupe collected values (set semantics)." }, + { "type": "attr", "subType": "boolean", "name": "distinct", "min": 0, "max": 1, "description": "Set (collect-only) to dedupe collected values (set semantics). Not supported on a whole-object collect (@of omitted): it is a guaranteed no-op whenever the value object carries the primary key, and a silent no-op is worse than a refusal." }, { "type": "attr", "subType": "string", "name": "orderBy", "isArray": true, "min": 0, "max": 1, "description": "Ordering keys as 'field[:asc|desc]' (default asc) over the related entity's fields; nulls sort last. On @agg:collect sets element order (non-distinct only); on origin.first (required) selects the row. Semantic — carries no SQL syntax." } ] },