fix: serve pg_class and pg_namespace from SQLite instead of Rust handlers (#87) - #101
Merged
Conversation
\dt returns no tables because psql's `n.nspname !~ '^pg_toast'` predicate is rewritten to `NOT regexp(...)` before catalog interception, WhereEvaluator defaults unknown functions to true, and NOT inverts that into a universal row filter. Design deletes PgClassHandler and lets SQLite execute pg_class queries, since the views and required UDFs already exist. Also records two further defects found while investigating: `~` is silently ignored on catalog tables, and pg_class OIDs (hash31) already disagree with the persisted OIDs (unicode formula) used by pg_constraint/pg_index/pg_attrdef. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
8 tasks, TDD throughout: failing test for #87, trusted_schema guarantee, migration v28 (enriched pg_class view + information_schema namespace), removal of PgClassHandler, regression tests for the regex operators and cross-catalog joins, then follow-up issues. Also corrects the spec: the new migration is v28, not v26 -- v26 and v27 are both registered, and v26 owns the current pg_class view. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ass and correct root cause analysis
pragma_table_info() is a virtual table; SQLite refuses virtual tables inside views unless trusted_schema is ON. It defaults to ON in the C API (so this already works), but the sqlite3 CLI disables it, and a future default change or defensive-mode setting could silently break Task 3's pg_class.relnatts column. Pin it explicitly on every connection-init site that registers UDFs: the per-session connection pool, the shared initial/migration connection, and the --migrate CLI path. Add a regression test that creates a view calling pragma_table_info() and queries it, pinning the guarantee end-to-end over the wire protocol.
Adds a pg_class view with the full 33 PostgreSQL columns and a pg_namespace view with pg_catalog/public/information_schema rows, so SQLite can serve pg_class directly instead of the hand-rolled PgClassHandler evaluator. Internal pg_*/information_schema_* relations are assigned to their proper namespaces via the relname pattern. PgClassHandler still intercepts pg_class queries and shadows the new view until it is removed in a follow-up task; most new tests in tests/pg_class_view_test.rs are expected to fail until then. Also updates tests/migration_test.rs's hardcoded migration counts (27->28) to match the new migration.
Implementing Task 3 revealed pg_namespace is intercepted separately by handle_pg_namespace_query, which hardcodes two rows and shadows v28's new information_schema namespace. Human ruling: remove that branch in Task 4. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…lers (#87) Removes the pg_class and pg_namespace interception branches in CatalogInterceptor::check_table_factor along with the now-orphaned PgClassHandler module and handle_pg_namespace_query function, so these queries fall through to SQLite and execute against the v28 catalog views instead of the hand-rolled Rust responders. Also removes is_count_star_projection/count_response, which were only reachable from the deleted pg_namespace handler. This is the fix for \dt reporting "Did not find any tables": the RegexTranslator-rewritten pg_table_is_visible/regexp predicates in the \dt query were being evaluated by WhereEvaluator's default-true fallback for unknown functions, inverted by NOT into an always-false filter. Routing to SQLite avoids that evaluator entirely.
… stale pg_namespace test expectations Code review response for the pg_class/pg_namespace-from-SQLite change (#87): - --in-memory mode: switch to a shared-cache SQLite URI (file:pgsqlite_mem?mode=memory&cache=shared) and keep the migration connection alive for the DbHandler's lifetime, since SQLite destroys a shared-cache in-memory database as soon as its last connection closes. Without this, the pg_class/pg_namespace views created at migration time vanished before the first session connection opened, breaking \dt and every other catalog query in --in-memory mode. - Unify all table-identity OID generation on one canonical formula (crate::utils::generate_table_oid), replacing the old hash31 formula in pg_attribute.rs, pg_trigger.rs, and pg_sequence.rs. Those had drifted from the formula used by the v28 pg_class view and constraint_populator.rs, breaking pg_class.oid joins to pg_attribute.attrelid/pg_trigger.tgrelid/pg_sequence's table OID in the default file-backed mode. - Repoint catalog_alias_test.rs's pg_namespace-only assertions to pg_roles/pg_database, since pg_namespace is now a SQLite view (3 rows, including information_schema) rather than a Rust-intercepted catalog, and the tests exist to exercise CatalogInterceptor's projection logic. - Apply the same shared-cache-URI fix to catalog_where_simple_test.rs, which constructs DbHandler directly and bypasses main.rs. - Reword a stale "PgClassHandler" doc comment in migration/registry.rs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
pg_constraint.rs:256 has a fourth OID formula (DefaultHasher) and synthesizes conrelid at query time, so pg_class JOIN pg_constraint returns zero rows. Measured: pg_class.oid=197947 vs pg_constraint.conrelid=51945. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…nt joins to SQLite (#87) pg_constraint.rs synthesized conrelid/confrelid with Rust's DefaultHasher, diverging from the canonical crate::utils::generate_table_oid formula that pg_class, pg_attribute, and constraint_populator already agree on. Delegate to the shared implementation instead, matching pg_attribute.rs and constraint_populator.rs. That alone was not sufficient: the query interceptor routed any pg_class-main JOIN into another SQLite-backed catalog table (pg_constraint, pg_index, pg_depend, ...) through PgConstraintHandler's single-table WHERE evaluator, which strips qualified identifiers like c.oid/c.relname down to bare column names and evaluates them against pg_constraint's own row data - so the join predicate could never match regardless of OID consistency. Extend the existing "let SQLite handle it" fallback (already used when pg_index or pg_attribute is the main table) to also apply when pg_class is the main table and the join target is one of those real SQLite-backed catalog tables, so the query executes as genuine SQL against the pg_class view and the persisted pg_constraint table. Add tests/catalog_join_test.rs covering both the OID formula and the cross-catalog join. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…fallback (#87) Review finding on the pg_class/pg_constraint JOIN fix: pg_stats and pg_tablespace are handled entirely by the catalog interceptor with no backing SQLite view (migrations v19 and v24 say so explicitly). They had leaked into the "let SQLite handle this JOIN" fallback list, which is only correct for tables that really exist as SQLite views/tables. On the default connection this was masked by substring interception further down db_handler.rs, but on the connection-pooling path (query_router.rs -> read_only_handler.rs) there is no such rescue, so a pg_class/pg_tablespace or pg_class/pg_stats JOIN would fail with "no such table" under PGSQLITE_USE_POOLING=true. Remove both from the fallback list and extract the now-shared 7-name list into is_sqlite_backed_catalog() so the two call sites (main-table check and the new join-target check) can't drift again - the drift is what let pg_stats/pg_tablespace end up in the list in the first place. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Full test suite (1296 tests, 194 binaries) passes with zero failures; earlier commits on this branch already fixed the expected pg_class OID and pg_namespace fallout, so no test files needed changes. Only CLAUDE.md's stale "Current Migrations (v1-v25)" section needed updating to v28, documenting v26-v27 and the new v28 pg_class/pg_namespace work.
…ly (#87) generate_table_oid used name.len() (UTF-8 byte count) where the v28 pg_class/ pg_namespace view SQL uses SQLite's length(name) (character count), so any non-ASCII table name got a pg_class.oid that didn't match its persisted conrelid/indrelid/adrelid/attrelid, causing silent zero-row joins. The per-character arithmetic also overflowed u32 for high-codepoint leading characters (e.g. "日本語"), panicking in debug builds. Switch length to name.chars().count() and widen the intermediate arithmetic to u64 before reducing with % 1_000_000 + 16_384, which keeps the result well within u32 range. All ASCII OIDs (e.g. "customers" -> 197947) are byte-identical to before. Non-ASCII OIDs necessarily change to agree with the SQL side; any such OID already persisted on disk was produced by the old byte-counting formula and needs a follow-up data migration, not attempted here. Adds unit tests pinning the ASCII value, verifying non-ASCII values against the sqlite3 CLI running the identical v28 SQL expression, and covering a high-codepoint leading character that used to panic. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
#87) - object_resolver.rs: correct the comments on resolve_table_oid and its private generate_table_oid helper, which claimed to use "the same algorithm as pg_class view". They actually use a DefaultHasher and intentionally diverge from crate::utils::generate_table_oid; changing the formula would require a data migration for persisted __pgsqlite_comments.object_oid rows, so only the comment is corrected. - query_interceptor.rs: convert the println! debug traces that fire on every catalog query to debug!, using the tracing macro already imported in this file. - db_handler.rs: convert the eprintln! in query_with_session, which printed every session query's text to stdout, to debug!. - catalog_alias_test.rs: rename four test functions still named test_pg_namespace_* even though pg_namespace is now served directly from SQLite and their bodies query pg_roles instead (test bodies unchanged). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
No test exercised the shared-cache in-memory URI
(file:pgsqlite_mem?mode=memory&cache=shared) that src/main.rs uses for
--in-memory against pg_class specifically. Reverting that URI to bare
":memory:" would leave the whole suite green while re-breaking the hard
error ("no such table: pg_class") that a second session's private, empty
in-memory database used to produce.
Adds a regression test that opens a DbHandler on the shared-cache URI,
creates a table via one (temporary, memory-mode) session connection, then
queries pg_class from an independently created second session connection
and asserts the table is visible there. Verified the test actually guards
the regression by temporarily switching its URI to bare ":memory:", which
reproduces the exact "no such table: pg_class" failure, then reverting.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The regression test added for the --in-memory fix hardcoded its own copy of the shared-cache URI, so reverting src/main.rs to a bare ":memory:" left the test green. It guarded DbHandler's keepalive and cross-session visibility, but not the decision that was actually broken. Move that decision into Config::resolve_db_path(), backed by in_memory_db_uri()/IN_MEMORY_DB_NAME, and have both main.rs and the test go through it. Verified by temporarily making resolve_db_path return ":memory:": the test fails with "--in-memory must not resolve to a bare :memory:". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
generate_table_oid was widened to u64 earlier in this branch because char1 * 1_000_000 overflows u32 once the leading codepoint exceeds 4294 — a panic in debug builds, a silent wrap in release. Its sibling generate_oid in the same module still had the identical defect, and reaches persisted catalog OIDs through migration v5's populate_catalog_tables, so a database containing such a table name panicked on the upgrade path. Widen the same way. This is strictly panic-eliminating: measured against the pre-widening body with checked arithmetic, every input that already produced a value produces the same value, non-ASCII names included (café -> 1007190 before and after). Only previously-overflowing inputs change, from panic to a defined value. Pinned in tests. Also drop the module's doc comment claiming generate_oid "uses the same formula as the pg_class view" — it is a different, six-position formula for constraint and sequence OIDs, and only generate_table_oid matches the view. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #87
The bug
\dtreported "Did not find any tables" on a database that has tables. Threemechanisms combined to turn one psql predicate into a universal row filter:
RegexTranslatorruns before catalog interception and rewritesn.nspname !~ '^pg_toast'intoNOT regexp('^pg_toast', n.nspname).WhereEvaluatorhas no case forregexp(). Unknown functions fall throughto
true— "default to including the row".NOTinverts that default into an exclusion.!trueisfalse. Every rowfails.
The dedicated
PGRegexMatch/PGRegexNotMatcharms inWhereEvaluatorare deadcode on this path — the operator is gone before the AST reaches them.
Two further defects sat behind it, so a narrow regex fix would not have worked:
PgClassHandlersynthesized only its own columns, so theLEFT JOIN pg_namespacewas never materialized andn.nspnameevaluated toNone(
\dt'sn.nspname <> 'pg_catalog'passed only by accident, becauseNone != Some(..)); and joins were not executed at all.The fix
Rather than patch the WHERE evaluator, delete the hand-rolled Rust handlers for
pg_classandpg_namespaceand let SQLite execute those queries against realviews. Joins,
WHERE,regexp(),ORDER BY, and projection all become theengine's job. Migration v28 enriches the
pg_classview from 25 to 33 columnsso nothing is lost, computing
relnattsfrompragma_table_infoandrelhastriggersfromsqlite_masterrather than hardcoding them.The alternative of bailing out of interception on hard queries was rejected: the
bail-list is itself a guess-list, and a query that succeeds while returning
wrong data never triggers a fallback. Deleting the handler leaves one code path
and no decision logic to drift — a missing column is now a loud SQLite error
rather than silent wrong rows.
Design:
docs/superpowers/specs/2026-08-10-pg-class-sqlite-engine-design.mdAlso fixed
Found while implementing; each was silently returning wrong answers.
~was silently ignored on catalog tables, returning unfiltered resultswith no error.
pg_class.oidnever matched.pg_classserved
hash31OIDs whilepg_constraint,pg_index,pg_attrdef, andpg_dependpersist unicode-formula OIDs. Measured on a fresh database:pg_class.oid(customers) = 197947butpg_constraint.conrelid = 51945, soFK discovery — the first ORM example in CLAUDE.md — returned zero rows. Four
different OID formulas existed; there is now one,
crate::utils::generate_table_oid, which every producer delegates to.--in-memorycatalog queries hard-errored. Migrations ran on a throwaway:memory:connection while each session opened its own private emptydatabase. Now uses a shared-cache URI with a keepalive connection.
\dtno longer lists pgsqlite's ownpg_*relations, because v28 assignsthem to the
pg_catalognamespace (oid 11) andinformation_schema_*to anew
information_schemanamespace (oid 13000). No special-casing inpgsqlite — psql's own
nspnamepredicate does the hiding.generate_table_oidandgenerate_oidoverflowedu32on a leadingcodepoint above 4294, panicking in debug builds and silently wrapping in
release. Both now compute in
u64.name.len()(UTF-8 bytes); SQLite useslength()(characters).caféproduced
996619in Rust and996612in SQL, so any non-ASCII table name gota
pg_class.oidmatching none of its persisted join keys. Every ASCII valueis unchanged —
customersis still197947.Three existing view bugs fixed in passing:
relkind_fullis not a realPostgreSQL column (dropped),
relreplidentshould be'd'not'v', andrelispartitionshould be'f'not't'.Behavior changes worth knowing
--in-memorysessions now share one database. Previously each session gotits own (empty, unmigrated) one, which is why catalog queries failed. This is
the fix, but it is a real semantic change.
pg_classOIDs changed value fromhash31to the unicode formula. Thisremoves an inconsistency — the new values are the ones the rest of the
catalog already stored on disk — but any external tooling that memoized a
pg_classOID will see it move.written by the byte-counting version and will not match until a data
migration. The prior state was already broken (view and persisted value
disagreed), so this is a fix-forward.
relhasindexis hardcoded't'andrelfilenode'0'in the view, wherethe deleted handler computed them from
PRAGMA index_listand the oid. Bothinherited from the v26 view and both fail safe.
\dtnow hides pgsqlite's internals whileinformation_schema.tablesstillshows them — it builds its own row set with a hardcoded
'public'. That gapis information_schema.tables lists pgsqlite's materialized pg_* and information_schema_* relations #88.
Testing
1302 passed, 0 failed, 195 binaries. Baseline before this branch was 1296/194.
New coverage: psql's exact
\dtquery (assertingcustomersappears andthat
pg_constraint/pg_attrdef/pg_index/pg_dependdo not);~and!~each pinned with a positive and a negative case so neither can pass vacuously;
pg_class JOIN pg_constraint ON conrelid = oidreturning rows; all 33 columnsselectable with
relnattsmatching real column counts; the OID formula pinnedagainst values computed independently through the
sqlite3CLI; and cross-sessionpg_classvisibility on the in-memory URI.\dtwas also verified end-to-end through real psql 18 against the built binary.Follow-ups filed
#93 OID collisions · #94
WhereEvaluatorunknown-predicate-under-NOT·#95 migrate the remaining catalog handlers · #96 JOIN routing bypassed by the
system-function gate · #97 pooling path registers no UDFs · #98
obj_description()can never match · #99regclassnon-canonical OID ·#100
PgAttributeHandlermissingCOUNT(*)🤖 Generated with Claude Code