Add --hide-internal-tables flag to hide __pgsqlite_* tables from client sqlite_master queries (#80)

[erans]

Closes #80.

Problem

pgsqlite's catalog queries already filter their own bookkeeping tables, but a sqlite_master / sqlite_schema query written directly by a client is passed through verbatim. Schema browsers, ORMs, and \dt-style tooling therefore see pgsqlite's internals mixed in with the user's tables. On a fresh database with one user table, a client listing returns 37 internal rows next to customers:

Kind	Count	Examples
__pgsqlite_* tables	15	__pgsqlite_schema, __pgsqlite_migrations
Indexes without the prefix	8	idx_enum_values_label, idx_comments_lookup
Implicit unique indexes	14	sqlite_autoindex___pgsqlite_schema_1

Note that a name LIKE '__pgsqlite_%' test matches only 15 of those 37. The other 22 are caught via tbl_name, which every index row in sqlite_master carries.

Solution

A new --hide-internal-tables flag (env PGSQLITE_HIDE_INTERNAL_TABLES), default off — with no flag set, behaviour is unchanged.

When on, SqliteMasterFilter parses the client's SQL and replaces each sqlite_master / sqlite_schema relation with a filtered derived table:

FROM (SELECT * FROM sqlite_master
      WHERE substr(name, 1, 11) <> '__pgsqlite_'
        AND (tbl_name IS NULL OR substr(tbl_name, 1, 11) <> '__pgsqlite_')) AS sqlite_master

The client's own projections, predicates, and joins are never modified, so there is no predicate to splice and nothing to corrupt — the failure mode that sank the earlier rewriting attempt in #82 on large NOT IN lists and ESCAPE clauses. Because the filtering happens inside the relation, count(*), joins, subqueries, CTEs, EXISTS, and SELECT sql are all correct with no extra handling.

substr(...) rather than LIKE '__pgsqlite_%': in LIKE, _ is a single-character wildcard, so the LIKE form also matches unrelated names such as abpgsqliteX.

Scope

Hidden: __pgsqlite_* tables and the index rows they own. Deliberately not hidden, and documented as such: the materialized pg_* / information_schema_* relations (filtering those is a broader policy call, since a user may legitimately name a table pg_something) and PRAGMA table_list.

Hiding is listing-only. SELECT * FROM __pgsqlite_schema still works when named explicitly, so a live deployment stays debuggable over the wire.

Hooks — client queries only

Protocol	Location
Simple	src/query/executor.rs, in preprocess_query
Extended	src/query/extended.rs, top of handle_parse, ahead of the prepared-statement cache and outside the unified_processor cfg block

Deliberately not at DbHandler::process_query, despite that being the single chokepoint all queries pass through. migration/runner.rs, metadata/enum_metadata.rs, rewriter/enum_rewriter.rs, cache/lazy_schema_loader.rs, and cache/schema.rs all probe sqlite_master for __pgsqlite_* names; filtering those would make SELECT 1 FROM sqlite_master WHERE name='__pgsqlite_metadata' return nothing and pgsqlite would re-run its migrations on every start.

Fail open throughout: a parse failure or unrecognized shape returns the query unchanged rather than erroring.

Notable fix included

The rewrite initially tripped pgsqlite's own SQL-injection detector. The added nesting level put sqlite_master at depth 2, so the depth > 1 "suspicious system table access" rule fired on benign listings over the extended protocol in text result format (psycopg2's default), logging HIGH-severity false positives into the security audit trail.

Fixed by exempting only pgsqlite's own generated wrapper from the depth increment. The exemption does not reset depth, so a client wrapping the generated form in their own subquery is still caught, and the recognized subquery is a fixed literal returning strictly fewer rows than the SELECT * FROM sqlite_master the detector already permits at depth 1.

Testing

• 20 unit tests on the translator: rewrite shapes (bare, aliased, main./temp. qualified, join, CTE, EXISTS, count(*), SELECT sql, sqlite_schema spelling), no-op cases (flag off, no reference, unparseable, otherdb. qualified), DELETE/UPDATE passthrough, INSERT ... SELECT rewrite, and a guard that the LIKE form is not generated.
• Wire-level integration tests in two separate binaries, one per flag state. The disabled binary asserts internal tables are still visible, proving the default is unchanged.
• Detector regression tests asserting the generated form is accepted and client-written nesting is still rejected.
• Full suite: 1281 passed, 0 failed.
• Manual two-restart verification against a real database with psql: a fresh start applies 27 migrations; a later start of that same database with the flag on logs zero migration activity, confirming internal probes still see the real catalog.

Follow-ups (not in this PR)

• The read-context restriction means CREATE VIEW v AS SELECT ... FROM sqlite_master is not rewritten, so a client can create a view over the unfiltered catalog once and query it thereafter. Consistent with the listing-only contract, but worth revisiting.
• \dt reports "Did not find any tables" on a database that has one.
• information_schema.tables lists all 27 materialized pg_* / information_schema_* relations alongside user tables.
• cargo test --lib sql_injection aborts on main today: db_handler's own tests construct a DbHandler, which dereferences CONFIG, which runs Config::parse() against the test binary's argv.

Supersedes #81, #82, and #83. The reasoning for choosing relation substitution over #83's result-row filtering is in docs/superpowers/specs/2026-08-05-hide-internal-tables-design.md: row filtering sees only what the client projected and fails open silently on count(*), joins, subqueries, and SELECT sql.

🤖 Generated with Claude Code