Skip to content

fix(vector-search): CROSS JOIN to stop vector_top_k being evaluated per-row - #264

Closed
sloemo01 wants to merge 1 commit into
tickernelz:mainfrom
sloemo01:fix/vector-search-container-tag-join
Closed

fix(vector-search): CROSS JOIN to stop vector_top_k being evaluated per-row#264
sloemo01 wants to merge 1 commit into
tickernelz:mainfrom
sloemo01:fix/vector-search-container-tag-join

Conversation

@sloemo01

Copy link
Copy Markdown

Fixes #247

What problem this solves

Memory search with a container_tag filter was taking ~8 seconds on a 337-row shard instead of tens of milliseconds, causing the Web UI to abort the fetch (Error: signal is aborted without reason).

Why it happened

searchKind() combined vector_top_k() with an INNER JOIN memories m ON m.rowid = v.id. When a container_tag filter is present, SQLite's planner drove from the memories table (using idx_container_tag) and evaluated the vector_top_k virtual table once per memory row — turning one ANN lookup into N nested-loop evaluations.

The fix

Changed JOINCROSS JOIN, which forces the intended TVF-first plan:

SELECT m.id AS id, vector_distance_cos(m.<col>, vector32(?)) AS dist
FROM vector_top_k('<index>', vector32(?), ?) AS v
CROSS JOIN memories m ON m.rowid = v.id
WHERE m.<col> IS NOT NULL [AND m.container_tag = ?]

The WHERE filters still apply, so semantics are unchanged. This matches the issue reporter's own benchmark: INNER JOIN ≈ 8409ms, CROSS JOIN29ms.

User impact

Memory search with tag filters goes from ~8s (and UI aborts) to ~30ms.

Verification

  • bun run typecheck — passes
  • bun test tests/turso-vector-search.test.ts — passes (exercises the tag-filtered path)
  • bun test tests/turso-exact-fallback.test.ts tests/turso-vector-utils.test.ts — passes

Note

Both branches of searchKind() (with and without container_tag) were changed so the container_tag="" path also gets the correct plan.

searchKind() combined vector_top_k() with an INNER JOIN on memories,
and when a container_tag filter was present SQLite's planner drove from
the memories table (using idx_container_tag) and evaluated the
vector_top_k virtual table once per memory row — turning a single ANN
lookup into N nested-loop evaluations.

On a 337-row shard this made memory search take ~8s (7982/8113/8064ms)
instead of tens of milliseconds, causing the Web UI to abort the fetch.

Changing the JOIN to CROSS JOIN forces the intended TVF-first plan
(SCAN v VIRTUAL TABLE -> SEARCH m USING INTEGER PRIMARY KEY), which the
issue reporter benchmarked at ~28ms. The WHERE filters on
m.<column> IS NOT NULL and m.container_tag still apply, so semantics
are unchanged.

Fixes tickernelz#247
@sloemo01

Copy link
Copy Markdown
Author

Superseded by #257 which already applies the same CROSS JOIN fix for #247. Closing to avoid duplicating @lindixu6-hash's work.

@sloemo01 sloemo01 closed this Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: memory search takes ~8s when filtering by container_tag (vector_top_k + JOIN bad query plan)

1 participant