fix(vector-search): CROSS JOIN to stop vector_top_k being evaluated per-row - #264
Closed
sloemo01 wants to merge 1 commit into
Closed
fix(vector-search): CROSS JOIN to stop vector_top_k being evaluated per-row#264sloemo01 wants to merge 1 commit into
sloemo01 wants to merge 1 commit into
Conversation
searchKind() combined vector_top_k() with an INNER JOIN on memories, and when a container_tag filter was present SQLite's planner drove from the memories table (using idx_container_tag) and evaluated the vector_top_k virtual table once per memory row — turning a single ANN lookup into N nested-loop evaluations. On a 337-row shard this made memory search take ~8s (7982/8113/8064ms) instead of tens of milliseconds, causing the Web UI to abort the fetch. Changing the JOIN to CROSS JOIN forces the intended TVF-first plan (SCAN v VIRTUAL TABLE -> SEARCH m USING INTEGER PRIMARY KEY), which the issue reporter benchmarked at ~28ms. The WHERE filters on m.<column> IS NOT NULL and m.container_tag still apply, so semantics are unchanged. Fixes tickernelz#247
Author
|
Superseded by #257 which already applies the same CROSS JOIN fix for #247. Closing to avoid duplicating @lindixu6-hash's work. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #247
What problem this solves
Memory search with a
container_tagfilter was taking ~8 seconds on a 337-row shard instead of tens of milliseconds, causing the Web UI to abort the fetch (Error: signal is aborted without reason).Why it happened
searchKind()combinedvector_top_k()with anINNER JOIN memories m ON m.rowid = v.id. When acontainer_tagfilter is present, SQLite's planner drove from thememoriestable (usingidx_container_tag) and evaluated thevector_top_kvirtual table once per memory row — turning one ANN lookup into N nested-loop evaluations.The fix
Changed
JOIN→CROSS JOIN, which forces the intended TVF-first plan:The
WHEREfilters still apply, so semantics are unchanged. This matches the issue reporter's own benchmark:INNER JOIN≈ 8409ms,CROSS JOIN≈ 29ms.User impact
Memory search with tag filters goes from ~8s (and UI aborts) to ~30ms.
Verification
bun run typecheck— passesbun test tests/turso-vector-search.test.ts— passes (exercises the tag-filtered path)bun test tests/turso-exact-fallback.test.ts tests/turso-vector-utils.test.ts— passesNote
Both branches of
searchKind()(with and withoutcontainer_tag) were changed so thecontainer_tag=""path also gets the correct plan.