perf: inline hot varint/cell decode calls on the SeekRowid path (#657) - #659
Merged
Conversation
…rch path (#657) Profiling with `sample` on the read_join bench (16700-row bench_data joined to bench_lookup on its PK) showed 56% of samples landing in TableCursor::seek's binary search — decode_varint, decode_cell_head, and read_cell_pointer were compiled as real (non-inlined) function calls, paid twice per probed cell across ~8-9 binary-search probes per row against bench_lookup's ~230-cell leaf pages. read_full_scan, which never calls seek(), was already faster than the oracle (0.71x), confirming the gap was isolated to this path rather than general record decode. Marking these three small, always-hot functions #[inline] lets LTO fold them into the binary-search loop. read_join drops from 3.63ms to 2.95ms (measured via `cargo bench --bench crud -- read_join`), closing the oracle gap from 1.9x to 1.5x. spend: ~1.5x estimate (initial profiling pointed to diffuse overhead with no fix; a second pass comparing read_join against read_full_scan and the compiled opcode stream against oracle's EXPLAIN output pinned the actual hot loop).
4 tasks
iheitlager
force-pushed
the
fix/657-read-join-inline-hot-path
branch
from
August 30, 2026 08:44
f57e957 to
caa556f
Compare
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
read_joinbenched at 1.9x slower than the oracle with no static smoking gun. Deeper profiling (this PR) pinpointed the actual hot loop:sample-profiling the read_join bench (16700-rowbench_dataJOINbench_lookupon PK) showed 56% of samples landing inTableCursor::seek's binary search.read_full_scan(never callsseek()) showed that path already beats the oracle (0.71x) — so the gap was isolated toseek(), not general record decode.EXPLAINconfirmed byte-for-byte identical opcode shape/count per row — no extra/missing VDBE work, no access-path bug.bench_lookup's b-tree is depth-2 (root interior, 5 leaf children, ~227-240 cells/leaf), so eachSeekRowiddoes ~8-9 binary-search probes, each callingdecode_varint(x2),decode_cell_head, andread_cell_pointer— all compiled as real, non-inlined function calls (confirmed via disassembly of the hot address range).Marking those three functions
#[inline](src/record/varint.rs,src/btree.rs) lets LTO fold them into the loop.Result
cargo bench --bench crud -- read_join:Confirms the issue's own "diffuse per-row overhead" hypothesis was half right — the overhead was concentrated in one specific uninlined loop, not spread evenly.
Test plan
cargo test --release— full suite passes (allbtree/recordunit tests green)cargo clippy --release --all-targets— cleancargo bench --bench crud -- read_join— 18.6% improvement, statistically significantCloses #657
spend: ~1.5x estimate — first profiling pass found no fix (diffuse overhead), a second pass comparing against
read_full_scanand oracle'sEXPLAINoutput pinned the actual hot loop.