perf: SorterInsert reads GROUP BY sort key from source registers - #662
Merged
Conversation
SorterInsert always re-decoded its sort-key columns out of the record blob it had just been handed, even though those same values were still sitting in registers moments earlier, before MakeRecord encoded them — exactly the encode-then-decode overhead HashAggFind's own P3 parameter already avoids for the (unused) hash-agg path. Profiling read_group_by_agg found this redundant decode (decode_bytes_upto -> parse_header_into/ decode_serial_value) a meaningful share of per-row cost. SorterInsert gains an optional p3 (source-register run) honored when p5 is nonzero; compile_grouped_scan's MakeRecord source registers are still live at that point (record_reg is allocated after them), so it opts in. Every other SorterInsert emitter (ORDER BY paths) leaves p5=0 and keeps the original decode-from-blob behavior, so this is purely additive. spend: ~1x estimate (small). Refs #660 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
iheitlager
force-pushed
the
fix/656-hash-agg-allocations
branch
from
August 30, 2026 10:24
6f445fa to
89ec86f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
HashAgg*code — see perf: reduce per-row allocations in hash-aggregation execution (HashAggFind/HashAggStep) #656's closing comment)read_group_by_aggbenchmarks ~3.3x slower than oracle (5.0-5.2ms vs ~1.5ms). Profiling (sampleon the release bench binary) foundSorterInsertre-decoding its own sort-key columns out of the record blob it had just been handed (decode_bytes_upto→parse_header_into/decode_serial_value) — even though those same values were still sitting in registers moments earlier, beforeMakeRecordencoded them.HashAggFindalready avoids exactly this pattern via its ownP3(register-run) parameter;SorterInsertnever got the same treatment.SorterInsertgains an optionalp3(source-register run), honored whenp5is nonzero —compile_grouped_scan'sMakeRecordsource registers are still live at that point (record_regis allocated after them), so it opts in. Every otherSorterInsertemitter (ORDER BY paths, join paths) leavesp5=0and keeps the original decode-from-blob behavior untouched — purely additive to the opcode.group_by_aggbench ~3.5-4% faster at both fixture sizes (1mb: ~5.1ms → ~4.9ms; 50mb: ~284ms → ~274ms). Real but modest — remaining cost is dominated by the sort itself (compare::compareinside the stable sort, inherently O(n log n)) and row encode/decode during the scan, which are closer to structural floors. Filed as follow-up ideas in perf: SorterInsert re-decodes GROUP BY sort-key from the record blob it just built #660 if pursued further.Test plan
cargo test --lib— 976 passedcargo test --test '*'— full integration/oracle-parity suite, no failurescargo clippy --all-targets— cleancargo bench --bench engine -- group_by_agg— before/after comparison shows measurable improvement, no regression on other Sorter-backed paths (ORDER BY unaffected,p5=0default preserved)spend: ~1x estimate (small), plus the #656 diagnosis pass that preceded it.