Account for transformer-retained batches in symmetric hash join - #24364

Open

kosiew wants to merge 25 commits into

apache:mainfrom

kosiew:memcalc-03-23393

Commits on Aug 14, 2026

**feat(transformer, stream): add RecordBatchMemoryCounter for retained batches, deduplicate shared buffers, and update reservation lifecycle**
kosiew
committed
716b46c
refactor(transformer): deduplicate retained-batch accounting in private helper (no public API/behavior change)
kosiew
committed
958cc74
fix(symmetric_hash_join): preserve retained batch memory accounting
kosiew
committed
71eb7e9
feat: improve recursive unique ArrayRef overhead accounting and coverage for children (Struct, List, Map, Union, Dict, RunEnd), plus regression test for shared nested StructArray child
kosiew
committed
4e9965c
fix(stream-boundary): add nested shared-child regression test for NoopBatchTransformer and BatchSplitter
kosiew
committed
22f6a66
docs: document BatchTransformer::count_memory shared counter and dedup contract; add regression-boundary rationale comment for 2_400 test limit
kosiew
committed
188fea5
feat(memory): add table-driven shared-child cases for List, Map, Union, Dictionary, RunEnd and improve shared allocation delta checks
kosiew
committed
709a488
feat: add test‑private assertion helper and remove post‑construction ArrayRef clones
kosiew
committed
fb97c27
test(symmetric_hash_join): add tests for transformer output exhaustion and memory usage
kosiew
committed
792fa41

Commits on Aug 15, 2026

Commits on Aug 18, 2026

Commits on Aug 20, 2026

fix(runend,union): remove object‑identity branches/tests, add size_without_input_buffer, update metrics after successful resize, add failed‑resize regression assertion
kosiew
committed
200662b

Commits on Aug 26, 2026