Add composable chunked decompression via a vtable method

[joseph-isaacs]

Summary

Execution decompression currently forces a choice between two bad options for consumers that only want to iterate values: fully materialize the array and then re-read it (two passes over DRAM), or hand-fuse a kernel per parent/child encoding pair. This PR adds a composable vtable method, decompress_chunks, that streams an array's decompressed values through a ChunkSink in cache-resident ~1024-element chunks, so decompression and transformation happen while each block is L1-resident and the full array is never materialized.

Key properties:

• Chunk-granular dynamic dispatch. One virtual call per ~1024-value chunk per encoding level; chunks are real typed slices (ChunkMut is a type-erased &mut [T]), so all per-element work stays monomorphized. Measured dispatch cost: ~60–70 ns per chunk ≈ 0.06 ns/element.
• Composable through the erased ArrayRef. Wrapper encodings (FoR, Patched) interpose a stack-allocated sink adapter and recurse into their child — no heap state on the way down; the sink chain is plain nested stack frames.
• Fused fast paths stay available. FoR spots a BitPacked child and streams through the fused FoRStrategy unpack kernel (reference folded into unchecked_unfor_pack), matching the hand-written monomorphized loop within noise.
• Streaming is an explicit capability, never a silent fallback. supports_decompress_chunks (default false) is propagated recursively through wrappers, so support of the whole tree is decided before any chunk is emitted; ArrayRef::decompress_chunks errors on unsupported trees, and the materializing two-pass fallback must be chosen by name via decompress_chunks_or_materialize.

Changes

• vortex-array/src/chunk_iter.rs (new): ChunkSink, ChunkMut, DECOMPRESS_CHUNK_LEN, the ArrayRef entry points (supports_decompress_chunks, decompress_chunks, decompress_chunks_or_materialize), a debug-build coverage/contiguity checker, and the explicit materializing fallback helper.
• VTable::supports_decompress_chunks / VTable::decompress_chunks with wiring through DynArrayData.
• Implementations:
  • BitPacked: streams each FastLanes block from its unpack scratch; patches applied per block via a cursor over a once-built, row-sorted patch list.
  • FoR: fused FoRStrategy streaming when the child is BitPacked with an unsigned reference (patch values pre-shifted); otherwise generic composition via an in-place add adapter over any streaming child.
  • Patched: streams its inner child and patches each chunk in place; the lane-transposed patch layout is flattened once into row-sorted pairs.
  • Constant: re-emits one stack-resident scratch chunk filled with the constant (refilled per emission since sinks may mutate chunks).
  • Primitive: already decompressed; streams the buffer through one reusable L1 scratch chunk without any execute.
• Tests cover patches, slicing/offset, fused and generic FoR composition, Patched-over-Constant, non-chunk-aligned lengths, null constants, empty arrays, and the error-without-emitting contract on unsupported trees.
• encodings/fastlanes/benches/chunked_decompress.rs (new divan bench) with hand-written monomorphized baselines to isolate each overhead.

Benchmarks

4Mi elements, divan medians (cargo bench -p vortex-fastlanes --bench chunked_decompress; cloud machine, so medians of a noisy box — orderings are stable across runs):

FoR(ref=1M) over BitPacked(bw=10), u32, streaming sum

bench	median	notes
hand_fused_sum	1.014 ms	monomorphized upper bound (reference folded algebraically)
chunked_vtable_sum	1.291 ms	vtable path, fused FoR+BitPacked streaming — within noise of hand-fused
hand_chunked_add_pass_sum	1.411 ms	monomorphized, with the extra in-place add pass
chunked_vtable_sum_generic_compose	1.886 ms	generic composition (signed ref, i32): child streams, adapter adds in place
two_pass_sum	3.360 ms	fused decompress to PrimitiveArray, then re-read

Materialization, same array

bench	median
fused_decompress (execute)	2.095 ms
chunked_vtable_decompress_into	3.466 ms

Patched(Constant), u32, 1 patch per 1000 rows

bench	median	notes
chunked_vtable_sum_patched_constant	0.955 ms	streaming sum — the 16MB base is never written
sparse_decompress_patched_constant	0.981 ms	execute: splat full buffer + scatter patches
chunked_decompress_into_patched_constant	1.460 ms	chunked materialization (scratch + copy out)
two_pass_sum_patched_constant	1.953 ms	sparse decompress, then re-read

Takeaways: streaming consumption beats decompress-then-iterate ~1.7–2.6×; the fused streaming path matches hand-fused unpack_map-style kernels; for pure materialization the specialized execute paths remain faster (single write into the destination), which is why they are not replaced — a possible follow-up is letting the sink offer a destination slice per chunk so leaf producers can decode straight into the final buffer.

Checks run: cargo nextest run -p vortex-array (3439 passed), cargo nextest run -p vortex-fastlanes (318 passed), cargo clippy --all-targets on both crates (clean), cargo +nightly fmt --all. Not run: workspace-wide --all-features clippy (CUDA feature stack untouched by this change).

API Changes

New public API in vortex-array: the chunk_iter module (ChunkSink, ChunkMut, DECOMPRESS_CHUNK_LEN), ArrayRef::{supports_decompress_chunks, decompress_chunks, decompress_chunks_or_materialize}, and two defaulted VTable methods (supports_decompress_chunks, decompress_chunks). No existing APIs change behavior.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LNgQat1UYMr3pjJrdhPJuh

---

Generated by Claude Code

[codspeed-hq]

Merging this PR will regress 1 benchmark

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 4 improved benchmarks
❌ 1 regressed benchmark
✅ 2092 untouched benchmarks
🆕 11 new benchmarks
⏩ 106 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Simulation	compress_fsst[(500, 64, 8)]	476.8 µs	539.5 µs	-11.62%
⚡	WallTime	mul_u64_nonnull_neon	20.5 µs	15.1 µs	+35.12%
⚡	WallTime	mul_i64_nonnull_neon	20 µs	17.1 µs	+17.27%
⚡	WallTime	multiply_shapes_neon[(16384, PerRowPerRow)]	20.3 µs	17.3 µs	+17.11%
⚡	Simulation	cold_misaligned[(16, 64)]	389.9 µs	344.2 µs	+13.28%
🆕	Simulation	chunked_decompress_into_patched_constant	N/A	18.5 ms	N/A
🆕	Simulation	chunked_vtable_decompress_into	N/A	22.9 ms	N/A
🆕	Simulation	chunked_vtable_sum	N/A	8.8 ms	N/A
🆕	Simulation	chunked_vtable_sum_generic_compose	N/A	10.9 ms	N/A
🆕	Simulation	chunked_vtable_sum_patched_constant	N/A	4.3 ms	N/A
🆕	Simulation	fused_decompress	N/A	20.7 ms	N/A
🆕	Simulation	hand_chunked_add_pass_sum	N/A	10.4 ms	N/A
🆕	Simulation	hand_fused_sum	N/A	8.2 ms	N/A
🆕	Simulation	sparse_decompress_patched_constant	N/A	16 ms	N/A
🆕	Simulation	two_pass_sum	N/A	37 ms	N/A
🆕	Simulation	two_pass_sum_patched_constant	N/A	32.2 ms	N/A

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing claude/decompression-chunked-iteration-u3kcns (f62481a) with develop (253137f)}

Footnotes

1. 106 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩