Separate RowFn decode and semantic infallibility - #9644
Conversation
a7f19ac to
085fe76
Compare
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
085fe76 to
d49a7be
Compare
Merging this PR will regress 1 benchmark
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | WallTime | arrow_checked_add_u32_neon[16384] |
13.4 µs | 20.3 µs | -33.9% |
| ⚡ | WallTime | arrow_checked_add_u32_avx512[16384] |
21.4 µs | 17.7 µs | +21.14% |
| ⚡ | WallTime | arrow_checked_add_u32_avx2[16384] |
21.4 µs | 17.7 µs | +20.72% |
| ⚡ | WallTime | multiply_shapes_neon[(128, PerRowPerRow)] |
2.2 µs | 1.9 µs | +13.48% |
| ⚡ | WallTime | add_shapes_neon[(128, PerRowPerRow)] |
2.1 µs | 1.8 µs | +11.75% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing ct/row-fn-infallible (d49a7be) with develop (42a5d3b)
Footnotes
-
106 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
-
4 benchmarks were run, but are now archived. If they were deleted in another branch, consider rebasing to remove them from the report. Instead if they were added back, click here to restore them. ↩
gatesn
left a comment
There was a problem hiding this comment.
This is the wrong decomp I think.
You're trying to distinguish functions that can return errors, that may be caught. This is important when e.g. a push-down optimization would hide a failure that otherwise would semantically cause the query to fail.
Vs errors that you don't expect to happen / the user has asserted to ignore. For example, if the user opts in to not validating the WKB, then I think it is perfectly reasonably for us to push this through dictionary encoding and not error.
I wonder if this is related to our work on passing down a "defined" flag to allow ignoring errors in null positions etc.
|
Ok we dont actually need this now that we have the filter/scatter execute from #9521, but I still think there is something wrong here (not specific to RowFn, but to ScalarFnVTable in general) |
Summary
Tracking Issues: #9129, #9130
Separates semantic row totality from input decode totality.
ScalarFnVTable::is_infalliblerequires both guarantees because it permits evaluation of unreferenced values, while decode fallibility no longer selects the nullable-row policy.Arguably, we should separate these concepts directly on the
ScalarFnVTable, but that can come later.I'll just copy paste this from my conversation with @joseph-isaacs:
Details
so our
is_infallibleon the scalar fn vtable is overloaded. What we really want to say is "can I evaluate something speculatively" which is a harder constraint than just semantic totality:Changes
Adds the required
RowFn::DECODE_INFALLIBLEdeclaration and checks positive declarations against each input tuple selected bydispatch. Marks spatial distance as decode-fallible and adds regression coverage for early decode errors and dense execution of partially valid inputs.