Support SpatialBench in DataFusion benchmark - #9615
Conversation
Merging this PR will regress 1 benchmark
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | WallTime | arrow_checked_add_u32_avx2[16384] |
17.7 µs | 21.3 µs | -16.96% |
| ⚡ | WallTime | arrow_checked_add_u32_neon[16384] |
20.4 µs | 13.5 µs | +50.71% |
| ⚡ | WallTime | arrow_checked_add_u32_avx512[16384] |
21.3 µs | 17.6 µs | +20.72% |
| ⚡ | Simulation | take[duplicates/repeated/primitive/nonnull/chunks=16/indices=1000] |
245 µs | 207 µs | +18.39% |
| ⚡ | WallTime | mul_u32_nonnull_avx512 |
6.3 µs | 5.6 µs | +11.03% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing bp/datafusion-spatialbench (a55abdd) with develop (ee1ac25)
Footnotes
-
106 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
-
4 benchmarks were run, but are now archived. If they were deleted in another branch, consider rebasing to remove them from the report. Instead if they were added back, click here to restore them. ↩
ffe767d to
e1a9cef
Compare
palaska
added
changelog/feature
| if benchmark.dataset_name() == "spatialbench" { | ||
| geodatafusion::register(&session); | ||
| } |
There was a problem hiding this comment.
can we have a setup func
There was a problem hiding this comment.
moved this into datafusion_bench::setup_session
| fn queries(&self) -> anyhow::Result<Vec<(usize, String)>>; | ||
|
|
||
| /// Get all available queries for one engine. | ||
| fn queries_for_engine(&self, _engine: Engine) -> anyhow::Result<Vec<(usize, String)>> { |
There was a problem hiding this comment.
why would I filter like this?
|
|
||
| fn queries_for_engine(&self, engine: Engine) -> anyhow::Result<Vec<(usize, String)>> { | ||
| if engine != Engine::DataFusion { | ||
| return self.queries(); |
There was a problem hiding this comment.
so this works for duckdb?
| Ok(include_str!("../../sql/spatialbench-datafusion.sql") | ||
| .split_terminator(';') | ||
| .map(str::trim) | ||
| .filter(|statement| !statement.is_empty()) | ||
| .enumerate() | ||
| .map(|(index, statement)| (index + 1, statement.to_owned())) | ||
| .collect()) |
| geo-types = "0.7.19" | ||
| geoarrow = "0.8.0" | ||
| geoarrow-cast = "0.8.0" | ||
| geodatafusion = "0.5.0" |
There was a problem hiding this comment.
does this take ages to compile?
There was a problem hiding this comment.
it was fine on my laptop. it adds some geo kernels but reuses datafusion/arrow versions
|
How come we never actually run it |
e1a9cef to
4c1d7fa
Compare
added spatialbench to the matrix. kicked this manual job, lets see how it goes https://github.com/vortex-data/vortex/actions/runs/32867730381 |
9f90ba6 to
d1f40d1
Compare
Signed-off-by: Baris Palaska <barispalaska@gmail.com>
d1f40d1 to
a55abdd
Compare
2 participants
Adds automatic engine-specific SQL discovery for benchmark queries.
Adds a DataFusion SQL dialect for SpatialBench and registers GeoDataFusion functions. Adds an SF=0.6
pr-spatialpreset. Theaction/bench-spatiallabel runs the preset. SpatialBench does not run in default presets.The SQL corpus keeps all twelve queries. The performance preset excludes Q5, Q7, and Q9 because GeoDataFusion lacks their required functions.
Validation:
cargo test -p vortex-bench discovers_engine_query_corporacargo clippy -p vortex-bench --all-targets --all-features -- -D warningsPrepared with Codex.