feat: add Arrow IPC and refine benchmark summaries by lwwmanning · Pull Request #24 · vortex-data/benchmarks-website

lwwmanning · 2026-08-25T19:59:53Z

Add arrow-ipc to Compression Size and Random Access. Use decoded in-memory Arrow bytes for logical write and scan throughput, plus compression ratios.

Compression Size shows Arrow-relative compression and Parquet-zstd-relative minimum, geometric mean, and maximum size ratios. If any Arrow ratio is unavailable, the summary ranks every format by its Parquet-zstd geometric mean.

Random Access now sums chart medians by dataset before ranking. It reports the geometric mean of dataset ratios and the arithmetic mean of dataset timings. Each format uses its latest available run, while incomplete coverage receives the missing-series penalty.

Compression ratios use the latest logical Arrow size per dataset, independent of each format's file commit. The nullable schema field preserves historical rows. JSON field access keeps web deployment compatible before migration 009 runs.

The summary cache key advances for the new payload.

Signed-off-by: Will Manning <will@willmanning.io>

Signed-off-by: Will Manning <will@willmanning.io> # Conflicts: # web/lib/groups.test.ts # web/lib/summary.ts

github-actions · 2026-08-25T20:01:16Z

Vercel preview for 7203f5ba19d4238d2e4cc17a7f60c0067529b4d2: https://benchmarks-website-fgtj6q6db-vortex-data.vercel.app

Signed-off-by: Will Manning <will@willmanning.io>

Signed-off-by: Will Manning <will@willmanning.io> # Conflicts: # migrations/README.md

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ed30a6997c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Signed-off-by: Will Manning <will@willmanning.io>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3ad0796f7c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Signed-off-by: Will Manning <will@willmanning.io>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8fdd14a795

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Signed-off-by: Will Manning <will@willmanning.io>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 08c2924ebe

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-08-26T20:32:31Z

+      if (group.charts.size === chartsByDataset.get(dataset)?.size) {
+        grouped.push({ series, bucket: dataset, value: group.value });


Preserve datasets with no complete format

When a dataset's charts are split across formats so that no single format covers every chart, this condition emits no sample for that dataset. Since rankSeries derives its bucket universe only from emitted samples, the dataset disappears from total, receives no missing-dataset penalty, and—if it is the only dataset—the entire summary returns null despite having measurements. The new knownSeries fallback preserves format identities but not this dataset universe; pass the known datasets into the ranking model so incomplete datasets still affect coverage and scoring.

AGENTS.md reference: AGENTS.md:L64-L66

Useful? React with 👍 / 👎.

connortsui20

seems fine to me, we can test in prod

lwwmanning added 3 commits August 25, 2026 15:05

feat(web): show Arrow-based compression metrics

a74cd27

Signed-off-by: Will Manning <will@willmanning.io>

fix(web): distinguish Arrow IPC from logical bytes

906ff82

Signed-off-by: Will Manning <will@willmanning.io>

Merge remote-tracking branch 'origin/develop' into wm/arrow-benchmarks

d369edc

Signed-off-by: Will Manning <will@willmanning.io> # Conflicts: # web/lib/groups.test.ts # web/lib/summary.ts

github-actions Bot deployed to preview August 25, 2026 20:00 View deployment

lwwmanning added 2 commits August 26, 2026 11:08

fix(web): use latest Arrow IPC sizes

9e852e2

Signed-off-by: Will Manning <will@willmanning.io>

Merge remote-tracking branch 'origin/develop' into wm/arrow-benchmarks

ed30a69

Signed-off-by: Will Manning <will@willmanning.io> # Conflicts: # migrations/README.md

github-actions Bot deployed to preview August 26, 2026 17:55 View deployment

lwwmanning marked this pull request as ready for review August 26, 2026 17:57

chatgpt-codex-connector Bot reviewed Aug 26, 2026

View reviewed changes

Comment thread web/lib/summary.ts Outdated

feat(web): refine benchmark summary metrics

b8b9d13

Signed-off-by: Will Manning <will@willmanning.io>

github-actions Bot deployed to preview August 26, 2026 19:45 View deployment

fix(web): tolerate pending compression migration

3ad0796

Signed-off-by: Will Manning <will@willmanning.io>

github-actions Bot deployed to preview August 26, 2026 19:50 View deployment

chatgpt-codex-connector Bot reviewed Aug 26, 2026

View reviewed changes

Comment thread web/lib/summary.ts Outdated

Comment thread web/components/SummaryCard.tsx Outdated

Comment thread web/lib/summary.ts Outdated

fix(web): keep compression size naming

a285ff5

Signed-off-by: Will Manning <will@willmanning.io>

github-actions Bot deployed to preview August 26, 2026 19:57 View deployment

fix(web): clarify compression size baseline

0d040f1

Signed-off-by: Will Manning <will@willmanning.io>

github-actions Bot deployed to preview August 26, 2026 19:59 View deployment

fix(web): address compression summary review

8fdd14a

Signed-off-by: Will Manning <will@willmanning.io>

github-actions Bot deployed to preview August 26, 2026 20:02 View deployment

chatgpt-codex-connector Bot reviewed Aug 26, 2026

View reviewed changes

Comment thread web/lib/summary.ts

fix(web): fall back to Parquet size ranking

fcf4c66

Signed-off-by: Will Manning <will@willmanning.io>

github-actions Bot deployed to preview August 26, 2026 20:07 View deployment

fix(web): preserve incomplete random access formats

4ab2c2f

Signed-off-by: Will Manning <will@willmanning.io>

github-actions Bot deployed to preview August 26, 2026 20:11 View deployment

lwwmanning changed the title ~~feat: add Arrow IPC benchmark summaries~~ feat: add Arrow IPC and refine benchmark summaries Aug 26, 2026

fix(web): invalidate stale summary payloads

1f5b1ef

Signed-off-by: Will Manning <will@willmanning.io>

github-actions Bot deployed to preview August 26, 2026 20:21 View deployment

fix(web): avoid sparse two-column summaries

08c2924

Signed-off-by: Will Manning <will@willmanning.io>

lwwmanning enabled auto-merge (squash) August 26, 2026 20:26

github-actions Bot deployed to preview August 26, 2026 20:26 View deployment

chatgpt-codex-connector Bot reviewed Aug 26, 2026

View reviewed changes

connortsui20 mentioned this pull request Aug 26, 2026

empty datasets issue? #25

Open

connortsui20 approved these changes Aug 26, 2026

View reviewed changes

lwwmanning merged commit b0cba5b into develop Aug 26, 2026
4 checks passed

lwwmanning deleted the wm/arrow-benchmarks branch August 26, 2026 20:38

Navigation Menu

Uh oh!

feat: add Arrow IPC and refine benchmark summaries - #24

feat: add Arrow IPC and refine benchmark summaries#24
lwwmanning merged 14 commits into
developfrom
wm/arrow-benchmarks

lwwmanning commented Aug 25, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 25, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Aug 26, 2026

Uh oh!

connortsui20 left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		if (group.charts.size === chartsByDataset.get(dataset)?.size) {
		grouped.push({ series, bucket: dataset, value: group.value });

Uh oh!

Conversation

lwwmanning commented Aug 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Aug 26, 2026

Choose a reason for hiding this comment

Uh oh!

connortsui20 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

lwwmanning commented Aug 25, 2026 •

edited

Loading

github-actions Bot commented Aug 25, 2026 •

edited

Loading