Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New throughput performance example #908

Merged
merged 1 commit into from
Dec 21, 2023
Merged

New throughput performance example #908

merged 1 commit into from
Dec 21, 2023

Conversation

arthurpaulino
Copy link
Member

@arthurpaulino arthurpaulino commented Nov 21, 2023

Add example to measure throughput in it/s (raw and adjusted) across different rcs and number of folding steps

This is a PoC that we might want to later integrate with criterion somehow.

On a short run with LURK_RC=8,16 LURK_N_FOLDS=2 cargo run --release --example tp_table, it prints:

Raw throughput
┌────────────┬─────────────┬────────────┬────────────┐
│ rc\n_folds │ 0           │ 1          │ 2          │
├────────────┼─────────────┼────────────┼────────────┤
│          8 │  85.61±3.15 │ 64.45±0.92 │ 56.22±0.68 │
│         16 │ 111.48±2.30 │ 77.47±0.70 │ 67.54±0.61 │
└────────────┴─────────────┴────────────┴────────────┘
Adjusted throughput
┌────────────┬────────────┬────────────┐
│ rc\n_folds │ 1          │ 2          │
├────────────┼────────────┼────────────┤
│          8 │ 51.72±1.19 │ 48.00±0.75 │
│         16 │ 59.38±0.83 │ 56.42±0.63 │
└────────────┴────────────┴────────────┘

Note: these numbers are from my weak machine.

examples/tp_table.rs Outdated Show resolved Hide resolved
examples/tp_table.rs Outdated Show resolved Hide resolved
porcuquine
porcuquine previously approved these changes Dec 21, 2023
Copy link
Collaborator

@porcuquine porcuquine left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're already using this as an 'indicator' of something, I think it's fine to merge (though I would like to fix the 'source-code case' issue as a matter of good policy in examples) as-is. Before we start taking it too seriously, it would be useful to document the intended behavior and the implementation so:

  • users know exactly what this purports to measure
  • reviewers can verify that it actually does so

This review claims to have done neither of those but notes that as an n+1 'number go up' example, we're already getting some benefit out of running it from time to time.

@arthurpaulino arthurpaulino added this pull request to the merge queue Dec 21, 2023
Merged via the queue into main with commit 510d704 Dec 21, 2023
12 checks passed
@arthurpaulino arthurpaulino deleted the tp-table branch December 21, 2023 23:38
github-actions bot pushed a commit that referenced this pull request Dec 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants