Skip to content

Commit

Permalink
ci: Fix bugs in #790 (#822)
Browse files Browse the repository at this point in the history
* Fix bugs in #790

* Address feedback & cleanup
  • Loading branch information
samuelburnham authored Oct 31, 2023
1 parent ab4ff04 commit 174c3a5
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 9 deletions.
1 change: 1 addition & 0 deletions .github/workflows/bench-deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ jobs:
run: cargo install cargo-criterion
- name: Run benchmarks
run: just --dotenv-filename bench.env gpu-bench fibonacci_lem
working-directory: ${{ github.workspace }}/benches
# TODO: Prettify labels for easier viewing
# Compress the benchmark file and metadata for later analysis
- name: Compress artifacts
Expand Down
23 changes: 14 additions & 9 deletions .github/workflows/merge-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -96,16 +96,22 @@ jobs:
path: master
# Copy the script so the base can bench with the same parameters
- name: Copy source script to base branch
run: cd benches && cp justfile bench.env ../master/benches
run: cp justfile bench.env ../master/benches
working-directory: ${{ github.workspace }}/benches
- name: Set base ref variable
run: cd master && echo "BASE_REF=$(git rev-parse HEAD)" >> $GITHUB_ENV
- run: echo ${{ env.BASE_REF }}
run: echo "BASE_REF=$(git rev-parse HEAD)" | tee -a $GITHUB_ENV
working-directory: ${{ github.workspace }}/master
- name: Run GPU bench on base branch
run: cd master/benches && just --dotenv-filename bench.env gpu-bench fibonacci_lem
run: just --dotenv-filename bench.env gpu-bench fibonacci_lem
working-directory: ${{ github.workspace }}/master/benches
- name: Copy bench output to PR branch
run: cp master/${{ env.BASE_REF }}.json .
run: |
mkdir -p target
cp -r master/target/criterion target
cp master/${{ env.BASE_REF }}.json .
- name: Run GPU bench on PR branch
run: cd benches && just --dotenv-filename bench.env gpu-bench fibonacci_lem
run: just --dotenv-filename bench.env gpu-bench fibonacci_lem
working-directory: ${{ github.workspace }}/benches
# Create a `criterion-table` and write in commit comment
- name: Run `criterion-table`
run: cat ${{ github.sha }}.json | criterion-table > BENCHMARKS.md
Expand All @@ -117,11 +123,10 @@ jobs:
# Check for benchmark regression based on Criterion's configured noise threshold
- name: Performance regression check
id: check-regression
run: |
echo "regress_count=$(grep -c 'Regressed' ${{ github.sha }}.json)" >> $GITHUB_OUTPUT
run: echo "regress_count=$(grep -c 'Regressed' ${{ github.sha }}.json)" | tee -a $GITHUB_OUTPUT
# Fail job if regression found
- uses: actions/github-script@v6
if: ${{ steps.check-regression.outputs.regress_count }} > 0
if: ${{ steps.check-regression.outputs.regress_count > 0 }}
with:
script: |
core.setFailed('Fibonacci bench regression detected')
Expand Down

1 comment on commit 174c3a5

@github-actions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmarks

Table of Contents

Benchmark Results

LEM Prove

Fibonacci-rc=100 Fibonacci-rc=600
num-100 6.97 s (✅ 1.00x) 7.48 s (✅ 1.07x slower)
num-200 14.39 s (✅ 1.00x) 16.68 s (❌ 1.16x slower)

Made with criterion-table

Please sign in to comment.