Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Fix GPU merge regression check action #1056

Merged
merged 2 commits into from
Jan 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 12 additions & 11 deletions .github/workflows/bench-deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,18 +13,19 @@ jobs:
runs-on: [self-hosted, gpu-bench, gh-pages]
steps:
# Install deps
- uses: actions/checkout@v4
with:
repository: lurk-lab/ci-workflows
- uses: ./.github/actions/gpu-setup
with:
gpu-framework: 'cuda'
- uses: ./.github/actions/ci-env
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- uses: taiki-e/install-action@v2
with:
tool: just@1.15.0
# Set up GPU
# Check we have access to the machine's Nvidia drivers
- run: nvidia-smi
# Check that CUDA is installed with a driver-compatible version
# This must also be compatible with the GPU architecture, see above link
- run: nvcc --version
tool: just@1.22.0
# Run benchmarks and deploy
- name: Get old benchmarks
uses: actions/checkout@v4
Expand All @@ -36,15 +37,15 @@ jobs:
run: cargo install cargo-criterion
- name: Run benchmarks
run: |
just --dotenv-filename bench.env gpu-bench fibonacci
cp ${{ github.sha }}.json ..
just gpu-bench-ci fibonacci
cp fibonacci-${{ github.sha }}.json ..
working-directory: ${{ github.workspace }}/benches
# TODO: Prettify labels for easier viewing
# Compress the benchmark file and metadata for later analysis
- name: Compress artifacts
run: |
echo $LABELS > labels.md
tar -cvzf ${{ github.sha }}.tar.gz Cargo.lock ${{ github.sha }}.json labels.md
tar -cvzf fibonacci-${{ github.sha }}.tar.gz Cargo.lock fibonacci-${{ github.sha }}.json labels.md
working-directory: ${{ github.workspace }}
- name: Deploy latest benchmark report
uses: peaceiris/actions-gh-pages@v3
Expand All @@ -53,7 +54,7 @@ jobs:
publish_dir: ./target/criterion
destination_dir: benchmarks/criterion
- name: Copy benchmark json to history
run: mkdir history; cp ${{ github.sha }}.tar.gz history/
run: mkdir history; cp fibonacci-${{ github.sha }}.tar.gz history/
- name: Deploy benchmark history
uses: peaceiris/actions-gh-pages@v3
with:
Expand Down
14 changes: 9 additions & 5 deletions .github/workflows/gpu-bench-merge-regression.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,10 @@ jobs:
GPU_NAME=$(nvidia-smi --query-gpu=gpu_name --format=csv,noheader,nounits | tail -n1)
echo "GPU_ID=$(echo $GPU_NAME | awk '{ print $NF }')" | tee -a $GITHUB_ENV
echo "GPU_NAME=$GPU_NAME" | tee -a $GITHUB_ENV
# Load Lurk bench env vars used in `justfile`, namely `LURK_BENCH_NOISE_THRESHOLD`
- uses: cardinalby/export-env-action@v2
with:
envFile: 'benches/bench.env'
# Checkout gh-pages to check for cached bench result
- name: Checkout gh-pages
uses: actions/checkout@v4
Expand Down Expand Up @@ -122,6 +126,7 @@ jobs:
else
REGRESSION_FACTOR=1.1
fi
echo "NOISE_THRESHOLD=$(echo "(REGRESSION_FACTOR-1)*100" | bc) | tee -a $GITHUB_ENV

for r in $REGRESSIONS
do
Expand All @@ -130,14 +135,13 @@ jobs:
exit 1
fi
done

echo "NOISE_THRESHOLD=$("(REGRESSION_FACTOR-1)*100" | bc) | tee -a $GITHUB_ENV
continue-on-error: true
# Not possible to use ${{ github.event.number }} with the `merge_group` trigger
- name: Get PR number from merge branch
run: |
echo "PR_NUMBER=$(echo ${{ github.event.merge_group.head_ref }} | sed -e 's/.*pr-\(.*\)-.*/\1/')" | tee -a $GITHUB_ENV
- uses: JasonEtco/create-an-issue@v2
if: steps.regression-check.outcome == 'failure'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ env.PR_NUMBER }}
Expand All @@ -148,13 +152,13 @@ jobs:
filename: .github/PERF_REGRESSION.md
- name: Remove old main bench
run: |
rm ${{ env.BASE_COMMIT }}.json
mv ${{ github.sha }}.json ${{ github.sha }}-${{ env.GPU_ID }}.json
rm fibonacci-${{ env.BASE_COMMIT }}.json
mv fibonacci-${{ github.sha }}.json fibonacci-${{ github.sha }}-${{ env.GPU_ID }}.json
working-directory: ${{ github.workspace }}
- name: Commit bench result to `gh-pages` branch if no regression
if: steps.regression-check.outcome != 'failure'
uses: stefanzweifel/git-auto-commit-action@v5
with:
branch: gh-pages
commit_message: '[automated] GPU Benchmark from PR #${{ env.PR_NUMBER }}'
file_pattern: '${{ github.sha }}-${{ env.GPU_ID }}.json'
file_pattern: 'fibonacci-${{ github.sha }}-${{ env.GPU_ID }}.json'
Loading