Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build wheels alongside conda CI #12427

Merged
merged 30 commits into from
Jan 10, 2023
Merged

Build wheels alongside conda CI #12427

merged 30 commits into from
Jan 10, 2023

Conversation

sevagh
Copy link
Contributor

@sevagh sevagh commented Dec 20, 2022

Description

This PR adds pip wheel CI to the Conda CI, instead of having them work separately.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@sevagh sevagh requested a review from a team as a code owner December 20, 2022 17:52
@sevagh sevagh added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Dec 20, 2022
.github/workflows/pr.yaml Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Dec 20, 2022

Codecov Report

Base: 86.58% // Head: 85.67% // Decreases project coverage by -0.90% ⚠️

Coverage data is based on head (6492d1a) compared to base (b6dccb3).
Patch has no changes to coverable lines.

Additional details and impacted files
@@               Coverage Diff                @@
##           branch-23.02   #12427      +/-   ##
================================================
- Coverage         86.58%   85.67%   -0.91%     
================================================
  Files               155      155              
  Lines             24368    24803     +435     
================================================
+ Hits              21098    21251     +153     
- Misses             3270     3552     +282     
Impacted Files Coverage Δ
python/cudf/cudf/_version.py 1.41% <0.00%> (-98.59%) ⬇️
python/cudf/cudf/core/buffer/spill_manager.py 72.50% <0.00%> (-7.50%) ⬇️
python/cudf/cudf/core/buffer/spillable_buffer.py 90.04% <0.00%> (-2.81%) ⬇️
python/cudf/cudf/utils/dtypes.py 77.77% <0.00%> (-1.69%) ⬇️
python/cudf/cudf/options.py 86.11% <0.00%> (-1.59%) ⬇️
python/cudf/cudf/core/single_column_frame.py 94.30% <0.00%> (-1.27%) ⬇️
...ython/custreamz/custreamz/tests/test_dataframes.py 98.38% <0.00%> (-1.01%) ⬇️
python/dask_cudf/dask_cudf/io/parquet.py 91.81% <0.00%> (-0.59%) ⬇️
python/cudf/cudf/core/multiindex.py 91.66% <0.00%> (-0.51%) ⬇️
python/cudf/cudf/core/algorithms.py 90.00% <0.00%> (-0.48%) ⬇️
... and 36 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

Copy link
Member

@ajschmidt8 ajschmidt8 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(double approval because I forgot the 🔥 in the first one)

🔥 🔥 🔥 🔥 🔥 🔥

@github-actions github-actions bot added the ci label Dec 21, 2022
Copy link
Contributor

@vyasr vyasr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of minor suggestions but looks awesome!

package-name: cudf
package-dir: python/cudf
versioneer-override: ${{ inputs.wheel_versioneer_override }}
skbuild-configure-options: "-DCUDF_BUILD_WHEELS=ON -DDETECT_CONDA_ENV=OFF"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can leave this for now (something I will raise more broadly in the new year) but AFAIK this variable doesn't do anything right now. The question would be whether it should be set to do anything. That's a rapids-wide discussion though; I think only cuML is actively making use of this now and that's partly because it needs to forward along to deps like treelite.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with:
build_type: pull-request
package-name: cudf
test-before-arm64: "pip install tokenizers==0.10.2 cupy-cuda11x -f https://pip.cupy.dev/aarch64"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it's already working we can leave it for now, but I'll note that in #12377 we actually removed tokenizers from arm tests and only test on x86 for conda packages.

ci/wheel_smoke_test_cudf.py Outdated Show resolved Hide resolved
@sevagh sevagh requested a review from a team as a code owner January 6, 2023 16:37
@sevagh sevagh requested review from bdice and isVoid January 6, 2023 16:37
@github-actions github-actions bot added the Python Affects Python cuDF API. label Jan 6, 2023
@vyasr vyasr requested a review from a team as a code owner January 6, 2023 20:52
@github-actions github-actions bot added CMake CMake build issue libcudf Affects libcudf (C++/CUDA) code. labels Jan 6, 2023
.github/workflows/build.yaml Outdated Show resolved Hide resolved
sevagh added a commit to rapidsai/shared-workflows that referenced this pull request Jan 9, 2023
Create new wheel build parameters: `before-wheel`

This allows us to use `rapids-download-wheels-from-s3` to have
inter-dependent wheels within a single workflow download the dependency
wheels _built in this current workflow run_ instead of pulling the
nightly published copy.

Tested for pure wheels in cuDF:
rapidsai/cudf#12427
Tested for manylinux wheels in RAFT:
rapidsai/raft#1116
@sevagh
Copy link
Contributor Author

sevagh commented Jan 10, 2023

/merge

@rapids-bot rapids-bot bot merged commit 66b846a into rapidsai:branch-23.02 Jan 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CMake CMake build issue improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change Python Affects Python cuDF API.
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

4 participants