Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Restart CUDA 12 migration with CUDA arch builds #4799

Merged
merged 3 commits into from
Aug 23, 2023
Merged

Restart CUDA 12 migration with CUDA arch builds #4799

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e1d7706
Restart CUDA 12 migration with CUDA arch builds
jakirkham Aug 17, 2023
78c9f05
Note CUDA 12 w/arch in PR title & commit message
jakirkham Aug 23, 2023
32df682
Drop `cudnn` from CUDA 12 migration
jakirkham Aug 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 30 additions & 17 deletions recipe/migrations/cuda120.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ __migrator:
kind:
version
migration_number:
1
2
Comment on lines 5 to +6
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to double check this doesn't drop the migrator in migrated feedstocks

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC this code path in conda-smithy continues to use the migrator already in a feedstock when a new migration_number occurs. IOW it will wait until the migrator bot add the new migration file (or a user adds it manually)

Removal doesn't occur as long as migrator_ts is the same, which is the case here

Copy link
Member Author

@jakirkham jakirkham Aug 23, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also IIUC the migrators used by conda-smithy can be overridden by passing --exclusive-config-file during rerender. So tried this with the ucx-split-feedstock using latest changes and found no change. So it looks like this is not an issue

Edit: Should add can see this migration file being used based on log lines that look like this pointing to the local copy of the migrator

INFO:conda_smithy.configure_feedstock:Applying migrations: /Users/jkirkham/Developer/conda-forge/feedstocks/feedstocks/ucx-split/.ci_support/migrations/cuda_112_ppc64le_aarch64.yaml,/Users/jkirkham/Developer/conda-forge/feedstocks/feedstocks/ucx-split/.ci_support/migrations/cuda120.yaml

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also have done a few re-renderings myself and with the bot after this was merged and everything seems to be working correctly

build_number:
1
paused: false
Expand Down Expand Up @@ -37,14 +37,21 @@ __migrator:
- quay.io/condaforge/linux-anvil-cuda:11.0 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cuda:11.1 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cuda:11.2 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
# case: native compilation (build == target)
- quay.io/condaforge/linux-anvil-ppc64le-cuda:11.2 # [ppc64le and os.environ.get("BUILD_PLATFORM") == "linux-ppc64le"]
- quay.io/condaforge/linux-anvil-aarch64-cuda:11.2 # [aarch64 and os.environ.get("BUILD_PLATFORM") == "linux-aarch64"]
# case: cross-compilation (build != target)
- quay.io/condaforge/linux-anvil-cuda:11.2 # [ppc64le and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cuda:11.2 # [aarch64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
# case: non-CUDA builds
- quay.io/condaforge/linux-anvil-cos7-x86_64 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
cuda_compiler_version:
- None
- 10.2 # [(linux64 or win) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 11.0 # [(linux64 or win) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 11.1 # [(linux64 or win) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 11.2 # [(linux64 or win) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 12.0 # [(linux64 or win) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 11.2 # [(linux or win) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 12.0 # [(linux or win) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
commit_message: |
Rebuild for CUDA 12
jakirkham marked this conversation as resolved.
Show resolved Hide resolved

Expand All @@ -55,26 +62,32 @@ __migrator:
[see this issue]( https://github.com/conda-forge/conda-forge.github.io/issues/1963 ).
Please feel free to raise any issues encountered there. Thank you! :pray:

cuda_compiler: # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- cuda-nvcc # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
cuda_compiler: # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- cuda-nvcc # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]

cuda_compiler_version: # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 12.0 # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
cuda_compiler_version: # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 12.0 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]

c_compiler_version: # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 12 # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
c_compiler_version: # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 12 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]

cxx_compiler_version: # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 12 # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
cxx_compiler_version: # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 12 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]

fortran_compiler_version: # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 12 # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
fortran_compiler_version: # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 12 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]

cudnn: # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 8 # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
cudnn: # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 8 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]

cdt_name: # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- cos7 # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
cdt_name: # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- cos7 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]

docker_image: # [os.environ.get("BUILD_PLATFORM", "").startswith("linux-") and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- quay.io/condaforge/linux-anvil-cos7-x86_64 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64" and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
# case: native compilation (build == target)
- quay.io/condaforge/linux-anvil-ppc64le # [ppc64le and os.environ.get("BUILD_PLATFORM") == "linux-ppc64le"]
- quay.io/condaforge/linux-anvil-aarch64 # [aarch64 and os.environ.get("BUILD_PLATFORM") == "linux-aarch64"]
# case: cross-compilation (build != target)
- quay.io/condaforge/linux-anvil-cos7-x86_64 # [ppc64le and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cos7-x86_64 # [aarch64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
49 changes: 0 additions & 49 deletions recipe/migrations/cuda120_ppc64le_aarch64.yaml
View file
Open in desktop

This file was deleted.