Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC]: Add os_version option to use alma8 #6548

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 34 additions & 16 deletions recipe/conda_build_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,11 @@ c_stdlib:
m2w64_c_stdlib: # [win]
- m2w64-toolchain # [win]
c_stdlib_version: # [unix]
- 2.17 # [linux]
- 2.17 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 2.17 # [linux and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "cos7"]
- 2.28 # [linux and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "alma8"]
- 2.17 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 2.17 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "cos7"]
- 2.28 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "alma8"]
Comment on lines +18 to +22
Copy link
Member

@h-vetinari h-vetinari Oct 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure we don't want to touch the stdlib version here; for example, the newer image might just be required to satisfy the __glibc constraint of a build or test dependency, but doesn't necessarily mean that the package itself needs a newer glibc and run-constraint.

Despite the monster-zip it remains possible to override one component locally - e.g. c_stdlib_version, and then set os_version for the image. I'd also prefer the choice of c_stdlib_version to be more visible than something implicitly implied by a knob tucked away in conda-forge.yml.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure we don't want to touch the stdlib version here

OK, I've tested this hypothesis (already before commenting but) now also using a local checkout of #6626 - here's the resulting branch.

Basically, this works as intended if we also take the c_stdlib_version out of the CUDA zip, which I think is possible now. That would be the ideal scenario for me - the c_stdlib_version can be independently set (in CBC), from the image (in conda-forge.yml, or in CBC if the full zip is overridden).

- 10.13 # [osx and x86_64]
- 11.0 # [osx and arm64]
cxx_compiler:
Expand Down Expand Up @@ -125,17 +127,24 @@ channel_sources:
channel_targets:
- conda-forge main

cdt_name: # [linux]
- cos7 # [linux]
cdt_name: # [linux]
- cos7 # [linux and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "cos7"]
- conda # [linux and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "alma8"]
Comment on lines +130 to +132
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can just use

cdt_name:  # [linux]
  - conda  # [linux]

without any further lines/selectors (and remove it from the zips).

This is what we discussed today in the core-call; those CDTs will be available after conda-forge/cdt-builds#71 is merged.


- cos7 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- cos7 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- cos7 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- cos7 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "cos7"]
- conda # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "alma8"]

docker_image: # [os.environ.get("BUILD_PLATFORM", "").startswith("linux-")]
# Native builds
- quay.io/condaforge/linux-anvil-cos7-x86_64 # [os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-aarch64 # [os.environ.get("BUILD_PLATFORM") == "linux-aarch64"]
- quay.io/condaforge/linux-anvil-ppc64le # [os.environ.get("BUILD_PLATFORM") == "linux-ppc64le"]
# Native builds on CentOS 7
- quay.io/condaforge/linux-anvil-cos7-x86_64 # [os.environ.get("BUILD_PLATFORM") == "linux-64" and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "cos7"]
- quay.io/condaforge/linux-anvil-aarch64 # [os.environ.get("BUILD_PLATFORM") == "linux-aarch64" and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "cos7"]
- quay.io/condaforge/linux-anvil-ppc64le # [os.environ.get("BUILD_PLATFORM") == "linux-ppc64le" and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "cos7"]

# Native builds on AlmaLinux 8
- quay.io/condaforge/linux-anvil-alma-x86_64:8 # [os.environ.get("BUILD_PLATFORM") == "linux-64" and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "alma8"]
- quay.io/condaforge/linux-anvil-alma-aarch64:8 # [os.environ.get("BUILD_PLATFORM") == "linux-aarch64" and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "alma8"]
- quay.io/condaforge/linux-anvil-alma-ppc64le:8 # [os.environ.get("BUILD_PLATFORM") == "linux-ppc64le" and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "alma8"]

# CUDA 11.8
- quay.io/condaforge/linux-anvil-cuda:11.8 # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("BUILD_PLATFORM") == "linux-64"]
Expand All @@ -146,14 +155,23 @@ docker_image: # [os.environ.get("BUILD_PLATFOR
- quay.io/condaforge/linux-anvil-cuda:11.8 # [ppc64le and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cuda:11.8 # [aarch64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("BUILD_PLATFORM") == "linux-64"]

# CUDA 12
- quay.io/condaforge/linux-anvil-cos7-x86_64 # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("BUILD_PLATFORM") == "linux-64"]
# CUDA 12 on CentOS 7
- quay.io/condaforge/linux-anvil-cos7-x86_64 # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("BUILD_PLATFORM") == "linux-64" and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "cos7"]
# case: native compilation (build == target)
- quay.io/condaforge/linux-anvil-ppc64le # [ppc64le and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("BUILD_PLATFORM") == "linux-ppc64le" and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "cos7"]
- quay.io/condaforge/linux-anvil-aarch64 # [aarch64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("BUILD_PLATFORM") == "linux-aarch64" and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "cos7"]
h-vetinari marked this conversation as resolved.
Show resolved Hide resolved
# case: cross-compilation (build != target)
- quay.io/condaforge/linux-anvil-cos7-x86_64 # [ppc64le and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("BUILD_PLATFORM") == "linux-64" and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "cos7"]
- quay.io/condaforge/linux-anvil-cos7-x86_64 # [aarch64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("BUILD_PLATFORM") == "linux-64" and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "cos7"]

# CUDA 12 on AlmaLinux8
- quay.io/condaforge/linux-anvil-alma-x86_64:8 # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("BUILD_PLATFORM") == "linux-64" and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "alma8"]
# case: native compilation (build == target)
- quay.io/condaforge/linux-anvil-ppc64le # [ppc64le and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("BUILD_PLATFORM") == "linux-ppc64le"]
- quay.io/condaforge/linux-anvil-aarch64 # [aarch64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("BUILD_PLATFORM") == "linux-aarch64"]
- quay.io/condaforge/linux-anvil-alma-ppc64le:8 # [ppc64le and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("BUILD_PLATFORM") == "linux-ppc64le" and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "alma8"]
- quay.io/condaforge/linux-anvil-alma-aarch64:8 # [aarch64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("BUILD_PLATFORM") == "linux-aarch64" and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "alma8"]
# case: cross-compilation (build != target)
- quay.io/condaforge/linux-anvil-cos7-x86_64 # [ppc64le and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cos7-x86_64 # [aarch64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-alma-x86_64:8 # [ppc64le and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("BUILD_PLATFORM") == "linux-64" and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "alma8"]
- quay.io/condaforge/linux-anvil-alma-x86_64:8 # [aarch64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True" and os.environ.get("BUILD_PLATFORM") == "linux-64" and os.environ.get("DEFAULT_LINUX_VERSION", "cos7") == "alma8"]

zip_keys:
# For CUDA, c_stdlib_version/cdt_name is zipped below with the compilers.
Expand Down