Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add announcement for upcoming stdlib migration #2126

Merged
merged 7 commits into from
Mar 24, 2024
Merged
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions news/2024-03-20-stdlib-migration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Upcoming migration for stdlib("c")

Almost since the inception of conda-forge, the baseline version of our standard
library ("stdlib") for C has not changed. This library comes with extra
complications because it is an essential part of the operating system, and one
of the few things that conda/mamba/etc. cannot ship safely.

As the ecosystem has moved on and many packages are starting to require newer
baseline versions, we need to follow suit at some point. However, to avoid
breaking users on older systems, we need to have infrastructure in place that
allows our packages to have sufficiently accurate metadata, such that conda can
avoid the installation of a package requiring a newer stdlib on an old system.

After many discussions across conda-forge stakeholders, the solution we arrived
at is the introduction of a new Jinja2-function `{{ stdlib("c") }}`, which
reflects that a given recipe requires a C stdlib. Making this relationship
explicit will make it easy to correctly reflect the requirement for newer
stdlib versions per feedstock, as well as in our global pinning.

Up until now, the stdlib was handled implicitly as part of the compiler stack.
In order to allow this transition to happen, we need to introduce this function
to essentially all compiled (i.e. not pure-Python) packages. This will be done
h-vetinari marked this conversation as resolved.
Show resolved Hide resolved
in stages, first for a single migration, and then attached to _all_ ongoing
migrations in conda-forge.

The logic of the piggyback migrator tries to correctly handle most scenarios,
but it is impossible to cover all corner cases. As for some general rules that
all feedstock maintainers are free to apply independently:

- if a feedstock uses a `- {{ compiler(...) }}` jinja in the build section,
add a line with `- {{ stdlib("c") }}` to the build environment.
- if a feedstock uses `- sysroot_linux-64 2.17 # [linux64]` (or a variation),
remove this line and add the following to your `conda_build_config.yaml`:
```
c_stdlib_version: # [linux]
- 2.17 # [linux]
```
- if a feedstock sets `MACOSX_DEPLOYMENT_TARGET` in `conda_build_config.yaml`,
for example to 10.13 for `x86_64`, replace that section with the following
(note, this does _not_ apply to `MACOSX_SDK_VERSION`!):
```
c_stdlib_version: # [osx and x86_64]
- 10.13 # [osx and x86_64]
```
You should then also remove any line involving `__osx` from `meta.yaml`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we put this info in our knowledgebase entries on using compilers? It might already be there, IDK.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that this would be a welcome addition to the knowledge base or compilers. We could link to the entry from the announcement so it doesn't get as long. Should help with discoverability!

Copy link
Member Author

@h-vetinari h-vetinari Mar 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we put this info in our knowledgebase entries on using compilers?

Yeah, but that would realistically need to build on #1950, which has been waiting for feedback/resolution. Perhaps I need to bring up this topic in the core call.

We could link to the entry from the announcement so it doesn't get as long. Should help with discoverability!

Yeah, that sounds like a good idea.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That PR doesn't have the new stdlib setup stuff in it for the knowledgebase that I can see.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that PR's about a year old. My point is that if we want to add stdlib handling to our compiler docs, I want to build on the existing clean-up rather than do two conflicting updates.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could link to the entry from the announcement so it doesn't get as long. Should help with discoverability!

Yeah, that sounds like a good idea.

I actually reconsidered this and don't necessarily agree anymore. The docs definitely need to be updated, but they should describe the new status quo, not a transition (or at least not for long, as it will become stale quickly).

The announcement (or tracking issue) seems like a reasonable place for me to describe the main mechanics of the transition, especially to the degree that it allows maintainers to do it themselves, without having to wait for the piggyback migrator.

Copy link
Member

@jaimergp jaimergp Mar 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My main concern was about the discoverability of that information, but the search features should reveal both, so no strong feelings about it. I trust your judgment :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I only added a promise to update the docs for now, for two reasons:

Hope that's alright.

h-vetinari marked this conversation as resolved.
Show resolved Hide resolved

For more details, see this [issue](https://github.com/conda-forge/conda-forge.github.io/issues/2102).