Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python3-Cython: update to 3.0.1. #45086

Closed
wants to merge 31 commits into from

Conversation

ahesford
Copy link
Member

@ahesford ahesford commented Jul 17, 2023

This is a big update that reflects several years of extensive work, and needs some testing to be sure it doesn't break any Void packages.

Testing the changes

  • I tested the changes in this PR: in progress

The following packages need to be fixed to build with new Cython:

  • gbinder-python
  • jack_mixer
  • libimobiledevice
  • memray
  • pyliblo
  • python3-cypari2
  • python3-cysignals
  • python3-efl
  • python3-fpylll
  • python3-grpcio
  • python3-h5py
  • python3-jq
  • python3-logbook
  • python3-lxml
  • python3-mpi4py
  • python3-pandas-msgpack
  • python3-peewee
  • python3-pplpy
  • python3-pyFFTW
  • python3-pygame
  • python3-scikit-learn
  • python3-scipy
  • python3-shapely
  • python3-tables
  • python3-uvloop
  • python3-yaml
  • sagemath
  • urh
  • vapoursynth
  • vidcutter

@ahesford ahesford marked this pull request as draft July 17, 2023 17:20
@tornaria
Copy link
Contributor

Related: sagemath/sage#29863

I'll have a look into (at least) my pkgs: cypari2, cysignals, fpylll, pplpy, sagemath. I can also have a look into scipy as it's needed for sagemath.

Question: could we have cython2 and cython3 coexist for a while?

@ahesford
Copy link
Member Author

The two versions of Cython can't coexist without a lot of pain because they will conflict in the Python package namespace. We can just hold this update until everything is resolved.

@icp1994
Copy link
Contributor

icp1994 commented Jul 18, 2023

pyyaml realeased v6.0.1 which blocks building with Cython 3.0.0

@ahesford
Copy link
Member Author

Yeah, I'll bump this and patch around it if necessary.

@ahesford ahesford force-pushed the python3-Cython-3.0.0 branch 2 times, most recently from 75dec0e to e9e9480 Compare July 19, 2023 02:35
@tornaria
Copy link
Contributor

scipy needs patching to build with pybind 2.11 (or maybe --skip-dependency-check since all the patches are about relaxing dependencies (cython, numpy, and now pybind11).

Note that in their main branch they have relaxed the upper bound for cython and pybind11 (they still fix numpy to a particular version). See: https://github.com/scipy/scipy/blob/main/pyproject.toml

+ compiler_directives={
+ "binding": True,
+ "embedsignature": True,
+ "language_level": 2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
+ "language_level": 2
+ "language_level": 2,

My bad, that's what I get for hand-editing the patch after checking... This should fix it.

@ahesford ahesford force-pushed the python3-Cython-3.0.0 branch 2 times, most recently from f2643df to 3d47f7d Compare August 26, 2023 03:24
@ahesford ahesford changed the title python3-Cython: update to 3.0.0. python3-Cython: update to 3.0.1. Aug 26, 2023
@ahesford
Copy link
Member Author

@tornaria I decided it wasn't worth trying to patch around all of these packages and added a new python3-Cython0.29 which conflicts with python3-Cython and offers the "legacy" Cython 0.29.36. The legacy package can be a build-time dependency for the ~30 packages that have yet to support Cython 3. Only two packages have run-time dependencies on Cython: urh and sagemath. I have moved both the build-time and run-time dependencies of urh to python3-Cython0.29 for now. How should I handle sagemath? We can do the same if you think that is the best course of action.

@ahesford ahesford force-pushed the python3-Cython-3.0.0 branch 2 times, most recently from 3d45062 to 7fe7229 Compare August 26, 2023 20:43
@tornaria
Copy link
Contributor

@tornaria I decided it wasn't worth trying to patch around all of these packages and added a new python3-Cython0.29 which conflicts with python3-Cython and offers the "legacy" Cython 0.29.36. The legacy package can be a build-time dependency for the ~30 packages that have yet to support Cython 3. Only two packages have run-time dependencies on Cython: urh and sagemath. I have moved both the build-time and run-time dependencies of urh to python3-Cython0.29 for now. How should I handle sagemath? We can do the same if you think that is the best course of action.

I think we have a way forward with sagemath (sagemath/sage#36109 is almost done with review and sagemath/sage#36110 is small). I think it might be possible to have sagemath running on cython 3 within the week.

@ahesford ahesford marked this pull request as ready for review August 29, 2023 10:50
@ahesford ahesford force-pushed the python3-Cython-3.0.0 branch 2 times, most recently from 876a445 to bb84127 Compare August 31, 2023 01:15
@ahesford ahesford closed this Sep 1, 2023
@ahesford ahesford deleted the python3-Cython-3.0.0 branch September 1, 2023 17:32
@ahesford
Copy link
Member Author

ahesford commented Sep 1, 2023

With python3-pplpy built with python3-Cython0.29 (which is its current state in the repository), all sagemath tests in the --long suite pass (with the exception of some new warnings emitted by Pythran), so this is good to go; merged as cb4c843.

@tornaria
Copy link
Contributor

tornaria commented Sep 2, 2023

With python3-pplpy built with python3-Cython0.29 (which is its current state in the repository), all sagemath tests in the --long suite pass (with the exception of some new warnings emitted by Pythran), so this is good to go; merged as cb4c843.

sagemath/sage#36109 was merged a few hours after you merged this, and that is the only "long" part of the move to cython 3.

The actual patch needed on top of that to be able to use cython 3 is just 6 lines (plus 19 lines in doctests due to some changed strings). See #45887 for details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants