Add support for Python 3.12, pin proj back to 9.3.x, require geopandas>=1.0 #1453
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jameslamb
added
non-breaking
github-actions
bot
added
conda
jameslamb
changed the title
3 tasks
|
Some changes to the required version of Working on it. |
|
😬 😬 😬 I think being behind on I haven't 100% figured this out yet, but posting my thoughts so far. Short SummaryHere's what I think is happening:
If I'm right, options I think we have:
Other NotesHow to reproduce (and then fiddle with) the exact errors from CIcommands to run (click me)On an x86_64 machine with CUDA docker run \
--rm \
--gpus 1 \
--env RAPIDS_BUILD_TYPE=pull-request \
--env RAPIDS_REPOSITORY=rapidsai/cuspatial \
--env RAPIDS_REF_NAME=pull-request/1453 \
-v $(pwd):/opt/work \
-w /opt/work \
-it rapidsai/ci-conda:cuda12.5.1-ubuntu22.04-py3.12 \
bash
CPP_CHANNEL=$(rapids-download-conda-from-s3 cpp)
PYTHON_CHANNEL=$(rapids-download-conda-from-s3 python)
# guarantee that exactly the CI artifacts are used
# and that 'mamba install' doesn't silently fall back to
# other packages from rapidsai-nightly / rapidsai
BUILD_SUFFIX="240906_g4361f9ed_42"
mamba create \
--name delete-me \
-c conda-forge \
-c "${CPP_CHANNEL}" \
-c "${PYTHON_CHANNEL}" \
--dry-run \
"cuproj==24.10.00a42=cuda12_py312_${BUILD_SUFFIX}" \
"cuspatial==24.10.00a42=cuda12_py312_${BUILD_SUFFIX}" \
"libcuspatial==24.10.00a42=cuda12_${BUILD_SUFFIX}" \
"geopandas>=0.11.0" \
"python=3.12" full solver output (click me)Other notes
|
2 tasks
|
Yup, I agree with your assessment. Here's a screenshot that also points to spdlog/fmt issues: This is from the command Our other choice is to pin We don't want to let this block all Python 3.12 progress, so 0e25119 is probably our best bet in the short term. Once we complete the Python 3.12 migration we can go back to fix librmm exports and spdlog/fmt upgrades. |
|
The |
|
Ah thank you so much! I hadn't been able to isolate With that change, you can now see conda able to fall back to older versions of those things: Now we just have one other issue... the version of
HOWEVER... first, I'm going to try just moving This helps becuase |
jameslamb
commented
Sep 9, 2024
| @@ -145,42 +145,6 @@ pip install cuspatial-cu12 --extra-index-url=https://pypi.nvidia.com | |||
| pip install cuspatial-cu11 --extra-index-url=https://pypi.nvidia.com | |||
| ``` | |||
|
|
|||
| #### Troubleshooting Fiona/GDAL versions | |||
There was a problem hiding this comment.
Starting a threaded conversation for this one.
Proposal
pin to geopandas >= 1.0.0 so we can drop this project's reliance on fiona entirely
commit: 9295c3f
Benefits
No more need to install specific versions of fiona based on the compatible GDAL version provided by your operating system's package manager (see these deleted notes in the README).
Risks
geopandas==1.0.0 is only a few months old... it was released in June 2024, so it's only been out about 3 months (https://pypi.org/project/geopandas/1.0.0/). Being so new, it might have other dependency constraints that make it harder for users on older systems to use cuspatial.
Notes
In practice, most of this project's tests have been running against geopandas>=1.0.0 for a few months now. Choose any random successful pr workflow from the last 3 months at https://github.com/rapidsai/cuspatial/actions/workflows/pr.yaml?query=is%3Asuccess and look in the wheel-tests-* or conda-python-tests* jobs. You'll see geopandas>=1.0.0 getting pulled in. For example, here's one from July 8th (chosen randomly): https://github.com/rapidsai/cuspatial/actions/runs/9850995522/job/27197475692#step:7:348
more context on why this is coming up now as part of the Python 3.12 work: #1453 (comment)
There was a problem hiding this comment.
I don't have the strongest opinion on geopandas support, but given your breakdown, +1 for requiring >= 1.0.0
There was a problem hiding this comment.
Thanks very much!
Ok, I'm going to merge this as-is then. @harrism @-ing you for awareness... if you have any concerns about this, let me know and I'd be happy to address them in a follow-up PR.
jameslamb
requested review from
AyodeAwe and
bdice
and removed request for
AyodeAwe
jameslamb
changed the title
jameslamb
changed the title
jameslamb
changed the title
bdice
approved these changes
Sep 9, 2024
|
/merge |
jameslamb
changed the title
This was referenced Sep 10, 2024
This was referenced Sep 10, 2024
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Contributes to rapidsai/build-planning#40
This PR adds support for Python 3.12.
Notes for Reviewers
This is part of ongoing work to add Python 3.12 support across RAPIDS.
It temporarily introduces a build/test matrix including Python 3.12, from rapidsai/shared-workflows#213.
A follow-up PR will revert back to pointing at the
branch-24.10branch ofshared-workflowsonce allRAPIDS repos have added Python 3.12 support.
Other changes required to add that support:
projback to 9.3.x, reverting cuproj: remove pin on 'proj' in conda packages #1449 (Add support for Python 3.12, pin proj back to 9.3.x, require geopandas>=1.0 #1453 (comment))geopandas >= 1.0(Add support for Python 3.12, pin proj back to 9.3.x, require geopandas>=1.0 #1453 (comment))This will fail until all dependencies have been updates to Python 3.12
CI here is expected to fail until all of this project's upstream dependencies support Python 3.12.
This can be merged whenever all CI jobs are passing.