py-build-cmake cross-compile Windows arm64

[laggykiller]

From https://cibuildwheel.readthedocs.io/en/stable/faq/#windows-arm64

Currently, setuptools>=65.4.1 and setuptools_rust are the only supported backends.

This PR would allow py-build-cmake projects to cross-compile on Windows to arm64.

Before merging, please read the following:

Note that py-build-cmake.cross.toml has to be either present inside the project to be build, or -C--cross=/path/to/my-cross-config.toml flag has to be present during build. The former method is less clean, so I opted for the second option.

Unfortunately, in cibuildhwheel/windows.py:

                extra_flags = split_config_settings(build_options.config_settings, build_frontend)

                # Dirty hack for py-build-cmake cross-compiling
                if env.get('PY_BUILD_CMAKE_EXTRA_FLAGS'):
                    extra_flags.append(env['PY_BUILD_CMAKE_EXTRA_FLAGS'])

                if build_frontend == "pip":
                    extra_flags += get_build_verbosity_extra_flags(build_options.build_verbosity)
                    # Path.resolve() is needed. Without it pip wheel may try to fetch package from pypi.org
                    # see https://github.com/pypa/cibuildwheel/pull/369
                    call(
                        "python",
                        "-m",
                        "pip",
                        "wheel",
                        options.globals.package_dir.resolve(),
                        f"--wheel-dir={built_wheel_dir}",
                        "--no-deps",
                        *extra_flags,
                        env=env,
                    )

The extra_flags comes from build_options.config_settings, but setup_python() and in turn setup_py_build_cmake_cross_compile() does not have access to build_options.config_settings. Hence I had to create the 'dirty hack'. Any advice on making it cleaner before merging?

Also, code searching for suitable cl.exe from Visual Studio feels a bit long and might not work in all cases. Any suggestion to improve it before merging?

Useful references:
https://tttapa.github.io/py-build-cmake/Cross-compilation.html
https://tttapa.github.io/py-build-cmake/Config.html
https://stackoverflow.com/questions/66063056/cross-compiling-for-x64-arm-with-msvc-tools-on-windows

[laggykiller]

I think one of the solutions to the 'dirty hack' is to modify py-build-cmake such that it can recognize environment variable (PY_BUILD_CMAKE_CROSS) and specify /path/to/my-cross-config.toml using that environment variable.

I have created PR tttapa/py-build-cmake#15 that make this happen.

Once that PR is merged, I can then merge https://github.com/laggykiller/cibuildwheel/tree/py_build_cmake_env_var into main and add to this PR.

Edit: Somehow the PR is not working, can anyone help? Or any better idea?

[laggykiller]

btw, you may see a sample of cross-compile Windows arm64 in py-build-cmake project here: https://github.com/laggykiller/rlottie-python

[henryiii]

A few thoughts. First, we really shouldn't be doing backend-by-backend hacks here, especially for very rarely used backends. There are 18 matches for py-build-cmake on GitHub. It's better for backends to use what we provide (for setuptools) for now, while cross compiling is being worked on. That's what scikit-build-core does. (FYI, is there a reason you can't use scikit-build-core? It supports cross-compilation for Windows ARM on cibuildwheel already. You just have to be careful with the SOABI, scikit-build-core gives to the correct value, but FindPython doesn't)

Second, I am curious about the long term plans for it. I'm okay with multiple build backends, but if @tttapa is interested in helping with scikit-build-core, having more effort into one package would be best long term, I think. Also, since py-build-cmake is based on using flit-core internals, I'd rather expect it not to be viable long term, as it will need to adapt to changes in Flit. Scikit-build-core was built from the ground-up, has extensive tests, etc.

Third, py-build-cmake hasn't had a commit since April.

Finally, there is an attempt to standardize cross compiling, starting with PEP 720, which highlights the current status of cross compilation. I don't think we should be adding to many more hacks until that's settled.

[tttapa]

I agree with Henry that most of the tasks handled by this PR should be done inside of the backend, not inside of cibuildwheel.
I've just published py-build-cmake 0.1.9a1, which supports (cross-) compilation for 32-bit Windows and Windows on ARM64 using cibuildwheel. It uses the DIST_EXTRA_CONFIG environment variable set by cibuildwheel:

https://github.com/tttapa/py-build-cmake/blob/d320d1891da34bcbdd66a380cb3db557bfe033c1/src/py_build_cmake/quirks/config.py#L138-L146

It should now work out of the box with cibw, see e.g. https://github.com/tttapa/py-build-cmake-example/blob/e4172e092e8945beaa09dc509b7bf0f9bd77c807/.github/workflows/wheels.yml#L152

I'm okay with multiple build backends, but if @tttapa is interested in helping with scikit-build-core, having more effort into one package would be best long term,

I agree. py-build-cmake was born out of the necessity to easily cross-compile binary packages, mostly for Linux on ARM. At the time, there were no alternatives: PEP 517 was still very new, and setuptools-based tools like scikit-build did not provide the necessary settings to allow for painless cross-compilation, at least not without manually overriding messy setuptools methods.
Are there any open discussions about adding flexible Linux cross-compilation support to scikit-build-core?

Also, since py-build-cmake is based on using flit-core internals, I'd rather expect it not to be viable long term, as it will need to adapt to changes in Flit

Currently, py-build-cmake uses flit_core to read the PEP 621 metadata from pyproject.toml (including the dynamic parts), and to export the sdist. These are not worth reimplementing in py-build-cmake itself, but it would be nice if they could one day be included in distlib. I might open a PR when I have more time in a couple of months. Alternatively, I could just grab the necessary parts from flit and include them with py-build-cmake to make it more stand-alone.

Third, py-build-cmake hasn't had a commit since April.

Why make changes if it works? :)

Finally, there is an attempt to standardize cross compiling, starting with PEP 720, which highlights the current status of cross compilation. I don't think we should be adding to many more hacks until that's settled.

It's good to see this effort. This is definitely something I'll look into. Packaging binary Python packages has been quite a pain compared to e.g. Julia (where cross-compilation is the default, resulting in portable packages without accidental library dependencies and libc versioning troubles).

[laggykiller]

@tttapa thanks for your hardwork! I am testing it

[henryiii]

I've just published py-build-cmake 0.1.9a1

Excellent! @laggykiller, could you see if that works for you?

Are there any open discussions about adding flexible Linux cross-compilation support to scikit-build-core?

There's only scikit-build/scikit-build#1013, and nothing detailed in scikit-build-core yet. Though that would make sense to open. :) I'm curious as to how you handle linux - it's hard due to manylinux & the RH dev toolkit that's used to provide a newer GCC with the old GLIBC. Without that workaround, you are stuck with compilers that are too old to work on the special architectures (at least that's what NumPy hit, They needed GCC > 6 or 7 to work on ARM, IIRC). I've heard recently that the zig has something can compile manylinux C++, but I haven't looked into it yet. And you could statically link, of course, but then you are trading larger binaries for a faster compile time, and that's usually not worth it, IMO, when you can emulate. (Unless you time out emulating, of course).

I'd love to hear your thoughts and ideas, and what you'd do the same way and what you'd change.

PS, I've also focused on using FindPython & back porting it to older Pythons, but if it made sense to have a custom module for cross-compiling, scikit-build-core supports Python packages supplying CMake modules.

Currently, py-build-cmake uses flit_core to read the PEP 621 metadata from pyproject.toml (including the dynamic parts), and to export the sdist

If that's all it's using it for, then why not use pyproject-metadata? scikit-build-core and meson-python both use that.

[henryiii]

Also, would you be interested in adding a "see also" link to the readme or docs to scikit-build-core? We've had one to py-build-cmake for a couple of months. :)

Why make changes if it works? :)

Well, usually I at least bump the pre-commit hooks via pre-commit.ci every week or month. https://learn.scientific-python.org/development/guides/style :)

[laggykiller]

@tttapa Seems to be working well! I'm gonna close this and related PRs. This saves much headache on my side!

https://github.com/laggykiller/rlottie-python/blob/master/.github/workflows/build.yml

[laggykiller]

One small thing to ask though: Currently it depends on setup_setuptools_cross_compile() setting DIST_EXTRA_CONFIG environment variable in cibuildwheel/windows.py to cross-compile py-build-cmake projects, should the code in cibuildwheel/windows.py be structured / add comments to show that setting DIST_EXTRA_CONFIG is also helping py-build-cmake to cross-compile?

btw Consider updating documentation, specifically https://cibuildwheel.readthedocs.io/en/stable/faq/#windows-arm64 to mention that py-build-cmake>=0.1.9a1 backend also support cross-compile to arm64 on Windows?

[tttapa]

I'm curious as to how you handle linux - it's hard due to manylinux & the RH dev toolkit that's used to provide a newer GCC with the old GLIBC.

In short, I use my own cross-compilation toolchains that I build using crosstool-ng. This allows me to select an older GLIBC version with the latest GCC (I use C++17 and C++20 in many projects).
Once you have the toolchain with an appropriate CMake toolchain file, the rest is simple enough: creating one or more sysroot folders that you cross-compile your dependencies into. Usually, the number of dependencies is limited, so a handful of small shell scripts do the job, but I've been looking at Conan to improve the ergonomics there.
The toolchains and pre-compiled dependencies (including Python itself) are built using a big GitHub Actions matrix in a separate repository, and I simply download and cache them in the CI of my Python projects.

AFAIK, there's no way around statically linking the C++ standard library. Newer versions of GCC require newer versions of libstdc++, and since you cannot assume that they're available on the target system, you have to include them in your package somehow.

What a tool like cibuildwheel needs then, is a Docker container (or just a big archive, really) containing:

• A native Python installation for the build machine
• GCC cross-compilers for the host, with a sufficiently old GLIBC
• A minimal cross-compiled Python installation for the host: just the python-config script, the headers, and libs (or stubs thereof if you only need to build extension modules)

To initiate the build, cibuildwheel should pass to the build backend: 1. the path to the compilers, 2. the root path of both Python installations, and 3. the path to a user-provided staging area containing pre-built dependencies for the host.
It should then be up to the build backend to turn that information into e.g. a CMake toolchain file (or a Conan profile, or a Meson cross build definition file, etc.) and invoke the build system with the appropriate options.

IMHO, the Python build tools should only provide the necessary information (i.e. filesystem paths) to the native build tools. Handling all the quirks of cross-compilation is beyond their scope, and tools like CMake usually do a good job of supporting cross-compilation already, with the advantage of being the de facto standard with plenty of online resources. Determining Python3_SOABI for a given Python installation is CMake's job, not the Python build front- or back-end's.
The Python tools should be as shallow and as simple as possible: by adding workarounds and special cases to the Python tools, you get surprising behavior, making it much harder to debug when things go wrong (“who added this flag?” or “where did this environment variable get modified? Github Actions? CIBW? Pip? The build backend? My own CMake script?”).

This is getting quite long already, what would be the best place to discuss further? discuss.python.org? A GitHub discussion? In which repository?

Also, would you be interested in adding a "see also" link to the readme or docs to scikit-build-core?

Done! (I'll merge it into main shortly)

Well, usually I at least bump the pre-commit hooks via pre-commit.ci every week or month. https://learn.scientific-python.org/development/guides/style :)

Thanks for the link!

---

btw Consider updating documentation, specifically https://cibuildwheel.readthedocs.io/en/stable/faq/#windows-arm64 to mention that py-build-cmake>=0.1.9a1 backend also support cross-compile to arm64 on Windows?

I'll see if I can open a PR after I make the next stable release.