Assistance building pygmo2 wheels

[astrojuanlu]

I am collaborating with pygmo2, a Python & C++ under the European Space Agency Github, to try to build and publish manylinux wheels. At the moment the project is only publishing conda packages, the existing wheels got outdated, and no source distributions are on PyPI.

I made significant progress on esa/pygmo2#110 creating a custom Docker image based on quay.io/pypa/manylinux2014_x86_64 that includes all the dependencies. However, I'm hitting a wall now.

The project does not use a conventional procedure with a Python metadata file (setup.py, pyproject.toml, or anything else), but rather a custom CMakeLists.txt that shuffles the Python files around:

https://github.com/esa/pygmo2/blob/aac3942cf/pygmo/CMakeLists.txt

For this reason, to even start cibuildwheel I'm doing touch pyproject.toml, but of course this ends up producing a hollow, pure-Python wheel:

  cibuildwheel: Build failed because a pure Python wheel was generated.

Trying to compile the code inside CIBW_BEFORE_ALL fails because Python3 is not found:

CMake Error at /opt/_internal/pipx/venvs/cmake/lib/python3.9/site-packages/cmake/data/share/cmake-3.24/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find Python3 (missing: Python3_INCLUDE_DIRS Python3_LIBRARIES
  Development Development.Module Development.Embed) (found version "3.11.0")

I get that probably I shouldn't try to compile the code against a specific Python version in CIBW_BEFORE_ALL, and that cibuildwheel will take care of that. But I couldn't find any technical details in the documentation on how the manylinux Docker images work, and the "how it works" section of the docs is a bare flow chart with not much explanation.

In summary, I'm a bit lost on how to proceed. I'm wondering if it's technically possible at all to build the wheel with the current architecture (in which case I'd love to have some guidance), or rather if it will be difficult or impossible and I should first try to standardize the build process using something like scikit-build.

[astrojuanlu]

Looks like it will need some work but I will probably be more successful if I make use of scikit-build, will give it a try over the next few days.