KeOps 2.0 is now available

[jeanfeydy]

Major improvements for the compilation engine

We have fully rewritten the KeOps compilation engine to make it faster, lighter and easier to deploy.

The old KeOps engine was:

• Written using (or abusing…) the C++ templating syntax.
• Hard to debug, with cryptic compilation error stacks.
• Hard to extend, with verbose C++ meta-programming tools.
• Slow, with 5s-20s of compilation time per KeOps reduction.
• Hard to deploy with dependencies on CMake and the full CUDA environment (nvcc).

In sharp contrast, the new KeOps engine is:

• Written in Python, using straightforward string manipulation syntax.
• Easy to inspect and debug, since the engine creates a human-readable, template-free C++ file for every new “KeOps reduction”.
• Easy to extend, with a much lighter code base.
• Fast, with <0.5s compilation times for GPU reductions.
• Easy to deploy: KeOps only requires a c++ compiler (g++, clang) and, on the GPU, nvrtc headers that are provided by the CUDA drivers.

There should be no downsides to this transition. Notably, we intend this new release to be 100% backward-compatible. If your code breaks with the shift from KeOps v1.5 to KeOps v2.0, please open an issue on our GitHub repo: transition bugs will have top priority.

What should you expect for KeOps v2.1, v2.2?

The revamp of our compilation engine is a milestone. We hope that it will significantly reduce our maintenance work-load and let us focus on improving the library with new features, performance improvements and a better API.

In 2022-23, we intend to focus primarily on:

• Introducing a NumPy-compatible API. This will be done without breaking backward compatibility, with a new SymbolicArray wrapper and a numba-like @jit decorator.
• Add support for approximation strategies: see e.g. the Nystroem and Fast Multipole methods for kernel matrix products, or cluster-based methods for nearest neighbor search.
• Improving run times on CPU, for both compilation and computation steps.
• Adding support for Tensor cores, which are becoming increasingly available with Ampere GPUs.
• Improving support for other languages, especially R and Julia.

In parallel of these improvements, we are working on standard benchmarks for kernel computations and optimal transport solvers.

Feel free to contact us if you would like to join the team, or simply discuss the future of the library.

Other changes

• Add missing matplotlib in doc build dependencies by @gdurif in Add missing matplotlib in doc build dependencies #163
• Add reference to JMLR paper by @gdurif in Add reference to JMLR paper #160
• Fix doc by @chloesrcb in Fix doc  #179
• Rkeops fix keops kernel and more by @gdurif in Rkeops fix keops kernel and more #183
• Edit affiliation in readme by @mdiazmel in Edit affiliation in readme #188
• Move keopscore and pykeops into their own sub-directories by @gdurif in Move keopscore and pykeops into their own sub-directories #222
• Rename default branch to main by @gdurif in Rename default branch to main #228
• Build script for keopscore and pykeops by @gdurif in Build script for keopscore and pykeops #229
• Improving Jenkinsfile regarding PyKeOps to run tests in dedicated Python environments by @gdurif in Improving Jenkinsfile regarding PyKeOps to run tests in dedicated Python environments #230

New Contributors

• @chloesrcb made their first contribution in Fix doc  #179

Full Changelog: v1.5...v2.0

---

This discussion was created from the release KeOps 2.0 is now available.