Ideas for code reuse across Python classes when targeting wasm32-unknown-emscripten

[kylebarron]

I'm interested in using pyo3 to create Rust-based Python packages that can run in Pyodide in WebAssembly. I have various classes and would like to share methods between them in a way that works when compiling to the wasm32-unknown-emscripten target.

From a previous issue I created, it appears that multiple-pymethods does not and cannot support the wasm32-unknown-emscripten target. I also tried expanding a macro inside an impl block and got

macros cannot be used as items in `#[pymethods]` impl blocks

As a more explicit example, in my project I have various classes PointArray, LineStringArray, PolygonArray, MultiPointArray, MultiLineStringArray, and MultiPolygonArray for storing different types of geospatial data. Each of these will largely have the same geospatial operations, say area. Right now, the prototype uses multiple-pymethods and is implemented like so:

use crate::array::*;
use geoarrow::algorithm::geo::Area;
use pyo3::prelude::*;

macro_rules! impl_area {
    ($struct_name:ident) => {
        #[pymethods]
        impl $struct_name {
            pub fn area(&self) -> Float64Array {
                Area::unsigned_area(&self.0).into()
            }
        }
    };
}

impl_area!(PointArray);
impl_area!(LineStringArray);
impl_area!(PolygonArray);
impl_area!(MultiPointArray);
impl_area!(MultiLineStringArray);
impl_area!(MultiPolygonArray);

But this won't work when compiling to WebAssembly because it has multiple pymethods.

Likewise I wanted to see if macros could possibly work, but a simple test for that failed:

#[pyclass]
pub struct Test;

macro_rules! impl_hello_world {
    () => {
        pub fn hello_world(&self) -> String {
            "hello world".to_string()
        }
    };
}

#[pymethods]
impl Test {
    impl_hello_world!();
    // error: macros cannot be used as items in `#[pymethods]` impl blocks
}

So I wanted to ask: does anyone have any ideas for sharing code among classes that can be compiled to wasm32-unknown-emscripten?

[davidhewitt]

I've thought about this myself sometimes. The only option I think right now would be for you to add your own proc macro which you add before #[pymethods] which injects the code you need.

Ultimately my preferred solution long-term would be for "distributed slice" support like what's in the linkme crate to be merged into the rust compiler, but I suspect that has challenges and needs an RFC. Then we could make the multiple-pymethods feature work everywhere and not optional.