Bind child class only -> runtime confusion?

[tdegeus]

I have a heavily templated parent class that I do not want to bind directly. I only want to bind the non-templated child classes. However, this does not work at runtime. Here is a minimal example:

class Bar
{
private:
    size_t m_inc = 0;

public:
    Bar() = default;
    size_t inc() const { return m_inc; }
    void set_inc(size_t inc) { m_inc = inc; }
};

class Foo : public Bar
{
public:
    Foo() = default;
};

NB_MODULE(my_ext, m)
{
    nb::class_<Foo>(m, "Foo")
       .def(nb::init<>())
       .def_prop_rw("inc", &Foo::inc, &Foo::set_inc);
}

At runtime

a = my_ext.Foo()
a.inc = 10

this gives

    a.inc = 10
    ^^^^^
TypeError: <anonymous>(): incompatible function arguments. The following argument types are supported:
    1. <anonymous>(self, arg: int, /) -> None

Invoked with types: my_ext.Foo, int

(With pybind11 this strategy does work, and some informal test suggests that it is much faster at runtime than using py::class_<Foo, Bar>(m, "Foo") and binding Bar. This seems to hold true for nanobind.)

[wjakob]

It's surprising that this works with pybind11, and it seems more like an oversight to me rather than something that should be ported. I am not convinced by the performance argument -- the "slow" version should boil down to one more hash table lookup (in a hash table that is extremely fast -- we are talking about 1-2 loads). This seems vanishingly small to me in the bigger scheme of things. Happy to be proven wrong, but I would first like to see a benchmark demonstrating "much faster runtime".

In the case of pybind11, I can see how the cost would be high (casting from one type to another is one of its bottlenecks).

[tdegeus]

It's actually not quite clear to me how pybind11 figures out that the child and parent class are even related to each other in this case.

Can I investigate what is compelled the end? Could it be that pybind11 in the end compiles all derived methods as being members of the final class? In this end this is what I ask pybind11 to do I think. This would mean that the methods are duplicated in the executable, but that could also explain that I believe that there is a runtime advantage. I can remember having a discussion of gitter some time ago, but I cannot find it back and I cannot remember with whom I had the discussion. But in my recollection I was not alone presuming a runtime advantage.

(I tried nming the .so but that did not tell me much, and these kind of things are not something I usually care about investigating).

[wjakob]

I am actually not concerned about the duplication of the method. What is strange to me is that the pybind11 binding turns &Foo::inc into a Python method that accepts a Foo as implicit self type, which is not at all obvious from the C++ function signature of &Foo::inc (that signature will contain an unknown type Bar). I don't know how pybind11 manages to infer the relationship, and my message above was a request to dig deeper on this front if this feature is important to you. Whether or not such a feature can be accepted depends on the larger implications that are still unclear.

[oremanj]

This is done at compile time in pybind11: class_<T>.def() passes its function argument through method_adaptor<T>:

/// Given a pointer to a member function, cast it to its `Derived` version.
/// Forward everything else unchanged.
template <typename /*Derived*/, typename F>
auto method_adaptor(F &&f) -> decltype(std::forward<F>(f)) { return std::forward<F>(f); }

template <typename Derived, typename Return, typename Class, typename... Args>
auto method_adaptor(Return (Class::*pmf)(Args...)) -> Return (Derived::*)(Args...) {
    static_assert(detail::is_accessible_base_of<Class, Derived>::value,
        "Cannot bind an inaccessible base class method; use a lambda definition instead");
    return pmf;
}

template <typename Derived, typename Return, typename Class, typename... Args>
auto method_adaptor(Return (Class::*pmf)(Args...) const) -> Return (Derived::*)(Args...) const {
    static_assert(detail::is_accessible_base_of<Class, Derived>::value,
        "Cannot bind an inaccessible base class method; use a lambda definition instead");
    return pmf;
}

Seems like nanobind could do the same and it would be a nice little quality-of-life improvement (maybe with less std::forward).

[wjakob]

Good catch, that's indeed it it. But seeing the solution, I must say that I really don't like this. My suggestion would be that you either benchmark the base class option (which I suspect won't share the performance issue you observed in pybind11) or you create a small wrapper on top of nb::class_ (maybe your_project::class_<..>) that passes everything through such a wrapper.

[wjakob]

FWIW, this should now work out of the box thanks to PR #471