Random "TypeError: object not callable" for a bound method when called from C

[MHe999]

Hello!

We have a set of custom C modules and some Python code on top of them. We use a lot of callbacks defined in the Python code and called from the C code. That all works well.

However, just recently we made some modifications to the Python side and started using object methods as the callback functions. That seemed to work well too, but now I noticed that sometimes a callback that is set only once and has already been successfully called many times during the run, suddenly causes a "TypeError: object not callable" when it gets called from the C code. The Python object in question is created at the start at global level and never destroyed on purpose, so I don't see why it's method is suddenly not callable anymore. What could cause this and is there anything that I can try to fix it?

We're using the 1.20.0 release of the MicroPython and the mp_call_function_1 function for calling the Python callback function from C. I noticed that there are also mp_call_function_1_protected and mp_call_method_* functions available. Should either of those be used when the callback is a bound method and if yes, does using either of them prevent this problem? Like stated earlier, the callback call works many times during the run before it suddenly doesn't anymore.

I see that there are some changes related to bound methods in release 1.22.0, but I'm not sure do they have something to do with this.

[MHe999]

Haven't been able to solve this yet. Some updates:

• Sometimes there is no "TypeError: object not callable" when the callback gets called, but a plain crash. See details below.
• Tested with release 1.22.2, the issue is still present.
• If the callback function is a regular Python function which in turn calls an object's method, everything works with no errors.
• If I understand these correctly, it should be perfectly fine to use a bound method as a callback:
  https://docs.micropython.org/en/latest/reference/isr_rules.html
  https://github.com/orgs/micropython/discussions/9882

If the program crashes, the stacktrace ends either at line if (MP_OBJ_TYPE_HAS_SLOT(type, call)) { or return MP_OBJ_TYPE_GET_SLOT(type, call)(fun_in, n_args, n_kw, args); of the mp_call_function_n_kw function (below) in py/runtime.c

// args contains, eg: arg0  arg1  key0  value0  key1  value1
mp_obj_t mp_call_function_n_kw(mp_obj_t fun_in, size_t n_args, size_t n_kw, const mp_obj_t *args) {
    // TODO improve this: fun object can specify its type and we parse here the arguments,
    // passing to the function arrays of fixed and keyword arguments

    DEBUG_OP_printf("calling function %p(n_args=" UINT_FMT ", n_kw=" UINT_FMT ", args=%p)\n", fun_in, n_args, n_kw, args);

    // get the type
    const mp_obj_type_t *type = mp_obj_get_type(fun_in);

    // do the call
    if (MP_OBJ_TYPE_HAS_SLOT(type, call)) {
        return MP_OBJ_TYPE_GET_SLOT(type, call)(fun_in, n_args, n_kw, args);
    }

    #if MICROPY_ERROR_REPORTING <= MICROPY_ERROR_REPORTING_TERSE
    mp_raise_TypeError(MP_ERROR_TEXT("object not callable"));
    #else
    mp_raise_msg_varg(&mp_type_TypeError,
        MP_ERROR_TEXT("'%s' object isn't callable"), mp_obj_get_type_str(fun_in));
    #endif
}

[stinos]

Sometimes there is no "TypeError: object not callable" when the callback gets called, but a plain crash

This seemingly random behavior often is a result of a piece of memory allocated for the callback getting garbage collected. The first error is then quite typically because the memory which was originally allocated for the callback (i.e. some function object) itself was freed, and then allocated again for another type of object which happens to not be callable. To fix this you 'd have to make sure that the pointer to your callback memory is reachable by the grbage collector such that it sees it's in use, e.g. by storing it in a list which itself is registered as root pointer. Alternatively it's a bug in MicroPython and it's doing things which it shouldn't, but then you should post minimal code which reproduces the issue.

[MHe999]

I was thinking it could be caused by something like this, but didn't know how to fix it. Now it works perfectly, thank you!