"Incompatible function arguments" for pickling __setstate__

[tbridel]

Hello everybody !

I am in a bit of a situation with the setstate method necessary to allow pickling of an object.
I actually get the following error message:

TypeError: __setstate__(): incompatible function arguments. The following argument types are supported:
    1. __setstate__(self, arg: tuple[int, numpy.ndarray[dtype=float64, shape=(*)], numpy.ndarray[dtype=float64, shape=(*)], numpy.ndarray[dtype=float64, shape=(*)], numpy.ndarray[dtype=float64, shape=(*)], numpy.ndarray[dtype=float64, shape=(*)], numpy.ndarray[dtype=float64, shape=(*)], numpy.ndarray[dtype=float64, shape=(*)], numpy.ndarray[dtype=float64, shape=(*)], numpy.ndarray[dtype=float64, shape=(*)], numpy.ndarray[dtype=float64, shape=(*)], numpy.ndarray[dtype=float64, shape=(*)]]) -> None

Invoked with types: fast.vehicles._vehicle_mesh_kernel.VolumeMeshKernel, tuple

with the following binding:

            .def("__setstate__",
                 [](VolumeMeshKernel &self,
                    std::tuple<int, nb::ndarray<nb::numpy, double, nb::shape<nb::any>>,
                               nb::ndarray<nb::numpy, double, nb::shape<nb::any>>,
                               nb::ndarray<nb::numpy, double, nb::shape<nb::any>>,
                               nb::ndarray<nb::numpy, double, nb::shape<nb::any>>,
                               nb::ndarray<nb::numpy, double, nb::shape<nb::any>>,
                               nb::ndarray<nb::numpy, double, nb::shape<nb::any>>,
                               nb::ndarray<nb::numpy, double, nb::shape<nb::any>>,
                               nb::ndarray<nb::numpy, double, nb::shape<nb::any>>,
                               nb::ndarray<nb::numpy, double, nb::shape<nb::any>>,
                               nb::ndarray<nb::numpy, double, nb::shape<nb::any>>,
                               nb::ndarray<nb::numpy, double, nb::shape<nb::any>>> state) {...});

The binding is defined in a file where i have the following includes for now (amongst other local includes):

#include <nanobind/nanobind.h>
#include <nanobind/ndarray.h>
#include <nanobind/stl/tuple.h>

The method doc yields this signature, which to me looks correct but obviously something is not suiting the pickling kernel here ...

     |  __setstate__(...)
     |      __setstate__(self, arg: tuple[int, numpy.ndarray[dtype=float64, shape=(*)], numpy.ndarray[dtype=float64, shape=(*)], numpy.ndarray[dtype=float64, shape=(*)], numpy.ndarray[dtype=float64, shape=(*)], numpy.ndarray[dtype=float64, shape=(*)], numpy.ndarray[dtype=float64, shape=(*)], numpy.ndarray[dtype=float64, shape=(*)], numpy.ndarray[dtype=float64, shape=(*)], numpy.ndarray[dtype=float64, shape=(*)], numpy.ndarray[dtype=float64, shape=(*)], numpy.ndarray[dtype=float64, shape=(*)]], /) -> None

I have a feeling it is related to my specializing the type of the expected tuple, but I am not sure how to do without that specialization.

Could you please advise ?

Thank you very much in advance !! 🙏

[wjakob]

Your description lacks context to unambiguously infer the issue. My guess would be that it's not actually about the tuple, but rather some kind of shape issue (maybe the tensors being used in the function call are 0D or 2D instead of 1D? That would cause the caster to fail.)

Try binding a regular function that takes such an array and calling it with the output of __getstate__ to see what it accepts.

Minor: I would recommend make an alias using np_array = nb::ndarray<nb::numpy, double, nb::shape<nb::any>;, which will signifcantly shorten your __setstate__ function binding declaration.

[tbridel]

Hi @wjakob !

Thank you for your answer.
I think you're right: i changed the __getstate__ serializer to simply send back a list containing the concatenation of all the nb::ndarray and the __setstate__ deserializer to accept such a list as argument, and I do not have any typing validation error anymore.

I get a segmentation fault though so I am still doing something wrong.

Can you please clarify whether the right procedure is similar to what is implemented in your unit test here, or to what is presented in the docs here ?
I.e. are we supposed to assume the C++ instance is already up and ready when __setstate__ is called, or are we supposed to return a new instance and call the constructor ourselves ?
Python documentation mentions that __new__() is called before __setstate__ so I would tend to go for the former idea but ...

[wjakob]

The behavior in the documentation (with placement new) is the way to go. In the Struct testcase, it still works because the object is plain old data (POD).

Basically nanobind allocates memory for the object, but it is still uninitialized. So __setstate__ in all regards works just like __init__: it fills the object with valid contents and transitions it into "valid" state. It is simply a special form of the constructor. If you still have issues with segfaults after incorporating this advice, can you post a small reproducer?