IntoPy and FromPyObject integration #1

Activity

1. 1tgr commented on Jun 17, 2021

   

   1tgr

   on Jun 17, 2021

   More actions

   I had a crate (closed source unfortunately) that does the same as pythonize. The idea is to opt in to custom serialization/deserialization.

   I exposed a choice as there are pros and cons with each:

   1. A module that can be used at the field level with #[serde(serialize_with = "as_py_object")]
   2. A SerdePyObject<T> wrapper which has its own Serialize and Deserialize, implemented in terms of the same as_py_object module

   (1) is transparent on the struct as it requires no changes to field types, whereas (2) is finer grained and can be applied like things: HashMap<String, SerdePyObject<MyThing>>.

   In terms of implementation, serialization works as follows:

   1. Borrow a pointer from the PyObject
   2. Write a struct with a specially named [u8] field like { "__pyobject_ptr": [ /* 8 pointer bytes go here */ ] }

   Deserialization works by looking for the specially named field and treating it as a pointer.
2. jmrgibson commented on Apr 27, 2022

   

   jmrgibson

   on Apr 27, 2022

   Last edited by jmrgibson

   More actions

   I would certainly appreciate this. My use case is as follows:

   1. Define a rust enum that can be exposed to python using the pyclass attribute
   2. Include that enum in a rust struct
   3. Use pythonize to convert struct to python dict

   Example:

   use pyo3::prelude::*;
   #[pyclass]
   enum E {
       Foo,
       Bar,
   }
   
   #[derive(Serialize)]
   struct S {
       other_field: String
       enum_field: E,
   }
   
   let s = S {
       other_field: "potato".to_owned(),
       enum_field: E:Foo,
   }
   pythonize::pythonize(py, &s)

   Then in python, this results in a str for the enum field, which can't be compared with the strongly-typed E.Foo.

   out = {'other_field': 'potato', 'enum_field': 'Foo'}
   
   from rust_module import E
   E.Foo == out['enum_field'] # False :(
3. davidhewitt commented on Apr 28, 2022

   

   davidhewitt

   on Apr 28, 2022

   OwnerAuthor

   More actions

   At the moment I'm too busy to look into this myself; contributions are welcome. The proposal by @1tgr sounds like it could work as an opt-in mechanism!

   (I think to have something work automatically would require specialization, though I haven't thought too hard!)