Repository navigation
as-jvm failed for a returned list #269
Description
Hi,
I experienced an issue while I was using libpython-clj to encode the returned list as json via cheshire, it threw an exception with this snippet:
(println "Just something interesting from Python:"
(-> (py/py. simple a_list)
py/as-list
json/generate-string))
The exception:
Execution error (ClassCastException) at cheshire.generate/generate$fn (generate.clj:135).
class java.lang.String cannot be cast to class java.util.Map$Entry (java.lang.String and java.util.Map$Entry are in module java.base of loader 'bootstrap')
And it worked fine for a map.
I've created a demo repo for this: https://github.com/whatacold/libpython-clj-as-jvm-issue, hopefully I've put all necessary info there.
Was there something I was missing? Thanks.
Activity
[-]as-jvm for a returned list failed[/-][+]as-jvm failed for a returned list[/+]cnuernber commentedon Feb 18, 2025
I think what you are missing is that as-jvm creates bridge objects that implement a subset of the interfaces in java.util -- ->jvm creates copies of the data. Passing a bridge object into cheshire isn't a use case we support as the resulting pathway is going to call into the python interpreter in a fine grained fashion and is thus very likely to perform quite badly.
Python has json libraries - using their libraries and then only marshalling the string itself across language boundaries is likely to be faster and more correct than as-jvm -> cheshire.
That all being said - this is interesting to me in that it is something that should work -- albeit there are better ways -- and this may represent an area where we could have better support for the jvm interfaces. ham-fisted library interface defaults could be used here.
kenhuang-io commentedon Feb 18, 2025
Thanks for the explanations, I had a feeling that maybe it was wrong to feed it into cheshire directly.
Yes, that was what I did, by using
json.dumpsto encode it to a string for clojure. It's even faster by doing this way? I didn't expect that.I'm wondering, is it rare to return a list/map/set, etc. from python? What do you usually use libpython-clj for? For me, I was rewriting a webapp that was in python, so I had some logic using python regex to do the trick, hence I needed to feed a python function a map and then get back a list.
cnuernber commentedon Feb 18, 2025
The libpython-clj default is to use ->jvm via copy which I think is usually faster than converting to json. Bridging is used for object that have behavior on them and ->jvm is used for datastructures.