Geoff Goodman (Mar 18 2024 at 15:15): Geoff Goodman (Mar 18 2024 at 15:15):Messagepack serialization is baked into quickjs-wasm-rs and javy. There are other schemaless encoding schemes like cbor. What kind of thought went into the adoption of messagepack over others? I'm trying to built a set-up for high-performance message passing between QuickJS and the WASM host and hope to avoid going down roads that are already well trodden.
Saúl Cabrera (Mar 18 2024 at 15:51):Hi @Geoff Goodman the answer here is probably two-fold: the first thing is developer familiarity. Messagepack is well-known serialization format, even though there might be other formats that are faster, it'll be probably challenging to find a wide range of tooling for it. Second, performance, at least for some of the initial benchmarking that we did, we observed very good performance compared to JSON.
Geoff Goodman (Mar 18 2024 at 16:13):Oh boy, I imagine that the delta w/ JSON was pretty significant, not to mention that encoding some complex types like Date requires jumping through hoops on both sides.
I'm working on finding ways so that the host would see host-native data types. I think this could be accomplished by the host being required to go through a library that does this under the hood.
I was also thinking that in such an environment performance could be dramatically improved by coordinating on shared memory regions for this back-and-forth over the WASM membrane. WDYT?
Saúl Cabrera (Mar 18 2024 at 16:23):Interesting. Yeah, having a way for that cooperation to happen would probably result in a performace improvement, one of the drawbacks of serialization formats like JSON or messagepack is that they require (most of the time), serializing everything up front, which could involve a un wanted performance hit. That said, have you considered the Component Model for this kind of approach? It's not supported today in Javy, but we're working on migrating our current bindings to rquickjs and from there make it easier to support the component model.
Geoff Goodman (Mar 18 2024 at 17:49):I've tried to consume a bunch of the component model and love the concept / theory but don't know how to make practical use of it yet for my use-case. An intuition is that it might help simplify the nitty-gritty of crossing the WASM boundary but since I'm very interested in passing schema-less data (in a potentially schemaful envelope), I think I'd be in a pretty similar position.
What kind of things are you hoping to get out of the component model for javy?
Saúl Cabrera (Mar 19 2024 at 11:01):What kind of things are you hoping to get out of the component model for javy?
Mostly optimizing data passing between host <> guest. In my experience, this problem is already complex enough and even though you could probably achieve something similar by other means (having your own system or using an already existing serialization format), the Component Model gives you a series of building blocks you can use to optimize it as much as possible depending on your needs. But I also do agree that all this depends on your use-case!
Geoff Goodman (Mar 19 2024 at 12:17):Are there any existing libraries that can be used today for encoding into the component model's binary serialization without actually building components?
Saúl Cabrera (Mar 19 2024 at 12:29):I don't think so, at least I'm not aware of any myself.
Last updated: Nov 22 2024 at 17:03 UTC