Messagepack serialization is baked into quickjs-wasm-rs
and javy
. There are other schemaless encoding schemes like cbor
. What kind of thought went into the adoption of messagepack over others? I'm trying to built a set-up for high-performance message passing between QuickJS and the WASM host and hope to avoid going down roads that are already well trodden.
Hi @Geoff Goodman the answer here is probably two-fold: the first thing is developer familiarity. Messagepack is well-known serialization format, even though there might be other formats that are faster, it'll be probably challenging to find a wide range of tooling for it. Second, performance, at least for some of the initial benchmarking that we did, we observed very good performance compared to JSON.
Oh boy, I imagine that the delta w/ JSON was pretty significant, not to mention that encoding some complex types like Date
requires jumping through hoops on both sides.
I'm working on finding ways so that the host would see host-native data types. I think this could be accomplished by the host being required to go through a library that does this under the hood.
I was also thinking that in such an environment performance could be dramatically improved by coordinating on shared memory regions for this back-and-forth over the WASM membrane. WDYT?
Interesting. Yeah, having a way for that cooperation to happen would probably result in a performace improvement, one of the drawbacks of serialization formats like JSON or messagepack is that they require (most of the time), serializing everything up front, which could involve a un wanted performance hit. That said, have you considered the Component Model for this kind of approach? It's not supported today in Javy, but we're working on migrating our current bindings to rquickjs and from there make it easier to support the component model.
I've tried to consume a bunch of the component model and love the concept / theory but don't know how to make practical use of it yet for my use-case. An intuition is that it might help simplify the nitty-gritty of crossing the WASM boundary but since I'm very interested in passing schema-less data (in a potentially schemaful envelope), I think I'd be in a pretty similar position.
What kind of things are you hoping to get out of the component model for javy?
What kind of things are you hoping to get out of the component model for javy?
Mostly optimizing data passing between host <> guest. In my experience, this problem is already complex enough and even though you could probably achieve something similar by other means (having your own system or using an already existing serialization format), the Component Model gives you a series of building blocks you can use to optimize it as much as possible depending on your needs. But I also do agree that all this depends on your use-case!
Are there any existing libraries that can be used today for encoding into the component model's binary serialization without actually building components?
I don't think so, at least I'm not aware of any myself.
Last updated: Dec 23 2024 at 12:05 UTC