Stream: git-wasmtime

Topic: wasmtime / issue #5616 Adding optional rkyv support.

Wasmtime GitHub notifications bot (Jan 21 2023 at 19:09):

jakubDoka opened issue #5616:

Feature

Adding the rkyv support behind a cargo feature.

Benefit

Allows zero copy serialization, which is not supported by serde formats. I particularly find it very useful in my compiler.

Implementation

Traits can be easily derived with proc macros, for all types that derive serde Traits.

I have some experience after integrating it into my project, and it is in my interest to be supported, so I can also implement this.

Alternatives

I don't have notion of other libraries with same capabilities.

Wasmtime GitHub notifications bot (Jan 21 2023 at 20:48):

bjorn3 commented on issue #5616:

Doesn't rkyv produce a separate type for every type which uses the derive for zero-copy deserialization? That would require duplicating a big chunk of the api surface. I believe it also doesn't allow mutating this zero-copy deserialized version, which makes it impossible to compile it without copying it into a regular mutatable Function first. Or were you not talking about adding support for it to Cranelift?

Wasmtime GitHub notifications bot (Jan 21 2023 at 21:18):

jakubDoka commented on issue #5616:

Yes I mean cranelift, sorry should have hed clarified.

Wasmtime GitHub notifications bot (Jan 27 2023 at 23:37):

jameysharp commented on issue #5616:

I think I'm missing something. What do you want Cranelift to use rkyv for? What do you want to serialize/deserialize, and why is it performance-critical?

Wasmtime GitHub notifications bot (Jan 30 2023 at 17:33):

jakubDoka commented on issue #5616:

My compiler uses rkyv to cache incremental builds, and it improves size and speed at which compiler can load and save metadata. I inevitably need to serialize things like signatures and relocations. This leads to defining shadow types and lot of boilerplate. For now I transmute what I can but really don't like code necessary to work around this.

Wasmtime GitHub notifications bot (Jan 30 2023 at 17:47):

bjorn3 commented on issue #5616:

You could use a serde compatible format to serialize the cranelift types and then put the result as opaque blob of type Vec<u8> in your rkyv serializable cache.

Wasmtime GitHub notifications bot (Jan 30 2023 at 18:05):

jakubDoka commented on issue #5616:

Welp, I don't like the idea but seems like it could do the trick. So should i close this? Is there no chance of this being added?

Wasmtime GitHub notifications bot (Jan 30 2023 at 19:52):

jameysharp commented on issue #5616:

I wouldn't say there's no chance. We just need to have a clear picture of what we're signing up for, including any ongoing maintenance burden. That's a little trickier with any new dependencies like rkyv since we need to audit them, and any crates they depend on, for security concerns.

One way to help answer those questions is to have a proof-of-concept PR to look at. Usually I'd prefer to decide whether we want a particular feature through discussion first, such as an issue like this. But in this case it sounds to me like you're familiar enough with both Cranelift and rkyv that you could quickly put together a draft pull request illustrating the basic idea.

I won't promise that we'll merge it, so don't spend too much time on it. But I think for me at least it would help a lot to see what exactly is involved in doing what you're asking for.

Wasmtime GitHub notifications bot (Jan 30 2023 at 21:09):

fitzgen commented on issue #5616:

We already support one serialization library. It is the most popular serialization library for Rust. I'd really rather not support additional serialization libraries unless there is a very good reason. Do we also support miniserde when someone asks for smaller binary sizes? At what point does it stop? I feel like supporting just the number one library is an incredibly reasonable place for us to be at.

Last updated: Nov 22 2024 at 17:03 UTC