Notification Bot (Nov 19 2024 at 21:16): Notification Bot (Nov 19 2024 at 21:16):ocxide has marked this topic as unresolved.
ocxide (Nov 19 2024 at 21:26):I realize that this "conversion problem" is not about passing array-like types but about passing meaningful types.
Creating a wrapper library that handles the conversion from primitives to UUID, etc would be titanic, specially as my API grows.
I think I would need a way to tell wit-bindgen how to transform from defined records to meaningful types such as UUID, datetime or any custom type of my own.
Joel Dice (Nov 19 2024 at 21:33):Would something like record uuid { upper-half: u64, lower-half: u64 } fit your needs?
ocxide (Nov 19 2024 at 21:35):Not exactly, I would like that wit-bindgen converts from this record "uuid" to the actual UUID type from the uuid crate.
Joel Dice (Nov 19 2024 at 21:36):Ah, so you're looking for a kind of "generated From trait implementation" feature
ocxide (Nov 19 2024 at 21:37):Yes, it would be great
Joel Dice (Nov 19 2024 at 21:40):Presumably wit-bindgen would need to have knowledge of the uuid crate (and any others containing types you wanted From impls for) to actually generate something like that.
Lann Martin (Nov 19 2024 at 21:45):Would there be any perf gain by passing the UUID as a tuple?
I think the pragmatic answer here is "probably not measurably unless you are sending millions (?) of them at a time"
ocxide (Nov 19 2024 at 21:45):Yes, I think that this conversion could only work with user-defined records as wit-bindgen would have to know whenever I want a custom type from a primitive.
For example, even if I can express month as a number, in order to transform it to a struct "Month" I would need to create a
record month { value: u64 }
and then somehow link the wit record to the rust struct.
ocxide (Nov 19 2024 at 22:06):Is there such thing as custom mappings?
Joel Dice (Nov 19 2024 at 23:18):You can use the with option to the bindgen! macro to tell it to use already-generated bindings on a per-interface basis, but there's no way to say e.g. "use this Uuid type instead of generating a type for the foo:foo/foo#uuid WIT type".
It's wit-bindgen's job to generate code that marshals each type to and from its canonical ABI representation, and it can't do that if it doesn't know the layout of the type ahead of time. The way it currently has that knowledge is by being the one that generates the type.
Hypothetically, wit-bindgen could provide one or more traits which take care of that ABI marshaling process, which would allow you to supply your own type to the binding generator as long as it implements those traits. That's what the Lift and Lower traits do in Wasmtime's host interface for working with components. Adding something similar for the guest side seems possible. @Alex Crichton did you ever consider something like that?
ocxide (Nov 20 2024 at 01:53):By those traits I think you might refer to something like this:
pub trait FromABI {
type ABI;
fn from_abi(abi: Self::ABI) -> Self;
}
pub trait IntoABI {
type ABI;
fn into_abi(self) -> Self::ABI;
}
This way, wit-bindgen would know any information it needs for transporting data between host and guests.
That being said, I agree that telling wit-bindgen to use certain types instead of generating new ones seems to be out of its responsibility and a bit complicated since some rust types generated differ between caller and owner.
E.g. strings that are represented as &str when calling a function and represented as String when receiving the response.
ocxide (Nov 20 2024 at 02:04):Still, I need that my API receives and returns these custom types.
As I said, a solution for this problem would be to create a wrapper library that exposes every function, type, etc generated by wit-bindgen & bindgen! but with conversions from/to my custom types. That sounds like a lot of work and maintenance, Is there a way to automatically do this? Is there any other method that prevents me from coding a huge glue library?
Pat Hickey (Nov 20 2024 at 03:58):I’ve been dealing with this by writing a wrapper crate that re-exports the types I don’t need to modify, and defining new ones when I do need to modify things. It hasn’t been a huge burden to do so. As someone who has worked on wit-bindgen as well as its predecessors I find it more straightforward to spend some time writing some glue code (I’ve taken to calling it “chrome” instead) on top of the bindgen and not worrying about generalizing bindgen to do everything for me.
Pat Hickey (Nov 20 2024 at 04:01):(deleted)
Alex Crichton (Nov 20 2024 at 15:45):@Alex Crichton did you ever consider something like that?
Not yet, but not because I don't think it'd be plausible! One of the tricky parts is using Rust generics to hook up to multiple parameters to a function which has historically been something difficult, but I think it'd be pretty nice to make things more extensible on the Rust side of things
ocxide (Nov 21 2024 at 19:45):Pat Hickey said:
I’ve been dealing with this by writing a wrapper crate that re-exports the types I don’t need to modify, and defining new ones when I do need to modify things.
Do you have any public repo demonstrating this approach that I can review? I am currently experimenting with creating a glue crate as you suggested.
Specifically, I am using the wit_bindgen::generate! macro within a module of the glue crate and then write the glue functions, types, and traits. However, upon examining the expanded code, it seems this macro is intended to be called at the root of the guest crate.
The main issue is that the generated export! macro, which expects a struct implementing the auto-generated Guest trait. When I try to forward this macro to the guest crate by re-exporting it, I encounter a limitation: the macro's visibility is restricted to the crate where it is defined, preventing it from being re-exported.
So the problem is that the wit_bindgen macros seem designed to be executed directly within the guest crate, while my goal is for the glue code to reside in a separate crate since there will be multiple guest crates sharing the same WebAssembly interface.
How do you address this issue in your glue crate? Any advice or examples would be greatly appreciated. Thanks!
Christof Petig (Nov 28 2024 at 08:24):Although I am a bit late to answer, for uuid either a record or a fixed length list https://github.com/WebAssembly/component-model/issues/181 are best fit, but the second option is not implemented in bindings generators yet, iirc.
raskyld (Dec 06 2024 at 19:51):The main issue is that the generated
export!macro, which expects a struct implementing the auto-generatedGuesttrait. When I try to forward this macro to the guest crate by re-exporting it, I encounter a limitation: the macro's visibility is restricted to the crate where it is defined, preventing it from being re-exported.
I think the best example is: https://github.com/bytecodealliance/wasi-rs
I like to think the bindings are really low-level and that you will almost always have a thin wrapper in most language to make them more user-friendly / idiomatic in their respective language.
Otherwise, I wanted to reference: https://github.com/protocolbuffers/protobuf/issues/2224 which I've been reading recently, and is about similar requirements in Protobuf :upside_down:
Lann Martin (Dec 06 2024 at 20:51):FWIW I do not personally think that UUID encoding should be standardized in this way. In my experience UUIDs are almost always used as opaque string identifiers in their hex-with-hyphens encoding and using a more compact encoding at the ABI will introduce complexity for every target language that seems unlikely to outweigh the (usually very small) benefits. On the flip side, systems that really need compact encoding are more likely to use smaller integer identifiers in the first place. I understand the appeal of leveraging codegen here but this specific feature isn't something I would choose to invest in. (re: the protobuf issue, as it relates here)
Last updated: Dec 23 2024 at 12:05 UTC