Stream: general

Topic: Combining different implementations of worlds


view this post on Zulip Mendy Berger (Mar 12 2024 at 02:19):

I'm trying to provide an implementation of a world for wasmtime.

Is it possible to build a library that implements a world, and have the consumers of the library be able to combine it with other worlds to build one larger world, without having to rewrite a lot of code?
E.g. one library provides wasi-webgpu, and another one provides wasi-cli, and the consumer want's to use both.

At a first glance that doesn't seem possible, unless we force the consumers of the library to specify all the same configurations in wasmtime::component::bindgen! as all the libraries it's using, which is really cumbersome.

Is there an easier way to do this?

view this post on Zulip Alex Crichton (Mar 12 2024 at 02:22):

This should be possible to do, but it requires some manual configuration as well as some fiddling with paths. For example with wasi:http it's generated in the wasmtime-wasi-http crate but it refers to other wasi types in the wasmtime-wasi crate and is conditionally added to the CLI.

I think adding things to a linker is probably one of the more manual parts, there's not a great way to orchestrate that happening at WIT bindgen time when there's lots of libraries in play (but we could improve that too)

view this post on Zulip Mendy Berger (Mar 12 2024 at 19:45):

Thanks!
So I guess for now, the consumer will have to do a lot of manual configuration.

view this post on Zulip Mendy Berger (Mar 12 2024 at 19:46):

Actually, I can probably provide a macro that outputs the mappings for with, since that seems like the largest part.

view this post on Zulip Ralph (Mar 12 2024 at 19:46):

baby steps

view this post on Zulip Ralph (Mar 12 2024 at 19:47):

do NOT help your users too much, lest they immediately ask for LOTS MORE HELP

view this post on Zulip Ralph (Mar 12 2024 at 19:47):

:-)

view this post on Zulip Mendy Berger (Mar 12 2024 at 19:47):

:lol:

view this post on Zulip Mendy Berger (Mar 12 2024 at 19:56):

Mendy Berger said:

Actually, I can probably provide a macro that outputs the mappings for with, since that seems like the largest part.

Actually that's not possible, wasmtime::component::bindgen! won't take a macro call.

view this post on Zulip Alex Crichton (Mar 12 2024 at 19:58):

Yeah it does require a chunk of configuration, and we haven't had a great idea about how to reduce the amount required yet

view this post on Zulip Mendy Berger (Mar 12 2024 at 20:01):

What if I structure all the webgpu resources with the exact same paths and names as the wit expects. Can I then at least map it with a single item in with?
That would be a lot better than having to map each resource separately.

view this post on Zulip Alex Crichton (Mar 12 2024 at 20:17):

Last night you inspired me to work on https://github.com/bytecodealliance/wasmtime/pull/8083, and would that do what you're thinking?

This commit implements bytecodealliance/wit-bindgen#840 for Wasmtime. Currently with keys to bindgen only support either mapping resources or interfaces. Now they additionally support mapping entir...

view this post on Zulip Alex Crichton (Mar 12 2024 at 20:17):

that'd help users of webgpu but it wouldn't help you defining webgpu itself

view this post on Zulip Mendy Berger (Mar 12 2024 at 20:21):

Oh wow! That's so cool to hear!

view this post on Zulip Mendy Berger (Mar 12 2024 at 20:21):

Now, how do I inspire you to do my dishes? :lol:

view this post on Zulip Mendy Berger (Mar 12 2024 at 20:23):

If I understand correctly what it's doing, then yes, it will be a major help!
I'm okay having to specify a bit more on the library's side, if I can cut down the code on the consumer's side.

view this post on Zulip Alex Crichton (Mar 12 2024 at 20:39):

Ok cool, that's been on my todo list for awhile so I'm glad you were able to give me the push to finally implement it :)

view this post on Zulip Alex Crichton (Mar 12 2024 at 20:39):

sorry on the dishes, I've got enough of those over here heh

view this post on Zulip Mendy Berger (Mar 12 2024 at 21:05):

Do I lose all the code generation if I specify a whole interface in with? Seems like I now have to implement my own add_to_linker

view this post on Zulip Alex Crichton (Mar 12 2024 at 21:17):

currently, yeah, but I was thinking that was something we should add back in

view this post on Zulip Alex Crichton (Mar 12 2024 at 21:17):

at the very least put a pub use but ideally also synthesize it in add_to_linker

view this post on Zulip Mendy Berger (Apr 15 2024 at 02:10):

Just an update: I ended up splitting the libraries up at the interface level, and managed to get them to play nicely together.
Here you can see the libraries side.
And here you can see the consumer combining multiple libraries.
Ended up working really well.
Thanks for the help!!

This was all on the side of the runtime, now I'm having a similar issue on the guest side with wit-bindgen. I've created a separate thread for that.

Contribute to MendyBerger/wasi-webgpu development by creating an account on GitHub.
Contribute to MendyBerger/wasi-webgpu development by creating an account on GitHub.

Last updated: Dec 23 2024 at 12:05 UTC