Stream: wasm

Topic: [Alternatives to] Running wasmtime on the web

Philpax (Feb 27 2023 at 12:07):

Hi there!

This is an odd question and I wouldn't be surprised if there are no easy answers. That being said - I work on Ambient, which is a platform for games and other 3D experiences, where all serverside user logic is defined in WASM with wit-bindgen powered bindings and run by wasmtime.

We're looking at getting our client to run on the web through compiling to WASM and don't anticipate too many issues there. However, we also want to support clientside WASM logic in the short to medium future, so that people can define complex clientside logic / show UI / what have you.

This puts us in an unfortunate situation: we'll need a way to run this WASM logic on the web, and preferably without too much maintenance burden. Running wasmtime on the web is, as far as I can tell, not really a viable option: there's no interpreter support. (It's also unclear if running a WASM interpreter inside WASM is really that good of an idea, even if the work done will be fairly limited.)

I'm thinking that we could potentially run the modules through Web Workers, but the next open question there is how we'd handle wit-bindgen. We're using an older pre-CM version and will update soon. Even if we do, though, is there a solution to using the CM/WIT on the web today?

Any suggestions would be appreciated. Happy to clarify anything as well :rocket:

bjorn3 (Feb 27 2023 at 12:46):

https://github.com/bytecodealliance/jco allows turning wasm components into wasm code modules and javascript for bindings. I don't know if it works directly in the browser though, which is why I opened https://github.com/bytecodealliance/jco/issues/42.

bjorn3 (Feb 27 2023 at 12:46):

I think the intention is that wasm components will be natively supported by browsers in the future.

bjorn3 (Feb 27 2023 at 12:48):

I think it would be possible to implement most of at least the non-async version of wasmtime's api to work in the browser using the browser's native wasm support to allow code to run with minimal changes, but I have no clue how much work that would be.

Till Schneidereit (Feb 27 2023 at 12:53):

as bjorn3 says, using jco you can run components in the browser as well as in JS runtimes such as Node.js and Deno. I would strongly recommend trying to structure components such that they don't rely on wasmtime-specific APIs, and instead use ones that are easy and efficient to implement in browsers and other environments as well. That might admittedly not be straight-forward, but we'll be happy to support you here on Zulip however we can

Till Schneidereit (Feb 27 2023 at 12:53):

(also, Ambient is really cool and I'm very excited you're betting on WebAssembly and Wasmtime in this way! :partying_face: :heart:)

Philpax (Feb 27 2023 at 13:23):

bjorn3 said:

https://github.com/bytecodealliance/jco allows turning wasm components into wasm code modules and javascript for bindings. I don't know if it works directly in the browser though, which is why I opened https://github.com/bytecodealliance/jco/issues/42.

Hm, that's a really interesting idea! We could potentially unpack the components (using wit-component etc) and drive them. The only problem would be actually calling the functions, but we could potentially define our own wit-bindgen host generator to swap out wasmtime calls for talking to the other modules?

Philpax (Feb 27 2023 at 13:25):

Till Schneidereit said:

as bjorn3 says, using jco you can run components in the browser as well as in JS runtimes such as Node.js and Deno. I would strongly recommend trying to structure components such that they don't rely on wasmtime-specific APIs, and instead use ones that are easy and efficient to implement in browsers and other environments as well. That might admittedly not be straight-forward, but we'll be happy to support you here on Zulip however we can

Yeah, we don't actually explicitly use any wasmtime APIs - we just use WASI (but only really time / stdio) and wit-bindgen-generated bindings. I suspect that our guest code would just work if run on another VM if the bindings were hooked up correctly.

Philpax (Feb 27 2023 at 13:25):

Till Schneidereit said:

(also, Ambient is really cool and I'm very excited you're betting on WebAssembly and Wasmtime in this way! :partying_face: :heart:)

Thank you! Yeah, we're really excited about it too - it's something that I've wanted to see for years now, and I'm glad it's starting to become possible today :)

Till Schneidereit (Feb 27 2023 at 14:47):

if you're only using WASI, then jco in combination with the preview2 work should "just work", at least once you've moved to components, and once all the WASI features you need are implemented

Philpax (Feb 27 2023 at 14:59):

I'm not sure if jco would work for us? The WASM/component to run would be streamed to the client when they join the server; they wouldn't have access to a JS toolchain. I suspect what we'd need to do is use the component library tooling to unpack the component, spawn the WASM bytecode as its own module, and then figure out a way to bridge our client (responsible for spawning the components) to the spawned components. Some kind of synchronous messaging, perhaps?

bjorn3 (Feb 27 2023 at 17:30):

If jco runs in the browser and produces javascript and wasm that can directly be evaluated you could run it on the client right before running the wasm component. I don't know if this is the case, hence why I opened https://github.com/bytecodealliance/jco/issues/42 to request clarification.

Philpax (Feb 28 2023 at 12:41):

Gotcha, thanks! I'll keep an eye on this - for now, I have my hands full getting us up to date with the CM :sweat_smile:

Philpax (Jun 28 2023 at 18:20):

Hi all! I return from the future :smile:

We've been chugging away for the last few months, and are getting pretty close to getting Ambient running on the web. (Un)fortunately, this also means we need to figure out how to solve this :sweat_smile:

Using the proposed solution above, I believe the workflow would be:

• Ambient (runtime, compiled as WASM with wasm-bindgen) downloads the clientside logic WASM/WIT component
• This component is then sent to the "host" JavaScript that's managing Ambient
• The JavaScript then runs JCO on the component to create a JavaScript object that can be called
• The component and runtime are then dynamically linked by the JavaScript, so that when
  • the runtime requests that the component run, its entrypoint is called
  • the component requests to call something from the runtime
    - the component will call out to the JavaScript, which will then talk to the runtime somehow (can't use WIT, as the runtime's compiled with wasm-bindgen)
    - the JavaScript then gets the result from the runtime, and returns to the component with the result

The open questions I have about this are:

• Can JCO be made to run on the web? It looks like people have started working on this last week (lucky timing!), so I'm hoping that will turn out to be a yes.
• Is there overhead to using JCO for this? It seems like a lot of marshalling back/to JavaScript.
• It seems like we'd have to set up our own communication protocol/layer from guest+JS -> runtime, because we can't reuse WIT.

This seems doable, but not ideal.

I was wondering how accessible the machinery for WIT is. I think the ideal solution would look like is something like wasmtime::component::bindgen, but instead of using wasmtime APIs on the web, we use the web APIs for manipulating and calling WebAssembly.

How feasible would it be for us to write our own bindgen-like macro that generates calls to web-sys instead? Would this be a feasible plan of attack, or is the JCO plan still conceptually simpler? Ideally, we can replicate the wasmtime logic and swap out the wasmtime calls for web-sys calls. I imagine it's not quite so simple, though...

We'd base it off wasmtime's bindgen to maintain compatibility as much as possible - it'd be a real shame if we had to write two sets of bindings for the same thing!

I appreciate any input on this; I'm really looking forward to seeing a WASM-"on"-WASM deployment!

Guy Bedford (Jun 28 2023 at 18:22):

On the jco side, there is a tracking issue in https://github.com/bytecodealliance/jco/issues/42 for jco in browsers, and we have already started chipping away on this. I'm currently working on WASI virtualization which should also unlock the remaining part of this.

Philpax (Jun 28 2023 at 18:37):

Thanks for letting me know!

My primary concern with the jco solution is calling into the runtime WASM from the guest WIT-WASM; because the former doesn't have WIT bindings, there's no convenient way to call into it without inventing a WIT-like scheme to request operations from the runtime. (i.e. all you would have available are the C ABI function calls for WASM, and not the full WIT interface, so that layer gets very gnarly)

I can't think of a nice way around this that wouldn't involve a lot of manual maintenance, or writing our own WIT-like solution... and if the latter's necessary, it might be better to build actual WIT bindgen for this scenario. Just not sure about relative efforts.

Karel Hrkal (kajacx) (Jul 02 2023 at 14:44):

Hello @Philpax, I am also working on a hobby video game in Rust that loads WASM modules (now wit components), and I also want to make it work on the web. I have created a new crate for this purpose, called wasm-bridge.

The goal is to "make wasmtime work on the web", the implementation is to provide the same API as the wasmtime rust crate, but use js-sys instead on the web. The only two implemented features are calling an exported function, and importing a function.

Karel Hrkal (kajacx) (Jul 02 2023 at 14:47):

What's more interesting is wit-bindgen / component model support. I am excited to say that I have a minimal possible example of "wit bindgen working on the web", where your code in rust defines imports, loads a wasm component and runs it ON THE WEB.

You can check out the wit-bindgen branch in this repo, but the code is an absolute mess.

The reason I am mentioning this is because I wanted to start working on adding component-model support to wasm-bridge, and I was looking to see if there are other people who would find it useful.

Philpax (Jul 03 2023 at 13:19):

Hi @Karel Hrkal! That's awesome, that's more or less exactly what we need. It looks like it's still early days, but it's promising. What's your estimation of the complexity of the task?

I was looking at taking the existing wasmtime bindgen and decoupling it from wasmtime, then trying to hook up a js-sys backend to it. My concern is that there's a lot of work involved in capturing the nuances of the WIT ABI, especially as it's still somewhat of a moving target, so it's easier to start from something that already works. It'd also make it easier to reuse the same traits between wasmtime and js-sys for the actual bindings.

What do you think of the scope of the problem?

Karel Hrkal (kajacx) (Jul 03 2023 at 17:03):

@Philpax Making wit bindgen work with the component model will definitely be challenging. There are several approaches that I could take, but currently I am thinking this:

Incompatible API

Give up on making the component api 1:1 compatible with wasmtime's. Lot of the code using the API is generated from the "generate!" macro anyway, so the macro could generate different code on the js target and the user would not know the difference anyway.

The bigger problem is that some changes will have to affect the user. For example, loading a component and instantiating a component will require different signature and will be asynchronous. This is kind of bad, but honestly, I do not know a way around it. It's because how the jco transpiled modules are loaded.

The only bright side is that this will be "temporary", once component model is supported in the browser (and node, etc), the API can be unified where you load a component from it's bytes.

Generating the bindgen

Other than that, it should be possible to make the API identical. Most importantly, implementing the ImportObject, calling exported functions on an instance, and generating the used types.

This will definitely be a lot of work, but it might be possible to copy the existing macro and slightly edit it's output instead of re-implementing it from the ground up. I will have to see how it goes once I get there.

Till Schneidereit (Jul 04 2023 at 00:22):

@Karel Hrkal in case you haven't seen it yet, the jco toolchain supports running Wasm Components in any JS environment, including web browsers. That's of course not the same as implementing the Wasmtime API in JS, but in general that shouldn't be necessary, since Components aren't meant to be tied to a specific runtime environment

Karel Hrkal (kajacx) (Jul 04 2023 at 07:26):

Till Schneidereit said:

Karel Hrkal ... the jco toolchain supports running Wasm Components in any JS environment, including web browsers ...

Thanks. I am using jco to run wasm components on the web. when it comes to implementing the Wasmtime API in JS, that's not exactly what I'm trying to do. I want to implement the wasmtime's API in Rust, and have the code compile to WebAssembly so that it can run in the browser.

Karel Hrkal (kajacx) (Jul 04 2023 at 07:58):

The rust code will load the transpiled-by-jco component on the web the same way you would load it from JS using js-sys, but doing it with the same interface as wasmtime's will be the challange. But I have an idea on how to make loading the module and instantiating it synchronous.

Philpax (Jul 04 2023 at 14:01):

@Karel Hrkal Yup, that's definitely a conundrum. We're already using the async bindgen for wasmtime because wasmtime's wasi-preview2 is currently async-only, so component instantiation is already async (...that we block on with pollster), but the rest should be fine.

Agreed on the bindgen - I think the interface should be mostly identical after that, so it shouldn't be too much of a problem from the user's perspective.

The main issue I see is calling back _into_ the host from the guest - host to guest calls aren't too bad, you just need to prime the WIT ABI state and call the function, but I'm not sure what calling a host function would look like. If it's orchestrated entirely from within Rust, it might be possible to do the same kind of WIT ABI state setup and then calling the relevant host function. It gets more complicated if it exits the host's execution context while executing (i.e. guest.exec() leaves the host WASM, so that you cannot call functions inside of it.)

I think this might especially be an issue with JCO, because you can't make a component out of the host WASM, so you have no way to communicate back to the host through JCO alone.

Did you have anything in mind for calling host functions from the guest?

Karel Hrkal (kajacx) (Jul 04 2023 at 15:13):

Philpax said:

It gets more complicated if it exits the host's execution context while executing (i.e. guest.exec() leaves the host WASM, so that you cannot call functions inside of it.)

I'm not sure what you mean. The host calls an exported function on the plugin, and the plugin calls an imported function from the host, so the execution is now is the host, which itself runs in wasm. I guess you could call into JS code from the imported function, is this what you meant by "code execution leaving the host wasm"?

If so, then there shouldn't be any problem, since that JS code cannot interact with the (jco transpiled) wit component, because it would be loaded elsewhere. The JS code execution would finish, then the execution would return to the host imported function, and then to the plugin, and finally back to the host where you called the exported function.

Did you have anything in mind for calling host functions from the guest?

Yes. wasm-bridge already supports imported functions, see imported functions tests, but it's for "normal" modules, not wit components.

When it comes to wit components, there are two options that I'm considering:
1) Read the value's content with Reflect
2) Convert the value to string with json

The idea on how the first option can work is here, but a better way would of course be to generate code that converts the JsValueto Point, and then give it to the user instead of making them use JsValue.

The second option would definitely be easier, JS has built-in JSON support and in Rust you can easily derive (de)serialize with serde. The only issue would be to make sure that the wit -> jco -> Json.serialize path makes the same json as wit -> rust -> serde.serialize would. I would have to try it and see.

That's the issue with converting values both in imported functions and exported functions, both parameters and results.

Philpax (Jul 04 2023 at 16:19):

I'm not sure what you mean. The host calls an exported function on the plugin, and the plugin calls an imported function from the host, so the execution is now is the host, which itself runs in wasm. I guess you could call into JS code from the imported function, is this what you meant by "code execution leaving the host wasm"?

Ah okay, sounds good. My concern was that the nature of the bridging code (i.e. maybe having to exit out to JS to call the JCO component) would lead to the host not being accessible to call back into, but it doesn't sound like that's a problem. Ignore this :)

Yes. wasm-bridge already supports imported functions, see imported functions tests, but it's for "normal" modules, not wit components.

Fantastic! Great work - having that work is already a great proof of concept.

When it comes to wit components, there are two options that I'm considering:
1) Read the value's content with Reflect
2) Convert the value to string with json

The idea on how the first option can work is here, but a better way would of course be to generate code that converts the JsValue to Point, and then give it to the user instead of making them use JsValue.

The second option would definitely be easier, JS has built-in JSON support and in Rust you can easily derive (de)serialize with serde. The only issue would be to make sure that the wit -> jco -> Json.serialize path makes the same json as wit -> rust -> serde.serialize would. I would have to try it and see.

That's the issue with converting values both in imported functions and exported functions, both parameters and results.

Hmm... yeah, that's a frustrating problem. I'd go for the JSON route for now and see what happens. That was one of my concerns with using JCO - going through JavaScript would result in some semantic munging - but it's likely the fastest way to get what we want.

Do you know what happens with values that can't be represented in pure JS/JSON (e.g. u64s and such)? Will JCO round them to the nearest value representable in a f64, or will it use a string?

In an ideal world, we'd do all the interop ourselves, but that would require an implementation of the WIT ABI, which is (probably) non-trivial. (Would appreciate any insight anyone has into this!)

Is there anything we can do to help you with this? Figuring out a solution for WASM-on-WASM-on-web is important to us, so just let us know if there's any kind of assistance we can provide.

Karel Hrkal (kajacx) (Jul 04 2023 at 17:37):

Philpax said:

what happens with values that can't be represented in pure JS/JSON (e.g. u64s and such)?

i64 is represented as BigInt when loading a "normal" module, I'm confident JCO does the same thing. When it comes to converting BigInt to JSON ...

JSON.stringify(100n)
Uncaught TypeError: BigInt value can't be serialized in JSON

So there goes that idea. Another option might be to use wasm-bindgen and their bindings, but I have not really managed to get that to work. So adding a custom derive-able trait for (de)serialization via the Reflect method might be the way forward.

Is there anything we can do to help you with this? Figuring out a solution for WASM-on-WASM-on-web is important to us, so just let us know if there's any kind of assistance we can provide.

Thanks! I can start by showing you what I got with shared screen on discord, feel free to join here: https://discord.gg/VTPDUkmX
Then we can discuss on how to move forward.

Last updated: Nov 22 2024 at 16:03 UTC