Stream: git-wasmtime

Topic: wasmtime / Issue #1567 Add unique identifier to the Calle...

Wasmtime GitHub notifications bot (Apr 21 2020 at 14:39):

autodidaddict opened Issue #1567:

Feature

As of 0.13.0, host functions are no longer supported via the Callable trait, and instead with functions that accept a Caller as a parameter. Because of this, we _lost_ a lot of functionality that we can't easily replicate, such as the ability to create an instance of a struct that would be the target of the invocation. These instances were capable of storing per-module unique metadata, which we can no longer do.

Benefit

Something like this is _crucial_ for being able to support running multiple modules within the same Rust process that do not interfere with each other's execution. Uniquely identifying them gives our host functions the ability to store per-module call state, metadata, and properly segment/isolate host resources.

Implementation

The simplest approach would be to just assign a monotonically increasing number to each module created within a given runtime, and then expose that same identity as a field/function on the Caller struct.

Alternatives

For more flexibility, you could alternatively expose a hashmap that stores arbitrary per-module metadata. This would let the consumers of wasmtime decide what they want to do with the metadata (e.g. manage an atomic counter for module identity) rather than wasmtime itself.

Wasmtime GitHub notifications bot (Apr 21 2020 at 17:17):

alexcrichton commented on Issue #1567:

Thanks for the report! I'm not sure I fully understand how functionality was lost, though, can you elaborate a bit more on what worked previously but doesn't work any more?

Wasmtime GitHub notifications bot (Apr 21 2020 at 17:49):

autodidaddict commented on Issue #1567:

Use case is for wapc and, by extension, wascc. In both of these projects, we load multiple webassembly modules at once. These modules are able to make host calls wherein the "call state" has fields like the current error, if there is one, the current result, etc. tl;dr this "RPC" contract requires the host call to access data that only belongs to a specific module.

In 0.12.0, with the Callable trait, I was able to drop an Rc<RefCell<State>> into each of my callables, which allowed me to do things like grab the unique ID that I'd assigned to the module and the fields like current error, etc.

In 0.13.0+, the host imports are no longer Callable, they're just bare functions. This means that I cannot bind any state or metadata to that function, so that function has no way of knowing _which fields_ it needs to use in order to satisfy requests from the guest.

A concrete example is we provide an import called host_response. This function lets the guest wasm module obtain a binary blob containing the host's reply to the last function call. We do this to avoid exposing memory allocation patterns in the RPC contract. So, when the guest module calls host_response, we need a way to determine _which response_ to give back to the wasm module, because the assumption is that the host is holding multiple modules, all of which can be executing in parallel via threads.

To recap - in the current version of waPC, we can maintain per-module call state because we have structs that implement Callable. In 0.15.0, we don't have a way to maintain per-module call state because we're running bare functions now, and these bare functions have no "key" with which to look up per-module call state in a static.

Wasmtime GitHub notifications bot (Apr 21 2020 at 18:59):

alexcrichton commented on Issue #1567:

Hm sorry I'm still not sure I fully understand. Each Func is created with an Fn which can capture any state that it wants, which means you can still create per-Func state which can be passed on to host implementations. Do you have some code I could perhaps peek at which might illustrate the problem?

Wasmtime GitHub notifications bot (Apr 21 2020 at 19:08):

autodidaddict commented on Issue #1567:

https://github.com/wapc/wapc-rust/blob/master/src/callbacks.rs#L169

here's a function that uses the old Callable trait. It requires access to a ModuleState in order to satisfy the request.

The parameter you pass to Func::new() needs to be impl Fn(Caller, &[Val], &mut [Val]) .. so this is where I can't figure out how to give that function access to module-specific state.

Wasmtime GitHub notifications bot (Apr 21 2020 at 19:38):

autodidaddict commented on Issue #1567:

I think I might have figured it out. From the documentation on Func::new(), it didn't look like it would take the kind of closure I needed. I'm going to try and create a function that captures state in the closure...

Wasmtime GitHub notifications bot (Apr 21 2020 at 20:44):

autodidaddict commented on Issue #1567:

Confirmed... I was able to convert everything to closures and it all works now. Sorry for wasting your time with an issue - I just didn't realize I could satisfy that function requirement with a closure. This has given me some ideas on how to optimize my own library :)

Wasmtime GitHub notifications bot (Apr 21 2020 at 20:44):

autodidaddict closed Issue #1567:

Feature

As of 0.13.0, host functions are no longer supported via the Callable trait, and instead with functions that accept a Caller as a parameter. Because of this, we _lost_ a lot of functionality that we can't easily replicate, such as the ability to create an instance of a struct that would be the target of the invocation. These instances were capable of storing per-module unique metadata, which we can no longer do.

Benefit

Something like this is _crucial_ for being able to support running multiple modules within the same Rust process that do not interfere with each other's execution. Uniquely identifying them gives our host functions the ability to store per-module call state, metadata, and properly segment/isolate host resources.

Implementation

The simplest approach would be to just assign a monotonically increasing number to each module created within a given runtime, and then expose that same identity as a field/function on the Caller struct.

Alternatives

For more flexibility, you could alternatively expose a hashmap that stores arbitrary per-module metadata. This would let the consumers of wasmtime decide what they want to do with the metadata (e.g. manage an atomic counter for module identity) rather than wasmtime itself.

Last updated: Dec 23 2024 at 12:05 UTC