Stream: general

Topic: building own WASI-like module


view this post on Zulip Maciej Woś (Feb 25 2020 at 07:26):

@Alex Crichton in wasmtime 0.9 it was possible to crate a WASI-like module through InstanceHandle and then Instance::from_handle (https://github.com/bytecodealliance/wasmtime/blob/v0.9.0/crates/wasi/src/instantiate.rs#L85).

@Krzysztof Woś @Maciej Kot and I are working on a project that relies on this to instantiate our own API module with access to instances memory.

However, your recent changes (https://github.com/bytecodealliance/wasmtime/commit/3dd5a3cb3f0943bfa2c448f5029e975d343e9a2b) close that route. And I don't see any alternatives. Even the comment in latest version of wasi crate states "crates in general should not be doing that" when talking about accessing Instance's memory. Hence, we can't migrate from 0.9 to latest and it looks like we're stuck.

How can we construct a module with our own hostcalls with access to memory? Can you point us at an example?

Standalone JIT-style runtime for WebAssembly, using Cranelift - bytecodealliance/wasmtime
* Reimplement `wasmtime-wasi` on top of `wasmtime` This commit reimplements the `wasmtime-wasi` crate on top of the `wasmtime` API crate, instead of being placed on top of the `wasmtime-*` fam...

view this post on Zulip Alex Crichton (Feb 25 2020 at 15:39):

@Maciej Woś the theory is that this is "solved" with interface types at some point, where that automatically generates adapters to transfer data between memories

view this post on Zulip Alex Crichton (Feb 25 2020 at 15:40):

In the meantime it should work by copying this -- https://github.com/bytecodealliance/wasmtime/blob/f8abe1169c9d0c0e034511bac7bacf030d4ecab8/crates/wasi/src/lib.rs#L20-L75 -- into your crate

Standalone JIT-style runtime for WebAssembly, using Cranelift - bytecodealliance/wasmtime

view this post on Zulip Alex Crichton (Feb 25 2020 at 15:40):

while it's internal and not expected to be used generally, we'll keep something like that working for wasmtime-wasi at least

view this post on Zulip Alex Crichton (Feb 25 2020 at 15:40):

which I think should work for what you're doing?

view this post on Zulip Maciej Woś (Feb 26 2020 at 03:56):

I need to read up on interface types. I'm not sure how would they work in our use case. We simply need hostcalls that can access and modify module's memory.

Regarding WasiCallerMemory, is there an example of how to use it? The only use in wasmtime repo is inside WASI-specific define_struct macro. How can I use WasiCallerMemory without copying all that code?

view this post on Zulip Krzysztof Woś (Feb 26 2020 at 05:16):

The way we implement hostcalls presently is the same as in the example below. So there are two questions at hand:
1) How to migrate that into something compliant with current API
2) What the final API for this should be
I'm not familiar with interface types beyond the basic idea, but it seems to me that what we are after is the API through which you would implement interface types. Due to the nature of how we are going to use the runtime, we would like to be as close to the actual bits as possible, not walled off behind additional interfaces, however convenient these may be in the general case.

view this post on Zulip Krzysztof Woś (Feb 26 2020 at 05:50):

Here's the repo: https://github.com/krzysztofwos/wasmtime-example

Contribute to krzysztofwos/wasmtime-example development by creating an account on GitHub.

view this post on Zulip Alex Crichton (Feb 26 2020 at 15:47):

@Maciej Woś currently you'd have to copy in the WasiCallerMemory type, it's not great :(

view this post on Zulip Alex Crichton (Feb 26 2020 at 15:47):

in general we'd prefer to encourage instantiating a runtime with a Memory value

view this post on Zulip Alex Crichton (Feb 26 2020 at 15:48):

so you'd instantiate the module first, get your memory, then put the memory somewhere you can reference

view this post on Zulip Alex Crichton (Feb 26 2020 at 15:48):

it's a bit circular since you're satifsying imports as well

view this post on Zulip Alex Crichton (Feb 26 2020 at 15:48):

so it's not a great solution

view this post on Zulip Maciej Woś (Feb 26 2020 at 17:02):

@Alex Crichton so we need to copy WasiCallerMemory and then somehow create Func Extern using one of the wrappers? Could you give me an example? Something like Func::wrap1(&store, |mem: WasiCallerMemory|)?

view this post on Zulip Alex Crichton (Feb 26 2020 at 17:02):

@Maciej Woś yeah that should work

view this post on Zulip Maciej Woś (Feb 26 2020 at 17:02):

OK, I'll try it out

view this post on Zulip Till Schneidereit (Feb 26 2020 at 17:36):

@Maciej Woś If none of this works well, or you think your use case could be supported well with a more direct API, then we should definitely look into that, too, so it'd be great if you could report back here once you've done some experiments

view this post on Zulip Maciej Woś (Feb 28 2020 at 05:05):

I rewrote @Krzysztof Woś's example https://github.com/krzysztofwos/wasmtime-example/pull/1/files#diff-639fbc4ef05b315af92b4d836c31b023R35.

Looks like it will work for now. Thanks @Alex Crichton and @Till Schneidereit

I'm just wondering about the interface though. Previously we could construct as single Instance with all the host calls we wanted. That was a pretty neat way to do it. They were resolved as any other import from a separate Wasm module.

With this approach they need to be resolved slightly differently.

Also, how stable is this interface? The comments suggest this is not something that should be done.

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

view this post on Zulip Till Schneidereit (Feb 28 2020 at 12:10):

@Alex Crichton and @Peter Huene I'd be curious to hear your thoughts on this. ISTM that this is a pattern we should support in a stable way, but perhaps there's some other way to support the underlying needs?

view this post on Zulip Alex Crichton (Feb 28 2020 at 15:51):

Yeah this is one thing I was thinking about at some point, the general problem is that you want to create a set of imports with one piece of shared state, and you don't really want to manage the Rc yourself.

view this post on Zulip Alex Crichton (Feb 28 2020 at 15:51):

We support this in a stable way in that you can put your state in an Rc and then share it amongst all of the imports, but it's not exactly the most ergonomic thing in the world (it is safe though unlike the previous iteration!)

view this post on Zulip Alex Crichton (Feb 28 2020 at 15:52):

In some sense though it's all technically the same thing since an instance handle is an Rc so technically it's just who's writing the Rc allocation

view this post on Zulip Alex Crichton (Feb 28 2020 at 15:52):

I'll file an issue about supporting this ergonomically though

view this post on Zulip Alex Crichton (Feb 28 2020 at 15:55):

https://github.com/bytecodealliance/wasmtime/issues/1018

One thing the current wasmtime::Func API isn't great at is sometimes (for example with WASI) you want to create a set of functions that all share the same piece of state. Sort of like a trait t...

view this post on Zulip Maciej Woś (Mar 04 2020 at 03:58):

I've migrated our code to the latest wasmtime and it all looks good except I needed to add more wrappers: https://github.com/bytecodealliance/wasmtime/pull/1222. We have functions with many parameters!

I've been migrating code to the latest version of wasmtime and found myself short of suitable wrapX functions. Hence this PR.

view this post on Zulip Till Schneidereit (Mar 09 2020 at 11:44):

@Maciej Woś does https://github.com/bytecodealliance/wasmtime/pull/1237 address your use case sufficiently?

This commit adds a first-class way to access a caller's memory when defining a Func implemented by the host. This is a very common usage scenario where the wasm module calls a host function wit...

view this post on Zulip Maciej Woś (Mar 10 2020 at 03:59):

Thanks Till. First glance looks great. I copied that type into our code and upgraded to latest. But I'm happy to see it becoming part of the standard API

view this post on Zulip Till Schneidereit (Mar 10 2020 at 10:00):

Great, thank you for confirming that this works for you!


Last updated: Dec 23 2024 at 12:05 UTC