Stream: general

Topic: building own WASI-like module

Maciej Woś (Feb 25 2020 at 07:26):

@Alex Crichton in wasmtime 0.9 it was possible to crate a WASI-like module through InstanceHandle and then Instance::from_handle (https://github.com/bytecodealliance/wasmtime/blob/v0.9.0/crates/wasi/src/instantiate.rs#L85).

@Krzysztof Woś @Maciej Kot and I are working on a project that relies on this to instantiate our own API module with access to instances memory.

However, your recent changes (https://github.com/bytecodealliance/wasmtime/commit/3dd5a3cb3f0943bfa2c448f5029e975d343e9a2b) close that route. And I don't see any alternatives. Even the comment in latest version of wasi crate states "crates in general should not be doing that" when talking about accessing Instance's memory. Hence, we can't migrate from 0.9 to latest and it looks like we're stuck.

How can we construct a module with our own hostcalls with access to memory? Can you point us at an example?

Alex Crichton (Feb 25 2020 at 15:39):

@Maciej Woś the theory is that this is "solved" with interface types at some point, where that automatically generates adapters to transfer data between memories

Alex Crichton (Feb 25 2020 at 15:40):

In the meantime it should work by copying this -- https://github.com/bytecodealliance/wasmtime/blob/f8abe1169c9d0c0e034511bac7bacf030d4ecab8/crates/wasi/src/lib.rs#L20-L75 -- into your crate

Alex Crichton (Feb 25 2020 at 15:40):

while it's internal and not expected to be used generally, we'll keep something like that working for wasmtime-wasi at least

Alex Crichton (Feb 25 2020 at 15:40):

which I think should work for what you're doing?

Maciej Woś (Feb 26 2020 at 03:56):

I need to read up on interface types. I'm not sure how would they work in our use case. We simply need hostcalls that can access and modify module's memory.

Regarding WasiCallerMemory, is there an example of how to use it? The only use in wasmtime repo is inside WASI-specific define_struct macro. How can I use WasiCallerMemory without copying all that code?

Krzysztof Woś (Feb 26 2020 at 05:16):

The way we implement hostcalls presently is the same as in the example below. So there are two questions at hand:
1) How to migrate that into something compliant with current API
2) What the final API for this should be
I'm not familiar with interface types beyond the basic idea, but it seems to me that what we are after is the API through which you would implement interface types. Due to the nature of how we are going to use the runtime, we would like to be as close to the actual bits as possible, not walled off behind additional interfaces, however convenient these may be in the general case.

Krzysztof Woś (Feb 26 2020 at 05:50):

Here's the repo: https://github.com/krzysztofwos/wasmtime-example

Alex Crichton (Feb 26 2020 at 15:47):

@Maciej Woś currently you'd have to copy in the WasiCallerMemory type, it's not great :(

Alex Crichton (Feb 26 2020 at 15:47):

in general we'd prefer to encourage instantiating a runtime with a Memory value

Alex Crichton (Feb 26 2020 at 15:48):

so you'd instantiate the module first, get your memory, then put the memory somewhere you can reference

Alex Crichton (Feb 26 2020 at 15:48):

it's a bit circular since you're satifsying imports as well

Alex Crichton (Feb 26 2020 at 15:48):

so it's not a great solution

Maciej Woś (Feb 26 2020 at 17:02):

@Alex Crichton so we need to copy WasiCallerMemory and then somehow create Func Extern using one of the wrappers? Could you give me an example? Something like Func::wrap1(&store, |mem: WasiCallerMemory|)?

Alex Crichton (Feb 26 2020 at 17:02):

@Maciej Woś yeah that should work

Maciej Woś (Feb 26 2020 at 17:02):

OK, I'll try it out

Till Schneidereit (Feb 26 2020 at 17:36):

@Maciej Woś If none of this works well, or you think your use case could be supported well with a more direct API, then we should definitely look into that, too, so it'd be great if you could report back here once you've done some experiments

Maciej Woś (Feb 28 2020 at 05:05):

I rewrote @Krzysztof Woś's example https://github.com/krzysztofwos/wasmtime-example/pull/1/files#diff-639fbc4ef05b315af92b4d836c31b023R35.

Looks like it will work for now. Thanks @Alex Crichton and @Till Schneidereit

I'm just wondering about the interface though. Previously we could construct as single Instance with all the host calls we wanted. That was a pretty neat way to do it. They were resolved as any other import from a separate Wasm module.

With this approach they need to be resolved slightly differently.

Also, how stable is this interface? The comments suggest this is not something that should be done.

Till Schneidereit (Feb 28 2020 at 12:10):

@Alex Crichton and @Peter Huene I'd be curious to hear your thoughts on this. ISTM that this is a pattern we should support in a stable way, but perhaps there's some other way to support the underlying needs?

Alex Crichton (Feb 28 2020 at 15:51):

Yeah this is one thing I was thinking about at some point, the general problem is that you want to create a set of imports with one piece of shared state, and you don't really want to manage the Rc yourself.

Alex Crichton (Feb 28 2020 at 15:51):

We support this in a stable way in that you can put your state in an Rc and then share it amongst all of the imports, but it's not exactly the most ergonomic thing in the world (it is safe though unlike the previous iteration!)

Alex Crichton (Feb 28 2020 at 15:52):

In some sense though it's all technically the same thing since an instance handle is an Rc so technically it's just who's writing the Rc allocation

Alex Crichton (Feb 28 2020 at 15:52):

I'll file an issue about supporting this ergonomically though

Alex Crichton (Feb 28 2020 at 15:55):

https://github.com/bytecodealliance/wasmtime/issues/1018

Maciej Woś (Mar 04 2020 at 03:58):

I've migrated our code to the latest wasmtime and it all looks good except I needed to add more wrappers: https://github.com/bytecodealliance/wasmtime/pull/1222. We have functions with many parameters!

Till Schneidereit (Mar 09 2020 at 11:44):

@Maciej Woś does https://github.com/bytecodealliance/wasmtime/pull/1237 address your use case sufficiently?

Maciej Woś (Mar 10 2020 at 03:59):

Thanks Till. First glance looks great. I copied that type into our code and upgraded to latest. But I'm happy to see it becoming part of the standard API

Till Schneidereit (Mar 10 2020 at 10:00):

Great, thank you for confirming that this works for you!

Last updated: Nov 22 2024 at 16:03 UTC