rust-in rust-out bindgen? · general

Hey! Just coming to check: I'm trying to use wasm to make plugins, which means a rust binary, and running rust-based wasm blobs. I'm hoping to be able to basically define traits in binary land and to implement them in blob land.
Do I understand correctly if I say that there currently is no tooling, that the things I see around wit aren't ready yet, and that for now the best I could do would be to implement eg. a serde-based pseudo-RPC protocol manually?

Pat Hickey (Dec 30 2020 at 23:45):

The closest thing to this right now is wiggle, which is for implementing an exported (to Wasm) interface in native rust. You want the other way around, which is to import an interface in native rust that is implemented in Wasm. Wiggle already has some of the pieces you’d want to do this, but it’s far from complete

Pat Hickey (Dec 30 2020 at 23:47):

If you’d like to work on adding that functionality to wiggle I’m happy to get you oriented and point you in the right direction but it’s not something we have in near term plans. Medium to long term we expect interface types to take care of many of the pieces of this problem that wiggle currently solves.

Pat Hickey (Dec 30 2020 at 23:50):

Léo Gaspard (Jan 04 2021 at 18:45):

Thank you for your pointers! That said… is there any documentation for wiggle? The only thing I could find is this ; which looks to me like it doesn't actually do any bindings but just define a trait based on a .witx file, and then it's up to the user to actually make the bindings themselves?

For some more context: not having noticed your answer due to being new to zulip, I have implemented a manual version of bindings based on bincode for serialization of complex types based on an horrible macro-generating macro for exposing rust wasm functions to rust host, and I'm now trying to expose rust host functions to rust wasm, which could maybe happen through wiggle — but then maybe it'd be better to just do another horrible macro for the reverse direction, and to wait for interface types to hopefully help clean up everything?

Writing a simple WASM API layer using interface types and Wasmtime - radu's blog

In this short article we explore how to get started with WebAssembly interface types by defining a simple API layer, then implementing that using Wiggle and Wasmtime

Ekleog/kannader

A highly configurable SMTP server written in Rust. Contribute to Ekleog/kannader development by creating an account on GitHub.

Andrew Brown (Jan 04 2021 at 19:16):

Implementing a WASI Proposal in Wasmtime: wasi-nn

In a previous post, Machine Learning in WebAssembly: Using wasi-nn inWasmtime, we described thewasi-nn specification and a user-level view of its usage in Wasmtime. In this post, we dive intothe details of implementing the proposal using...

Léo Gaspard (Jan 04 2021 at 20:41):

Thank you for the link! Wiggle looks much more useful in this blog post than in the one I had linked :)
There's just something worrying me here: GuestSlice is claimed to point straight into guest memory, but... with the threads-in-wasm proposal, will that not become unsound? I seemed to understand that for future-proofing guest memory should be volatile-read/volatile-written :/

Pat Hickey (Jan 05 2021 at 18:18):

Pat Hickey (Jan 05 2021 at 18:19):

we havent been putting any energy into implementing threads or future proofing our ecosystem for it.

Pat Hickey (Jan 05 2021 at 18:21):

and yeah, unfortunately i never wrote very many docs for wiggle, but andrew has written a great blog about it. im happy to chat here whenever about it

Pat Hickey (Jan 05 2021 at 18:23):

your use case of using wasmtime as a plugin engine is one i would love to see succeed, and I think would make wiggle more useful in general under module linking and other stuff thats coming up soon

Dan Gohman (Jan 05 2021 at 18:24):

Threads will use a linear memory with the shared flag declared on it, so when we add support for threads, one option is to have APIs like GuestSlice fail if the shared flag is set.

Pat Hickey (Jan 05 2021 at 18:25):

yep, and you could even build that functionality into your GuestMemory impl - never permit borrowing a region if the memory is shared

Dan Gohman (Jan 05 2021 at 18:27):

In a sense, the future-proofing is designed at a higher level. We won't ever be in a situation where we have a linear memory that's shared between threads without us knowing about it.

Léo Gaspard (Jan 05 2021 at 21:37):

Sounds great, thank you! (It also means I can remove my usage of core_intrinsics now I know I no longer need to volatile memcpy from/to the wasm memory the things I'm currently sending by serialization ^^)
I'll try to use wiggle for exposing callbacks from host to guest, while still keeping my awful macro for exposing functions from guest to host, and let you know if I hit any issues :)
(FWIW, the first thing I'd suggest would be "link Andrew's blog post from the wiggle readme so people can find it")

Pat Hickey (Jan 06 2021 at 18:58):

wiggle: introduce Trap enum by pchickey · Pull Request #2550 · bytecodealliance/wasmtime

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Léo Gaspard (Jan 06 2021 at 20:15):

Hmm... Is it possible to have the equivalent of a string hashmap? I've tried creating a stringpair type that just contains two strings, and then having an array of that stringpair, but it looks like the stringpair type breaks wiggle's codegen (as in it tries to say str: Sized)

Léo Gaspard (Jan 06 2021 at 22:39):

@Andrew Brown I just noticed, your blog post is missing the cargo:rerun-if-changed lines so cargo works properly with changing the .witx files, maybe it'd make sense to add that? (I'm personally using walkdir, but guess the example could just hardcode the paths)

Andrew Brown (Jan 06 2021 at 22:40):

Andrew Brown (Jan 06 2021 at 22:41):

bytecodealliance/wasmtime

Standalone JIT-style runtime for WebAssembly, using Cranelift - bytecodealliance/wasmtime

Léo Gaspard (Jan 06 2021 at 22:41):

Andrew Brown (Jan 06 2021 at 22:42):

That would be a nice thing to add... do you want to submit a PR to do this to the wasmtime repo and then I will update the blog post?

Léo Gaspard (Jan 06 2021 at 22:42):

Léo Gaspard (Jan 06 2021 at 22:45):

Hmm actually on what repo/branch is it? I can't find the file either in wasmtime master or your fork's master

Léo Gaspard (Jan 06 2021 at 22:45):

Andrew Brown (Jan 06 2021 at 22:45):