Stream: general

Topic: Architecture for Wasip2 Host/Plugin


view this post on Zulip benwis (Jul 15 2024 at 02:03):

I'm investigating how one might build a plugin system for a Rust CLI app using wasip2, and I have a few questions.

cargo-component works great to generate a rust project with a new empty wit file, and I can compile a wasip2 component. If I wanted to allow people to write plugins, would the correct way to do so is provide a versioned set of .wit files with all the necessary interface types and functions, and a helper crate for internal plugin functionality?

On the host end, it looks like the best way to do that would be to embed wasmtime. For this, it looks fairly straightforward, except for plugin detection and loading. It looks like if I have a wasip2 .wasm file, I can load it with Module::from_file(). Is there a way to verify it provides the expected interface before I try to call functions from it?

Anything else y'all think I should know?

view this post on Zulip Joel Dice (Jul 15 2024 at 13:51):

That all sounds reasonable to me, except you should use e.g. Component::from_file instead of Module::from_file, since WASIp2 files are Wasm components rather than modules. You'll also want to make sure wasmtime-wasi has been added to the component Linker, and I recommend using wasmtime::bindgen! to generate Rust host code from your plugin API WIT files. That generated code will take care of type checking for compatibility when you instantiate the component, as well as provided a statically-typed API you can use to call into plugins and provide host functionality to them.

view this post on Zulip Christof Petig (Jul 15 2024 at 20:54):

You can find example code using multiple different plugin types at (patch still in review) https://gitlab.com/veloren/veloren/-/blob/9929b03cd4425bcc616b6f7c0d14fb571c8e0e3b/common/state/src/plugin/module.rs#L384 - the details are explained in https://veloren.net/blog/devblog-228/ (60% down)

Veloren is a multiplayer voxel RPG written in Rust. It is inspired by games such as Cube World, Legend of Zelda: Breath of the Wild, Dwarf Fortress and...
Glider physics, Weblate, NPC animation plugins

view this post on Zulip benwis (Jul 15 2024 at 21:52):

Christof Petig said:

You can find example code using multiple different plugin types at (patch still in review) https://gitlab.com/veloren/veloren/-/blob/9929b03cd4425bcc616b6f7c0d14fb571c8e0e3b/common/state/src/plugin/module.rs#L384 - the details are explained in https://veloren.net/blog/devblog-228/ (60% down)

That's really awesome! With some help from Joel I've got a basic setup working. I've looked at plugin systems from Zed and Zellij, will have to add Veloren!

view this post on Zulip benwis (Jul 16 2024 at 19:52):

Christof Petig said:

You can find example code using multiple different plugin types at (patch still in review) https://gitlab.com/veloren/veloren/-/blob/9929b03cd4425bcc616b6f7c0d14fb571c8e0e3b/common/state/src/plugin/module.rs#L384 - the details are explained in https://veloren.net/blog/devblog-228/ (60% down)

I'm curious why the patch is still in review, could you link to the merge request? It looks like Veloren has a plugin system already based on wasi-preview1, is that correct?

view this post on Zulip Christof Petig (Jul 16 2024 at 22:54):

The plugin system is already based on p2, but this one is introducing different types of plugins and a more complex example. Sorry, I didn't realize that this link didn't contain the MR number... (searching)

view this post on Zulip Christof Petig (Jul 16 2024 at 22:55):

And it didn't get approved because of vacation, the initial feed back was positive.

view this post on Zulip benwis (Jul 16 2024 at 22:56):

It is? Like I read these docs: https://book.veloren.net/contributors/modders/writing-a-plugin.html
and it seems to just be compiling to wasip1

view this post on Zulip Christof Petig (Jul 16 2024 at 22:58):

https://gitlab.com/veloren/veloren/-/merge_requests/4351

By opening this merge request, you agree to release your code and all other changes under the GPL...

view this post on Zulip Christof Petig (Jul 16 2024 at 22:58):

Updating the docs is next on my list. :face_palm:

view this post on Zulip benwis (Jul 16 2024 at 23:00):

Background here is I just wrote a blog post about it comparing prior art: https://benw.is/posts/plugins-with-rust-and-wasi

It's just a draft right now, but I did make assumptions

One of the most promoted features of WASI preview 2 is the Component model, the idea that we can build binaries in any language that can be run in others. I wanted to do that with Rust, and found the documentation minimal. Here's how I built a plugin system in Rust!

view this post on Zulip Christof Petig (Jul 16 2024 at 23:00):

But the old plugin architecture was broken for some time and this patch has years of history, you can find more information in veloren's discord (plugins channel) .

view this post on Zulip benwis (Jul 16 2024 at 23:01):

So the wasip2 system is released? Guess I'll have to dig into the codebase

view this post on Zulip Christof Petig (Jul 16 2024 at 23:02):

PS: if you look for WIT plugins without wasm I have much more information to share, but this is not related to veloren.

view this post on Zulip Christof Petig (Jul 16 2024 at 23:03):

benwis said:

So the wasip2 system is released? Guess I'll have to dig into the codebase

I really recommend to not look at the state before this MR.

view this post on Zulip benwis (Jul 16 2024 at 23:07):

This is what I had written btw:

Veloren, an open source action RPG, currently has a plugin system based on WASI preview 1, with a system based on WASIp2 proposed here.

The WASIp2 system is interesting in that it appears the intention is to have developers use the WIT files to develop against, instead of hiding them and using a rust crate. All in all, this implementation looks closest to my desired approach. Looking forward to seeing it in production soon.

view this post on Zulip benwis (Jul 16 2024 at 23:13):

Christof Petig said:

PS: if you look for WIT plugins without wasm I have much more information to share, but this is not related to veloren.

Admittedly I have no idea how this would work

view this post on Zulip Christof Petig (Jul 16 2024 at 23:20):

benwis said:

Christof Petig said:

PS: if you look for WIT plugins without wasm I have much more information to share, but this is not related to veloren.

Admittedly I have no idea how this would work

See the links in this

view this post on Zulip Christof Petig (Jul 16 2024 at 23:20):

Thread

view this post on Zulip Christof Petig (Jul 16 2024 at 23:20):

https://bytecodealliance.zulipchat.com/#narrow/stream/327223-wit-bindgen/topic/Native.20plugins.20defined.20in.20WIT/near/447003081


Last updated: Dec 23 2024 at 12:05 UTC