Stream: wamr

Topic: WASI preview2 with WAMR, more easily

Christof Petig (Jul 29 2023 at 10:27):

I really love the more consistent system interface of preview2 (including wit-bindgen) over the old witx based one known as wasi_snapshot_preview1, but SDKs for C++ and Rust only support the old one, as does WAMR.

I found a way to bend wasmtime's wasi-preview1-component-adapter into crating a static library which will redirect preview1 calls to preview2 - without requiring a component linker at runtime. Additionally linking to this library generates a normal wasm module (no component) using the newer APIs. preview1_adapter.patch.gz

If you combine this with a wit-bindgen tailored towards generating a WAMR host stub (I have pieces of this ready in my fork at github.com/cpetig/wit-bindgen) you get close to supporting preview2 APIs with manageable effort, that is without needing import/export alignment (#1353).

I just wanted to share this information on how to use preview2 APIs in case it could be of help.

PS: There is some indication that it might only work with C/++ SDK though due to the linker trick used to implement other-module imports.

lum1n0us (Aug 03 2023 at 11:09):

Thanks for your contribution.

IIUC, the adapter, mentioned above, is a kind of wrapper to bridge the original wasi_snapshot_preivew1 import functions into new wasi_snapshot_preview2 functions. Therefore, toolchains, like wasm-tool component new, are able to create a component model file from an existing core WebAssembly file.

About the patch, https://github.com/bytecodealliance/wit-bindgen/commit/8ca4054013af612c4aca91a44c09dbfe455c73b2, I got some confusion on the concept level.

• the patch won't change the behavior of mentioned adapter. It actually changes the generation of host implementation. Generated "host.cpp" will include "wasm_export.h" additionally.
• Since "wasm_export.h" doesn't actually contain "type" information, I guess it is a trial to combine "native symbol" and "component type"

Christof Petig (Aug 03 2023 at 21:59):

This adapter library will, when linked to a wasm module using the normal wasi-SDK-20 (preview1 functions), redirect these calls to the preview2 functions (e.g. wasi:poll/poll.drop-pollable, wasi:clocks/monotonic-clock.subscribe) which use the modern canonical ABI - which wit-bindgen can create call bindings for, if you pass it the matching wit files.

The c++ host code generator in my repository is far from complete, but once it is ready it would be able to generate matching bindgen code, with a function calling wasm_runtime_register_natives - the actual implementation of course needs to be provided by a host "c library". A plain C host code generator is also possible, I just needed C++ first.

Christof Petig (Aug 03 2023 at 22:02):

poll_host_binding.tgz
This archive contains the generated bindings for the new poll interface - the implementation of the functions is out of scope for this tool.

Christof Petig (Aug 03 2023 at 22:06):

(oh, I see that the poll_oneoff function's binding is still incomplete, I later learned that I need to connect it to the cabi module of wit-parser)

Christof Petig (Aug 03 2023 at 22:07):

But my original intention was to tell how to create a module which imports preview2 functions and which tool helps with creating the host implementation.

Christof Petig (Aug 03 2023 at 22:09):

Also compared to wasm-tool component new --adapt this method still creates a single module - not a component of modules with linking information. Wamr can't parse components, but modules work.

YAMAMOTO Takashi (Sep 28 2023 at 02:10):

does this mean to make wamr implement core-wasm level abi directly?

Christof Petig (Oct 02 2023 at 22:13):

YAMAMOTO Takashi said:

does this mean to make wamr implement core-wasm level abi directly?

I think the answer is more complicated than I originally expected.

I was able to get parts of WASI preview2 working with WAMR using several workarounds:

• as long as the resulting binary is still a single module and not a component it can be loaded into the current WAMR without full component model support (new binary format, one linear memory per module and a linker implementation)
• using a modified preview2 adapter from wasmtime compiled as a static library I was able to get a module which only uses preview2 interfaces using the normal wasi-sdk (preview2 is still a moving target, resources were added last week)
• I can create C++ wamr host bindings using my wit-bindgen fork (very much work in progress), generating the bindings for WASI preview2 should be possible soon
• but the largest gap would be the implementation of the new WASI calls using libc-uvwasi

PS: Perhaps https://github.com/DouglasDwyer/wasm_component_layer can offer some more short cuts towards component model support.
PPS: WASI preview2 feels much more portable and coherent in comparison to preview1, preview1 still was very POSIX centric and I never fully grasped the witx to binding rules (e.g. when results are mapped to integers).

YAMAMOTO Takashi (Oct 03 2023 at 04:38):

what your single module imports is the canon-abi projection of preview2 (or whatever wit-defined interfaces) and wamr needs to provide the abi to make it work, right? it's what i meant by "implement core-wasm level abi directly".

Christof Petig (Oct 03 2023 at 08:07):

Yes, is C++ an option for the host side (you can see how easy an implementation is in my resource demo repo) or would you need plain C?
I have been toying with extending wit-bindgen to create resource bindings for a C wamr host, but this is very low priority for now.

YAMAMOTO Takashi (Oct 05 2023 at 03:39):

ok. ideally we need plain C i guess.

Christof Petig (Oct 05 2023 at 06:12):

I am currently rewriting the c++ code generation from scratch (copying over only the necessary code from the previous implementation), later I will also take a look into C for resources on the host, but I don't expect a full solution ready within this month.

Christof Petig (Nov 12 2023 at 13:14):

Update: The rewrite can already create correct guest C++ code. Host code needs more work.

And I just uploaded a statically linkable preview2 adapter to https://github.com/cpetig/wasmtime-adapter

Christof Petig (Nov 12 2023 at 13:15):

Also so far I was unable to test the adapter, because wasmtime doesn't yet implement all of the preview2 interfaces used by the adapter. I will give it another try with jco, but that will take time.

Ideally this adapter would consist of multiple (object) files, so that only used interfaces are linked in. wasm-opt (binaryen) can reduce unused dependencies for now.

lum1n0us (Nov 14 2023 at 01:18):

Please feel free to correct me. Just want to be very clear about the functionality of lwasi_snapshot_preview1.

image.png

Christof Petig (Nov 14 2023 at 07:12):

I would stack it differently :
C hello world
Wasi sdk libc.a
Libwasi_snapshot_preview2.a
=> a single classical module importing preview2 interfaces, you can still use wabt on this

While the traditional way is to use the classical preview1 module with the Wasi snapshot preview2 module into a component (of two modules) with preview2 interfaces, you can only use wasm-tools to inspect this binary

Christof Petig (Nov 15 2023 at 20:22):

I successfully tested the resulting binary with wasmtime, although I need to create a component out of it. If you want to take a look, here is a C hello world compiled with wasi-SDK 20. Due to linking with the adapter it imports only preview2 interfaces, but is still a normal module.
test_r.wasm
And I just uploaded the used static adapter libraries to https://github.com/cpetig/wasmtime-adapter/releases/tag/2023-11-15

lum1n0us (Nov 17 2023 at 00:55):

That's great. Would u mind uploading an introduction/guide to WAMR repo doc.

Christof Petig (Nov 18 2023 at 07:51):

:thinking: I feel this adds most value once you can also easily implement preview2 interfaces in wamr, which is exactly what I plan to add to wit bindgen (host side C or C++ with wamr function registration) in the upcoming weeks.
I had this working for C++ and a small wit subset, but the code became too awkward before the ongoing rewrite to use more of bindgen's core for lowering and lifting logic.

Ralph (Nov 22 2023 at 17:41):

if we can provide any help, do let us know. This is an area where I have "interested parties"

Christof Petig (Nov 22 2023 at 23:44):

Ralph said:

if we can provide any help, do let us know. This is an area where I have "interested parties"

I find it quite hard to predict the next steps for the code generation, as I basically rewrote my initial C++ code generation to use a Bindgeninterface for lowering and lifting. It works for very simple types (e.g. https://github.com/cpetig/resource-demo ) but structured data types are in progress. Please keep in mind that this code supports both the guest and host side (previously this was a separated crate with lots of code duplication).

The difference between importing and exporting resources is more complex than I initially thought. And the variety of data structures and encoding rules (flattening, variants) adds complexity as well.

You can find the current state of the rewrite at https://github.com/cpetig/wit-bindgen while the non-resource older version is at https://github.com/cpetig/wit-bindgen/tree/old .

Also the mapping to C++ is very interesting as Result is supported in C++23, move semantics map to rvalue references (forwarding references) and how to generate user objects (exporting resources) is difficult to balance. I have iterated between abstract base classes, private Implementation and including user classes in generated code; I like the last solution best.

My guess is that I should have a starting point within a few days and then missing parts could be parallelized. Feel free to already take a look and propose solutions.

Christof Petig (Nov 26 2023 at 22:23):

I posted an update at https://bytecodealliance.zulipchat.com/#narrow/stream/394175-SIG-Guest-Languages/topic/C.2B.2B.20language.20binding/near/404294769 , now even more complex data types are stubbed but already generated for C++ host.

@Ralph would C++ be a potential solution or do you need plain C host code?
At least the code to register host functions is operational with C++ (although most data types aren't correctly mapped to wamr character notation, yet).

Christof Petig (Apr 09 2024 at 23:10):

I promised an update on the progress here. @Ayako Akasaka @Till Schneidereit @Thomas Trenner (I wasn't quick enough to write down all interested names from the chat before the eSIG call ended, so I hope I remember you correctly)

The C++ bindgen in the repo above targets WAMR when using the --host CLI switch. Very simple WIT setups generate fully usable code, see the issues for known deficiencies.

I found the support for data types and resource support much more complex than anticipated last year, so I am nowhere near ready, but with the recent improvements I will give the WASI WITs another try and report back here.

Using the wasmtime-adapter you can already create p1 modules using the p2 WASI interfaces, but these of course can't conpose of several modules, unless you create a component. This file format isn't recognized by WAMR, yet. And it would require multi-memory support (or more specific "one memory per module inside the component" support), a shared everything object could create a shortcut here, but this hasn't been investigated into … although I have been using shared objects for natively compiled components successfully.

Ayako Akasaka (Apr 25 2024 at 15:40):

Thank you for updateing the information about C++ bindgen . @Christof Petig

I found the support for data types and resource support much more complex than anticipated last year, so I am nowhere > near ready, but with the recent improvements I will give the WASI WITs another try and report back here.

I would be happy if you could someday explain the difficult points while looking at the code in e-SIG meeting or maybe tomorrow's meeting or.. some other meeting

Christof Petig (May 05 2024 at 18:43):

@Ayako Akasaka
The problem basically was that I didn't expect the complexity of resources to be that high. It took me several tries to understand how guest exported resources are handled on the canonical ABI level. And then I had to implement both the guest and host side for both guest and host based resources. But today I got close to having working host bindings for resources, in the most simple case.

Also I ran into the exact same problem as Rust when it comes to references and ownership: Shall an option<list<u8>> argument map to optional<std::vector<uint8_t> const& or optional<std::span<uint8_t>> const&- or even define its own struct types. This gets more interesting once you put resources in the vector - because you might not have a vector of the right type around to pass to the function.

lum1n0us (May 09 2024 at 00:38):

a shared everything object could create a shortcut here, but this hasn't been investigated into …

wasm-tools component link will create a component in which all core instances share one memory.

Christof Petig (May 09 2024 at 03:34):

To further precise my statement: I am not aware of any attempt to combine the canonical ABI with shared everything linking on wasm to get wamr compatible core modules. I just know about my above mentioned PoC with native binaries.

I think that this would be a good way to support components in wamr without requiring https://github.com/bytecodealliance/wasm-micro-runtime/issues/1353 and full wasip2 binary format support. These components would be source code identical to wasip2 but use a different file format.

(this compilation target I would associate with the name wasit2)

Christof Petig (May 09 2024 at 03:37):

I really need to look into component link, I haven't used it before, and I can't tell which type of input and output files are supported.

Christof Petig (May 12 2024 at 19:06):

I just looked into component link and found that it won't merge modules, it will just stack them inside a component and create another module redirecting and listing the calls. I feel a shared everything dynamic link object of this type could help with component model implementations for wamr, where you want to combine pre-linked binaries at load time.

In this case the .wasm component would contain one classical app and multiple shared everything dynamic objects and simply act as a container for these files, with linking information.

I now understand that wamr's inability to import memory https://github.com/bytecodealliance/wasm-micro-runtime/issues/1353 will also prevent shared dynamic linking as described in https://github.com/WebAssembly/tool-conventions/blob/main/DynamicLinking.md - thus there is no easier solution, right?

Last updated: Nov 22 2024 at 17:03 UTC