Hi! This is my first message here, I hope I'm writing it in the right place. I am experimenting with using the latest WASI preview2 interfaces such as wasi-http from Rust, creating components and running them with wasmtime, using a custom host implementation of all these WASI interfaces. I ran into some tooling issues and I feel I'm close to the point of understanding what's going on but anyway I will describe the whole problem so maybe you can point out if I'm doing something different as it's intended to (I'm quite new to this whole world of WASM).
So what I'm trying to do is using Rust (1.69), and latest cargo component
to build the component. Using the preview1 adapter I can create components using some subset of the preview2 interfaces by mapping Rust's preview1 based std lib implementation. What I have trouble with is using some of the new interfaces, let's say wasi-http. To do so, I was using wit-deps
to add the WITs to the project as dependencies (wasi-http and all its transitive depencencies). Then I'm importing the outgoing-http
interface in my top level WIT so the binding generator generates code for it and I can use it in my implementation. This all works fine and compiles, but then in the step of creating the component from the core module, I get errors of merging WIT interfaces.
The error is the following:
error: failed merge WIT package sets together
Caused by:
failed to merge WIT package sets together
Caused by:
failed to merge package `http` into existing copy
Caused by:
failed to merge document `types` into existing copy
Caused by:
failed to merge interface `types`
Caused by:
expected function `incoming-request-path-with-query` to be present
Later I realized if I'm doing it "by hand" (not via cargo-component
but compiling to a core wasm module and running wasm-tools
myself) then I get a bit more info, it says it fails to merge the preview1 adapter module.
For a long time I did not understand where could a different types
interface coming in the picture. I even renamed the types
interface in wasi-filesystem
to see if that is colliding somehow with the one from wasi-http
. (That did not help).
I also have a trace output from wasm-tools
but that's also not very helpful
(Ends with:
[2023-05-25T11:18:36Z TRACE] merging duplicate package http / Some("path:/http")
error: failed to merge WIT packages of adapter `wasi_snapshot_preview1` into main packages
Caused by:
0: failed to merge package `http` into existing copy
1: failed to merge document `types` into existing copy
2: failed to merge interface `types`
3: expected function `incoming-request-path` to be present
)
After all this I think I realized that there is an other version of the WASI WIT documents in some binary form embedded as custom sections in the preview1 module, and that must be in conflict with the one I'm bringing in. Probably just an older version as the incoming-request-path
mentioned in the error was changed to incoming-request-path-and-query
I think.
So the question is how can I fix this? Should I build a preview1 adapter for myself using the latest WIT packages? Or am I doing something else wrong?
You are spot on with the reason; the conflicting types are coming from here: https://github.com/bytecodealliance/preview2-prototyping/tree/main/wit/deps/http
A previous conversation on this topic: https://bytecodealliance.zulipchat.com/#narrow/stream/327223-wit-bindgen/topic/dependency.20namespacing
Thanks for the link! Indeed by using the older WITs that are in the adapter module everywhere the merging error is gone. (Now I'm having a linking error on the host but that's another story to figure out). That said, I think the wasi-http implementation in wasmtime is already using the newer version of the interface not the one in the adapter module. I understand everything is very much work in progress. If there is any way I can contribute to improve this situation please let me know!
It sounds like, at a minimum, the adapters built into cargo-component
are pretty out of date, I can do an update to latest
Given we've just merged a change to the binary encoding of components, I'm going to hold off on updating cargo-component
until wit-bindgen and the WASI adapaters have been updated
the wasmtime-wasi-http implementation isnt set up to work correctly with the wasmtime-wasi::preview2 implementation yet
so even when these other issues are resolved, there are still some hurdles to overcome to use wasi-http in wasmtime
im working on that soon
Daniel Vigovszky has marked this topic as resolved.
Last updated: Dec 23 2024 at 12:05 UTC