Stream: general

Topic: wasi_snapshot_preview1 module


view this post on Zulip Ari Seyhun (Dec 05 2022 at 16:41):

I'm using wasm-tools component new to convert my wasm module to a component, but I need to pass it the wasi_snapshot_preview1 adapter.
I've previously been using the crate in wit-bindgen <https://github.com/bytecodealliance/wit-bindgen/tree/main/crates/wasi_snapshot_preview1>... but this doesn't have an export for random_get.

A language binding generator for WebAssembly interface types - wit-bindgen/crates/wasi_snapshot_preview1 at main · bytecodealliance/wit-bindgen

view this post on Zulip Ari Seyhun (Dec 05 2022 at 16:41):

I am trying to build https://github.com/bytecodealliance/wasi, but running cargo wasi build doesn't seem to produce any wasm file

Experimental WASI API bindings for Rust. Contribute to bytecodealliance/wasi development by creating an account on GitHub.

view this post on Zulip Alex Crichton (Dec 05 2022 at 16:45):

We're working on a full-fledged adapter with all the features at https://github.com/bytecodealliance/preview2-prototyping which you should be able to use for now (and has random_get implemented)

Polyfill adapter for preview1-using wasm modules to call preview2 functions. - GitHub - bytecodealliance/preview2-prototyping: Polyfill adapter for preview1-using wasm modules to call preview2 func...

view this post on Zulip Alex Crichton (Dec 05 2022 at 16:46):

Note though that we haven't done a ton of testing and the implementation is still in the works so there may be some bugs, but this is intended to be the eventual "official adapter" which may even be baked into wasm-tools

view this post on Zulip Ari Seyhun (Dec 05 2022 at 16:47):

Perfect, it'll do for now!

view this post on Zulip Alex Crichton (Dec 05 2022 at 16:50):

Note that I also added a precompiled binary release to that repo which you can download to avoid building yourself

Polyfill adapter for preview1-using wasm modules to call preview2 functions. - Release Latest Build · bytecodealliance/preview2-prototyping

view this post on Zulip Ari Seyhun (Dec 05 2022 at 16:59):

Hmm when running the command wasm-tools component new I am getting an error now:

view this post on Zulip Ari Seyhun (Dec 05 2022 at 17:00):

I've pulled down the latest from wasm-tools and rebuilt it, so that should be up to date

view this post on Zulip Ari Seyhun (Dec 05 2022 at 17:00):

 wasm-tools component new ./target/wasm32-wasi/release/counter.wasm --adapt ./wasi_snapshot_preview1.wasm -o ./target/wasm32-wasi/release/counter.component.wasm
Error: failed to encode a component from module

Caused by:
    0: failed to validate component output
    1: core instance 2 has no export named `_start` (at offset 0x21cbad)

view this post on Zulip Alex Crichton (Dec 05 2022 at 17:04):

ah you may need to comment out the command in the *.wit for now if you're not using that

view this post on Zulip Alex Crichton (Dec 05 2022 at 17:04):

this makes me think we probably need two separate adapters though

view this post on Zulip Ari Seyhun (Dec 05 2022 at 17:27):

I've tried commenting out the interface command { … } and default export command in the wasi.wit file, and built it with

RUSTFLAGS="-Clink-args=--import-memory -Clink-args=-zstack-size=0" cargo build --release --target=wasm32-unknown-unknown

And building the component with wasm-tools component new worked without errors, but I get an error in wasmtime:

failed to instantiate module

view this post on Zulip Alex Crichton (Dec 05 2022 at 17:33):

is there more to the error message?

view this post on Zulip Ari Seyhun (Dec 05 2022 at 17:34):

Ah sorry I just put the error through dbg! and got more info:

import wasi-filesystem not defined

view this post on Zulip Alex Crichton (Dec 05 2022 at 17:38):

ah yeah you'll need to define the host functions in the component::Linker, and this is where the works isn't finished yet but you can find a bit of a demo in the host folder

view this post on Zulip Alex Crichton (Dec 05 2022 at 17:38):

the theory is to use *.wit to generate host bindings which you implement for your host

view this post on Zulip Alex Crichton (Dec 05 2022 at 17:38):

or, for wasi, you'd use the stock implementation provided with wasmtime

view this post on Zulip Ari Seyhun (Dec 05 2022 at 17:59):

So what I've tried is to import host into my crate directly, and call the add_to_linker function. I believe this is correct. The only issue now is that the getrandom implementation is todo!()
https://github.com/bytecodealliance/preview2-prototyping/blob/main/host/src/random.rs#L6

Polyfill adapter for preview1-using wasm modules to call preview2 functions. - preview2-prototyping/random.rs at main · bytecodealliance/preview2-prototyping

view this post on Zulip Alex Crichton (Dec 05 2022 at 18:01):

heh indeed! As you can see it's earning its name of "prototyping" in the repo name

view this post on Zulip Ari Seyhun (Dec 05 2022 at 18:02):

Awesome, I can hopefully get through it all from here filling in any todos as neeeded. I'm just not too familiar with the wasi_snapshot_preview1 spec.
I assume getrandom(&mut self, len: u32) -> anyhow::Result<Vec<u8>> should just return len random bytes?

view this post on Zulip Ari Seyhun (Dec 05 2022 at 18:05):

Ok((&mut self.rng).sample_iter(Standard).take(len).collect())

view this post on Zulip Alex Crichton (Dec 05 2022 at 18:06):

indeed!

view this post on Zulip Alex Crichton (Dec 05 2022 at 18:06):

And this is where the WASI APIs are being iterated on sort of live here as well, there's not a strict specification for WASI yet for a *.wit-defined WASI

view this post on Zulip Ari Seyhun (Dec 05 2022 at 18:11):

perfect, it all works now ! Thanks for the help

view this post on Zulip Alex Crichton (Dec 07 2022 at 15:56):

I've updated the builds so now there's a wasi_snapshot_preview1.wasm which doesn't export command and there's a wasi_snapshot_preview1.command.wasm which does export a command

view this post on Zulip Amit Prasad (Dec 07 2022 at 19:54):

Hey @Alex Crichton

I've also been trying to use the host bindings from the preview2-prototyping repo. Weirdly, my component doesn't seem to ever call the wasi-logging/log or the wasi-filesystem/pwrite functions, even after compiling to wasm32-wasi, including a println!, and making sure that the component was created using the wasi_snapshot_preview1.wasm adapter module.

Any ideas why this would be the case? I would definitely think that something would print out given that if I don't add_to_linker, then my module won't instantiate because of unsatisfied imports...

view this post on Zulip Alex Crichton (Dec 07 2022 at 19:54):

Can you share the core wasm or the component that you're working with?

view this post on Zulip Amit Prasad (Dec 07 2022 at 19:57):

sure:
Source Code:
https://github.com/AmitPr/wasm-component-serverless/blob/wasi/crates/guest/src/lib.rs
Compiled WASM (after passing it through wasm-tools):
https://github.com/AmitPr/wasm-component-serverless/tree/wasi/wasm

Contribute to AmitPr/wasm-component-serverless development by creating an account on GitHub.
Contribute to AmitPr/wasm-component-serverless development by creating an account on GitHub.

view this post on Zulip Amit Prasad (Dec 07 2022 at 19:58):

I transpiled to WAT and manually traced the function executions.. Looks like it should call the imported pwrite or log

view this post on Zulip Amit Prasad (Dec 07 2022 at 19:59):

I have a copy of the host bindings in the crates subfolder of that repository. It's pretty much identical to what's in the preview2-prototyping repo

view this post on Zulip Amit Prasad (Dec 07 2022 at 20:02):

https://github.com/AmitPr/wasm-component-serverless/blob/wasi/crates/wasi/src/lib.rs#L10-L17

Could this change to wasmtime::component::bindgen! somehow be screwing something up? Everything looks like it checks out to me, but I could be wrong.

Contribute to AmitPr/wasm-component-serverless development by creating an account on GitHub.

view this post on Zulip Alex Crichton (Dec 07 2022 at 20:03):

trying to trace through all this myself

view this post on Zulip Amit Prasad (Dec 07 2022 at 20:05):

Haha there's a lot to unpack in that repo, but I think for the most part the gist of it is that the println!("Got Request!") line in the guest module should be doing something but seems like it's doing nothing

view this post on Zulip Alex Crichton (Dec 07 2022 at 20:06):

yeah nothing jumps out here to me

view this post on Zulip Alex Crichton (Dec 07 2022 at 20:06):

lemme try running and see what happens

view this post on Zulip Alex Crichton (Dec 07 2022 at 20:07):

I'm seeing


view this post on Zulip Alex Crichton (Dec 07 2022 at 20:07):

er, I'm seeing

thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: import `testwasi` not defined', crates/host/src/main.rs:70:22
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

view this post on Zulip Alex Crichton (Dec 07 2022 at 20:07):

are you seeing something different?

view this post on Zulip Amit Prasad (Dec 07 2022 at 20:08):

Ah - are you cloning main or wasi branch -- I've been moving away from a test WASI implementation so main is broken

view this post on Zulip Alex Crichton (Dec 07 2022 at 20:09):

I'm using main, should I look at something else?

view this post on Zulip Amit Prasad (Dec 07 2022 at 20:09):

yeah -- WASI branch
https://github.com/AmitPr/wasm-component-serverless/tree/wasi

Contribute to AmitPr/wasm-component-serverless development by creating an account on GitHub.

view this post on Zulip Amit Prasad (Dec 07 2022 at 20:11):

After cargo run, you should be able to hit curl -D - localhost:3000 from another terminal.

Results in the response you would expect:

HTTP/1.1 200 OK
x-wit-test: true
content-length: 13
date: Wed, 07 Dec 2022 20:08:39 GMT

Hello, world!

But there's no output in the rust server process that I would expect from the print statement that should be triggered by WASI

In essence it's instantiating and running the guest component every time an HTTP request is received

view this post on Zulip Amit Prasad (Dec 07 2022 at 20:12):

Let me see if I can make a minimum example that has the same issue -- This repo is pretty messy (as you can see ;P)

view this post on Zulip Alex Crichton (Dec 07 2022 at 20:14):

nah it's ok, I've got it now and something is definitely fishy

view this post on Zulip Amit Prasad (Dec 07 2022 at 20:15):

Hahaha yeah I was/am extremely confused about the lack of anything. Even putting in panic!() everywhere in the host bindings doesn't seem to do anything.

view this post on Zulip Alex Crichton (Dec 07 2022 at 20:26):

oh I see the bug

view this post on Zulip Alex Crichton (Dec 07 2022 at 20:26):

the problem is that the initial state isn't configured

view this post on Zulip Alex Crichton (Dec 07 2022 at 20:26):

I'll file an issue in a bit

view this post on Zulip Alex Crichton (Dec 07 2022 at 20:27):

basically the command entrypoint sets up fds 1/2/3 but you're not calling that

view this post on Zulip Alex Crichton (Dec 07 2022 at 20:27):

and you arguably shouldn't need to call it either

view this post on Zulip Amit Prasad (Dec 07 2022 at 20:31):

Ah! Interesting. I'll see if I can get around it for now. Thanks for the help :)

view this post on Zulip Amit Prasad (Dec 07 2022 at 20:32):

Is there any literature / documentation that explains what these command features/entrypoints all refer to?

view this post on Zulip Amit Prasad (Dec 07 2022 at 20:55):

https://github.com/bytecodealliance/preview2-prototyping/blob/main/src/lib.rs#L41-L53

I assume that I would have to modify & compile the adapter to include these lines in some other location that's called during initialization/instantiation? Would State::new() be inappropriate?

Polyfill adapter for preview1-using wasm modules to call preview2 functions. - preview2-prototyping/lib.rs at main · bytecodealliance/preview2-prototyping

view this post on Zulip Alex Crichton (Dec 07 2022 at 21:21):

the command was intended for CLI-like executables where it's the main entrypoint

view this post on Zulip Alex Crichton (Dec 07 2022 at 21:21):

what you sort of need though is some init-style function for non-command things

view this post on Zulip Alex Crichton (Dec 07 2022 at 21:21):

or... something like that, honestly I didn't see this coming at all so I dunno what the solution is

view this post on Zulip Alex Crichton (Dec 07 2022 at 22:41):

ok I talked with dan/pat a bit and I think there's a reasonable solution to this which I've now PR'd

When the command entrypoint isn't invoked, such as for non-command builds, then prints were not working as they would return ERRNO_BADF which is swallowed by at least Rust right now. This instead s...

Last updated: Dec 23 2024 at 12:05 UTC