Stream: general

Topic: wasi_snapshot_preview1 module

Ari Seyhun (Dec 05 2022 at 16:41):

I'm using wasm-tools component new to convert my wasm module to a component, but I need to pass it the wasi_snapshot_preview1 adapter.
I've previously been using the crate in wit-bindgen <https://github.com/bytecodealliance/wit-bindgen/tree/main/crates/wasi_snapshot_preview1>... but this doesn't have an export for random_get.

Ari Seyhun (Dec 05 2022 at 16:41):

I am trying to build https://github.com/bytecodealliance/wasi, but running cargo wasi build doesn't seem to produce any wasm file

Alex Crichton (Dec 05 2022 at 16:45):

We're working on a full-fledged adapter with all the features at https://github.com/bytecodealliance/preview2-prototyping which you should be able to use for now (and has random_get implemented)

Alex Crichton (Dec 05 2022 at 16:46):

Note though that we haven't done a ton of testing and the implementation is still in the works so there may be some bugs, but this is intended to be the eventual "official adapter" which may even be baked into wasm-tools

Ari Seyhun (Dec 05 2022 at 16:47):

Perfect, it'll do for now!

Alex Crichton (Dec 05 2022 at 16:50):

Note that I also added a precompiled binary release to that repo which you can download to avoid building yourself

Ari Seyhun (Dec 05 2022 at 16:59):

Hmm when running the command wasm-tools component new I am getting an error now:

Ari Seyhun (Dec 05 2022 at 17:00):

I've pulled down the latest from wasm-tools and rebuilt it, so that should be up to date

Ari Seyhun (Dec 05 2022 at 17:00):

❯ wasm-tools component new ./target/wasm32-wasi/release/counter.wasm --adapt ./wasi_snapshot_preview1.wasm -o ./target/wasm32-wasi/release/counter.component.wasm
Error: failed to encode a component from module

Caused by:
    0: failed to validate component output
    1: core instance 2 has no export named `_start` (at offset 0x21cbad)

Alex Crichton (Dec 05 2022 at 17:04):

ah you may need to comment out the command in the *.wit for now if you're not using that

Alex Crichton (Dec 05 2022 at 17:04):

this makes me think we probably need two separate adapters though

Ari Seyhun (Dec 05 2022 at 17:27):

I've tried commenting out the interface command { … } and default export command in the wasi.wit file, and built it with

RUSTFLAGS="-Clink-args=--import-memory -Clink-args=-zstack-size=0" cargo build --release --target=wasm32-unknown-unknown

And building the component with wasm-tools component new worked without errors, but I get an error in wasmtime:

failed to instantiate module

Alex Crichton (Dec 05 2022 at 17:33):

is there more to the error message?

Ari Seyhun (Dec 05 2022 at 17:34):

Ah sorry I just put the error through dbg! and got more info:

import wasi-filesystem not defined

Alex Crichton (Dec 05 2022 at 17:38):

ah yeah you'll need to define the host functions in the component::Linker, and this is where the works isn't finished yet but you can find a bit of a demo in the host folder

Alex Crichton (Dec 05 2022 at 17:38):

the theory is to use *.wit to generate host bindings which you implement for your host

Alex Crichton (Dec 05 2022 at 17:38):

or, for wasi, you'd use the stock implementation provided with wasmtime

Ari Seyhun (Dec 05 2022 at 17:59):

So what I've tried is to import host into my crate directly, and call the add_to_linker function. I believe this is correct. The only issue now is that the getrandom implementation is todo!()
https://github.com/bytecodealliance/preview2-prototyping/blob/main/host/src/random.rs#L6

Alex Crichton (Dec 05 2022 at 18:01):

heh indeed! As you can see it's earning its name of "prototyping" in the repo name

Ari Seyhun (Dec 05 2022 at 18:02):

Awesome, I can hopefully get through it all from here filling in any todos as neeeded. I'm just not too familiar with the wasi_snapshot_preview1 spec.
I assume getrandom(&mut self, len: u32) -> anyhow::Result<Vec<u8>> should just return len random bytes?

Ari Seyhun (Dec 05 2022 at 18:05):

Ok((&mut self.rng).sample_iter(Standard).take(len).collect())

Alex Crichton (Dec 05 2022 at 18:06):

indeed!

Alex Crichton (Dec 05 2022 at 18:06):

And this is where the WASI APIs are being iterated on sort of live here as well, there's not a strict specification for WASI yet for a *.wit-defined WASI

Ari Seyhun (Dec 05 2022 at 18:11):

perfect, it all works now ! Thanks for the help

Alex Crichton (Dec 07 2022 at 15:56):

I've updated the builds so now there's a wasi_snapshot_preview1.wasm which doesn't export command and there's a wasi_snapshot_preview1.command.wasm which does export a command

Amit Prasad (Dec 07 2022 at 19:54):

Hey @Alex Crichton

I've also been trying to use the host bindings from the preview2-prototyping repo. Weirdly, my component doesn't seem to ever call the wasi-logging/log or the wasi-filesystem/pwrite functions, even after compiling to wasm32-wasi, including a println!, and making sure that the component was created using the wasi_snapshot_preview1.wasm adapter module.

Any ideas why this would be the case? I would definitely think that something would print out given that if I don't add_to_linker, then my module won't instantiate because of unsatisfied imports...

Alex Crichton (Dec 07 2022 at 19:54):

Can you share the core wasm or the component that you're working with?

Amit Prasad (Dec 07 2022 at 19:57):

sure:
Source Code:
https://github.com/AmitPr/wasm-component-serverless/blob/wasi/crates/guest/src/lib.rs
Compiled WASM (after passing it through wasm-tools):
https://github.com/AmitPr/wasm-component-serverless/tree/wasi/wasm

Amit Prasad (Dec 07 2022 at 19:58):

I transpiled to WAT and manually traced the function executions.. Looks like it should call the imported pwrite or log

Amit Prasad (Dec 07 2022 at 19:59):

I have a copy of the host bindings in the crates subfolder of that repository. It's pretty much identical to what's in the preview2-prototyping repo

Amit Prasad (Dec 07 2022 at 20:02):

https://github.com/AmitPr/wasm-component-serverless/blob/wasi/crates/wasi/src/lib.rs#L10-L17

Could this change to wasmtime::component::bindgen! somehow be screwing something up? Everything looks like it checks out to me, but I could be wrong.

Alex Crichton (Dec 07 2022 at 20:03):

trying to trace through all this myself

Amit Prasad (Dec 07 2022 at 20:05):

Haha there's a lot to unpack in that repo, but I think for the most part the gist of it is that the println!("Got Request!") line in the guest module should be doing something but seems like it's doing nothing

Alex Crichton (Dec 07 2022 at 20:06):

yeah nothing jumps out here to me

Alex Crichton (Dec 07 2022 at 20:06):

lemme try running and see what happens

Alex Crichton (Dec 07 2022 at 20:07):

I'm seeing

Alex Crichton (Dec 07 2022 at 20:07):

er, I'm seeing

thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: import `testwasi` not defined', crates/host/src/main.rs:70:22
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Alex Crichton (Dec 07 2022 at 20:07):

are you seeing something different?

Amit Prasad (Dec 07 2022 at 20:08):

Ah - are you cloning main or wasi branch -- I've been moving away from a test WASI implementation so main is broken

Alex Crichton (Dec 07 2022 at 20:09):

I'm using main, should I look at something else?

Amit Prasad (Dec 07 2022 at 20:09):

yeah -- WASI branch
https://github.com/AmitPr/wasm-component-serverless/tree/wasi

Amit Prasad (Dec 07 2022 at 20:11):

After cargo run, you should be able to hit curl -D - localhost:3000 from another terminal.

Results in the response you would expect:

HTTP/1.1 200 OK
x-wit-test: true
content-length: 13
date: Wed, 07 Dec 2022 20:08:39 GMT

Hello, world!

But there's no output in the rust server process that I would expect from the print statement that should be triggered by WASI

In essence it's instantiating and running the guest component every time an HTTP request is received

Amit Prasad (Dec 07 2022 at 20:12):

Let me see if I can make a minimum example that has the same issue -- This repo is pretty messy (as you can see ;P)

Alex Crichton (Dec 07 2022 at 20:14):

nah it's ok, I've got it now and something is definitely fishy

Amit Prasad (Dec 07 2022 at 20:15):

Hahaha yeah I was/am extremely confused about the lack of anything. Even putting in panic!() everywhere in the host bindings doesn't seem to do anything.

Alex Crichton (Dec 07 2022 at 20:26):

oh I see the bug

Alex Crichton (Dec 07 2022 at 20:26):

the problem is that the initial state isn't configured

Alex Crichton (Dec 07 2022 at 20:26):

I'll file an issue in a bit

Alex Crichton (Dec 07 2022 at 20:27):

basically the command entrypoint sets up fds 1/2/3 but you're not calling that

Alex Crichton (Dec 07 2022 at 20:27):

and you arguably shouldn't need to call it either

Amit Prasad (Dec 07 2022 at 20:31):

Ah! Interesting. I'll see if I can get around it for now. Thanks for the help :)

Amit Prasad (Dec 07 2022 at 20:32):

Is there any literature / documentation that explains what these command features/entrypoints all refer to?

Amit Prasad (Dec 07 2022 at 20:55):

https://github.com/bytecodealliance/preview2-prototyping/blob/main/src/lib.rs#L41-L53

I assume that I would have to modify & compile the adapter to include these lines in some other location that's called during initialization/instantiation? Would State::new() be inappropriate?

Alex Crichton (Dec 07 2022 at 21:21):

the command was intended for CLI-like executables where it's the main entrypoint

Alex Crichton (Dec 07 2022 at 21:21):

what you sort of need though is some init-style function for non-command things

Alex Crichton (Dec 07 2022 at 21:21):

or... something like that, honestly I didn't see this coming at all so I dunno what the solution is

Alex Crichton (Dec 07 2022 at 22:41):

ok I talked with dan/pat a bit and I think there's a reasonable solution to this which I've now PR'd

Last updated: Nov 22 2024 at 17:03 UTC