Stream: git-wasmtime

Topic: wasmtime / issue #7697 unknown import: `wasi_snapshot_pre...

view this post on Zulip Wasmtime GitHub notifications bot (Dec 17 2023 at 07:44):

k82cn added the bug label to Issue #7697.

view this post on Zulip Wasmtime GitHub notifications bot (Dec 17 2023 at 07:44):

k82cn opened issue #7697:

Test Case

Refer to following files for rust & wasm files:

Local: https://github.com/xflops/flame/pull/75/files#diff-448546749b6625cbca1a43c400f6d86637db5f014968c96ece09705e07f65c7c
WASM: https://github.com/xflops/flame/pull/75/files#diff-cb3f76ea005ef90eb514c6baf71a09bd928befb8f267ef204ea8474036dc172b

Steps to Reproduce

Expected Results

WASM can execute as normal

Actual Results

$ ./target/debug/matrix-local
Error: NotFound("unknown import: `wasi_snapshot_preview1::fd_write` has not been defined")

Versions and Environment

Wasmtime version or commit: 15.0.1

Operating system: ubuntu 22.04

Architecture: x86

Extra Info

And I'd like to pass through a string and binary data into WASM, e.g. on_session_enter, on_session_leave and on_task_invoke.

view this post on Zulip Wasmtime GitHub notifications bot (Dec 17 2023 at 08:17):

bjorn3 commented on issue #7697:

You are missing this call to enable wasi support: https://github.com/bytecodealliance/wasmtime/blob/54d3727ac3df53e55489560656b1d68db3c0e3ba/examples/wasi/main.rs#L17 Make sure to also add a WasiCtx to the Store, eg: https://github.com/bytecodealliance/wasmtime/blob/54d3727ac3df53e55489560656b1d68db3c0e3ba/examples/wasi/main.rs#L26 It can also be a field of the store data, in which case you need to update the closure in the add_to_linker call to return a reference to this field.

view this post on Zulip Wasmtime GitHub notifications bot (Dec 17 2023 at 10:09):

k82cn commented on issue #7697:

Update PR to include them: https://github.com/xflops/flame/pull/75/commits/ac2482e06c7d29c96a0085e6af1823bc777aa182

But I got the following error when instance.get_func in on_session_enter:

$ ./target/debug/matrix-local
thread 'main' panicked at '`main` was not an exported function', examples/matrix/local/src/main.rs:53:14
stack backtrace:
   0: rust_begin_unwind
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/panicking.rs:593:5
   1: core::panicking::panic_fmt
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/panicking.rs:67:14
   2: core::panicking::panic_display
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/panicking.rs:150:5
   3: core::panicking::panic_str
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/panicking.rs:134:5
   4: core::option::expect_failed
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/option.rs:1932:5
   5: core::option::Option<T>::expect
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/option.rs:898:21
   6: matrix_local::WasmShim::on_session_enter::{{closure}}
             at ./examples/matrix/local/src/main.rs:50:22
   7: matrix_local::main::{{closure}}
             at ./examples/matrix/local/src/main.rs:77:40
   8: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/park.rs:282:63
   9: tokio::runtime::coop::with_budget
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/coop.rs:107:5
  10: tokio::runtime::coop::budget
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/coop.rs:73:5
  11: tokio::runtime::park::CachedParkThread::block_on
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/park.rs:282:31
  12: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/context/blocking.rs:66:9
  13: tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/scheduler/multi_thread/mod.rs:87:13
  14: tokio::runtime::context::runtime::enter_runtime
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/context/runtime.rs:65:16
  15: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/scheduler/multi_thread/mod.rs:86:9
  16: tokio::runtime::runtime::Runtime::block_on
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/runtime.rs:350:45
  17: matrix_local::main
             at ./examples/matrix/local/src/main.rs:91:5
  18: core::ops::function::FnOnce::call_once
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

view this post on Zulip Wasmtime GitHub notifications bot (Dec 17 2023 at 10:09):

k82cn edited a comment on issue #7697:

Update PR to include them: https://github.com/xflops/flame/pull/75/commits/ac2482e06c7d29c96a0085e6af1823bc777aa182

But I got the following error when instance.get_func in on_session_enter:

$ ./target/debug/matrix-local
thread 'main' panicked at '`main` was not an exported function', examples/matrix/local/src/main.rs:53:14
stack backtrace:
   0: rust_begin_unwind
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/panicking.rs:593:5
   1: core::panicking::panic_fmt
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/panicking.rs:67:14
   2: core::panicking::panic_display
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/panicking.rs:150:5
   3: core::panicking::panic_str
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/panicking.rs:134:5
   4: core::option::expect_failed
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/option.rs:1932:5
   5: core::option::Option<T>::expect
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/option.rs:898:21
   6: matrix_local::WasmShim::on_session_enter::{{closure}}
             at ./examples/matrix/local/src/main.rs:50:22
   7: matrix_local::main::{{closure}}
             at ./examples/matrix/local/src/main.rs:77:40
   8: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/park.rs:282:63
   9: tokio::runtime::coop::with_budget
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/coop.rs:107:5
  10: tokio::runtime::coop::budget
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/coop.rs:73:5
  11: tokio::runtime::park::CachedParkThread::block_on
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/park.rs:282:31
  12: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/context/blocking.rs:66:9
  13: tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/scheduler/multi_thread/mod.rs:87:13
  14: tokio::runtime::context::runtime::enter_runtime
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/context/runtime.rs:65:16
  15: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/scheduler/multi_thread/mod.rs:86:9
  16: tokio::runtime::runtime::Runtime::block_on
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/runtime.rs:350:45
  17: matrix_local::main
             at ./examples/matrix/local/src/main.rs:91:5
  18: core::ops::function::FnOnce::call_once
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

If any suggestions, please let me know.

view this post on Zulip Wasmtime GitHub notifications bot (Dec 17 2023 at 10:17):

bjorn3 commented on issue #7697:

The example uses

linker.module(&mut store, "", &module)?;
linker
    .get_default(&mut store, "")?
    .typed::<(), ()>(&store)?
    .call(&mut store, ())?;

If you want to keep the structure of the code unchanged I think you can use self.instance.get_func(&mut self.store, "_start"). For wasi the actual function exported by the wasm module is _start. Internally this initialized libc and the rust standard library before calling the main function you wrote in the rust source file.

view this post on Zulip Wasmtime GitHub notifications bot (Dec 17 2023 at 10:29):

k82cn commented on issue #7697:

self.instance.get_func(&mut self.store, "_start")

Thanks very much! That works for the main. Is there anyway to also call on_session_enter function? Honestly, I'd like to execute three callback funcs in wasm: on_session_enter, on_session_leave and on_task_invoke.

When replace _start with on_session_enter (which is a pub fn in main.rs), I got following errors:

$ ./target/debug/matrix-local
thread 'main' panicked at '`on_session_enter` was not an exported function', examples/matrix/local/src/main.rs:53:14
stack backtrace:
   0: rust_begin_unwind
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/panicking.rs:593:5
   1: core::panicking::panic_fmt
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/panicking.rs:67:14
   2: core::panicking::panic_display
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/panicking.rs:150:5
   3: core::panicking::panic_str
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/panicking.rs:134:5
   4: core::option::expect_failed
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/option.rs:1932:5
   5: core::option::Option<T>::expect
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/option.rs:898:21
   6: matrix_local::WasmShim::on_session_enter::{{closure}}
             at ./examples/matrix/local/src/main.rs:50:23
   7: matrix_local::main::{{closure}}
             at ./examples/matrix/local/src/main.rs:79:40
   8: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/park.rs:282:63
   9: tokio::runtime::coop::with_budget
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/coop.rs:107:5
  10: tokio::runtime::coop::budget
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/coop.rs:73:5
  11: tokio::runtime::park::CachedParkThread::block_on
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/park.rs:282:31
  12: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/context/blocking.rs:66:9
  13: tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/scheduler/multi_thread/mod.rs:87:13
  14: tokio::runtime::context::runtime::enter_runtime
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/context/runtime.rs:65:16
  15: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/scheduler/multi_thread/mod.rs:86:9
  16: tokio::runtime::runtime::Runtime::block_on
             at /hpc/mtr_scrap/users/klausm/cargo_home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/runtime.rs:350:45
  17: matrix_local::main
             at ./examples/matrix/local/src/main.rs:93:5
  18: core::ops::function::FnOnce::call_once
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

view this post on Zulip Wasmtime GitHub notifications bot (Dec 17 2023 at 10:59):

bjorn3 commented on issue #7697:

Have you defined these functions with #[no_mangle]? If not they will get a mangled name and will not be exported. Also I think you need to compile as cdylib instead of as bin if you want to export custom functions. For that you need to rename src/main.rs to src/lib.rs and in Cargo.toml add:

[lib]
crate-type = ["cdylib"]

view this post on Zulip Wasmtime GitHub notifications bot (Dec 17 2023 at 13:00):

k82cn commented on issue #7697:

That works to me, thanks very much!

In addition, do you have any suggestions on sending string, bytes array and string array? Something like that:

pub struct SessionContext {
    pub session_id: String,
    pub common_data: Option<bytes::Bytes>,
}

pub type TaskInput = bytes::Bytes;
pub type TaskOutput = bytes::Bytes;

pub struct TaskContext {
    pub session_id: String,
    pub task_id: String,
}

pub trait FlameService {
    fn on_session_enter(&self, ctx: SessionContext) -> Result<(), FlameError>;
    fn on_session_leave(&self, ctx: SessionContext) -> Result<(), FlameError>;

    fn on_task_invoke(
        &self,
        ctx: TaskContext,
        input: Option<TaskInput>,
    ) -> Result<Option<TaskOutput>, FlameError>;
}

view this post on Zulip Wasmtime GitHub notifications bot (Dec 17 2023 at 13:00):

k82cn edited a comment on issue #7697:

That works to me, thanks very much!

In addition, do you have any suggestions on sending/receiving string, bytes array and string array? Something like that:

pub struct SessionContext {
    pub session_id: String,
    pub common_data: Option<bytes::Bytes>,
}

pub type TaskInput = bytes::Bytes;
pub type TaskOutput = bytes::Bytes;

pub struct TaskContext {
    pub session_id: String,
    pub task_id: String,
}

pub trait FlameService {
    fn on_session_enter(&self, ctx: SessionContext) -> Result<(), FlameError>;
    fn on_session_leave(&self, ctx: SessionContext) -> Result<(), FlameError>;

    fn on_task_invoke(
        &self,
        ctx: TaskContext,
        input: Option<TaskInput>,
    ) -> Result<Option<TaskOutput>, FlameError>;
}

view this post on Zulip Wasmtime GitHub notifications bot (Dec 17 2023 at 13:05):

bjorn3 commented on issue #7697:

For that you may want to switch to using the component model together with wasi-preview2 (wasi-preview1 only works with core wasm, wasi-preview2 only with the component model) and then define your own interface. I have never used it myself, but I think the docs at https://docs.rs/wasmtime/latest/wasmtime/component/macro.bindgen.html should be clear.

view this post on Zulip Wasmtime GitHub notifications bot (Dec 18 2023 at 01:48):

k82cn closed issue #7697:

Test Case

Refer to following files for rust & wasm files:

Local: https://github.com/xflops/flame/pull/75/files#diff-448546749b6625cbca1a43c400f6d86637db5f014968c96ece09705e07f65c7c
WASM: https://github.com/xflops/flame/pull/75/files#diff-cb3f76ea005ef90eb514c6baf71a09bd928befb8f267ef204ea8474036dc172b

Steps to Reproduce

Expected Results

WASM can execute as normal

Actual Results

$ ./target/debug/matrix-local
Error: NotFound("unknown import: `wasi_snapshot_preview1::fd_write` has not been defined")

Versions and Environment

Wasmtime version or commit: 15.0.1

Operating system: ubuntu 22.04

Architecture: x86

Extra Info

And I'd like to pass through a string and binary data into WASM, e.g. on_session_enter, on_session_leave and on_task_invoke.

view this post on Zulip Wasmtime GitHub notifications bot (Dec 18 2023 at 01:48):

k82cn commented on issue #7697:

Re wasi-preview2: thanks very much, I'm going to have a try.

view this post on Zulip Wasmtime GitHub notifications bot (May 01 2024 at 15:21):

maxwellflitton commented on issue #7697:

@bjorn3 the links you have given no longer work. There seems to be a new add_to_linker function below:
https://docs.rs/wasmtime-wasi/latest/wasmtime_wasi/preview1/wasi_snapshot_preview1/fn.add_to_linker.html
but I can't see a shred of code that shows how to use this.

view this post on Zulip Wasmtime GitHub notifications bot (May 01 2024 at 15:24):

maxwellflitton deleted a comment on issue #7697:

@bjorn3 the links you have given no longer work. There seems to be a new add_to_linker function below:
https://docs.rs/wasmtime-wasi/latest/wasmtime_wasi/preview1/wasi_snapshot_preview1/fn.add_to_linker.html
but I can't see a shred of code that shows how to use this.

view this post on Zulip Wasmtime GitHub notifications bot (May 01 2024 at 15:33):

maxwellflitton commented on issue #7697:

@bjorn3 There's been some changes in the API, however, I'm getting the same error. My Cargo.toml has the following:

wasm-bindgen = { version = "0.2", features = ["serde-serialize"] }
wasm-bindgen-futures = "0.4.42"
bincode = "1.3.3"
tokio = { version = "1.0", features = ["full"] }
wasmtime = { version = "20.0.0", features = ["runtime", "async"] }
wasmtime-wasi = { version = "20.0.0", features = ["preview1"] }

And my code consists of the following:

use wasmtime::*;
use wasmtime_wasi::WasiCtxBuilder;
use wasmtime_wasi::preview1::wasi_snapshot_preview1::add_to_linker;

#[tokio::main]
async fn main() {
    let mut config = Config::new();
    config.async_support(true);
    let engine = Engine::new(&config).unwrap();

    let mut linker = Linker::new(&engine);

    let wasi = WasiCtxBuilder::new()
        .inherit_stdio()
        .inherit_args()
        .build_p1();
    let mut store = Store::new(&engine, wasi);

    add_to_linker(&mut linker, |s| s).unwrap();

    let module = Module::from_file(&engine, "../wasi-server/wasi_server.wasm").unwrap();
    let link = linker.instantiate_async(&mut store, &module).await.unwrap();
}

But I get the following error on the let link = linker.instantiate_async(&mut store, &module).await.unwrap();:

called `Result::unwrap()` on an `Err` value: unknown import: `wasi_snapshot_preview1::fd_write` has not been defined
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

view this post on Zulip Wasmtime GitHub notifications bot (May 02 2024 at 17:53):

alexcrichton commented on issue #7697:

@maxwellflitton you might be interested in the documentation to get started here, namely you're missing a call to add_to_linker_async

Last updated: Nov 22 2024 at 16:03 UTC