Ben Linsay (Jan 23 2024 at 16:59): Ben Linsay (Jan 23 2024 at 16:59):hey all,
i'm messing around and wanted to see if it'd be possible to redefine an existing wasi function. I don't really have a goal in mind, but thought it'd be neat to define my own behavior for a small subset of existing wasi functions. :)
the tutorial and docs made it really easy to get up and running (thanks!) but I'm getting an error about memory exports I don't really understand.
my little snippet so far is this code, shoved into the middle of the example from 2.3.4 in the guide:
let original_env_get: TypedFunc<(i32, i32), i32> = {
let Some(wasmtime::Extern::Func(env_get)) =
linker.get(&mut store, "wasi_snapshot_preview1", "environ_get")
else {
anyhow::bail!("missing environ_get");
};
env_get.typed(&store).unwrap()
};
linker.allow_shadowing(true);
linker.func_wrap(
"wasi_snapshot_preview1",
"environ_get",
move |caller: Caller<'_, _>, a: i32, b: i32| original_env_get.call(caller, (a, b)),
)?;
when I run it with a guest that actually tries to get an env var, I get a backtrace that looks like this. I'm not sure what to make of it - am I accidentally clobbering other parts of the wasi component by shadowing? any tips for reading this error better?
$ cargo run -p host -- ./target/wasm32-wasi/debug/guest.wasm
Compiling host v0.1.0 (/Users/benl/src/hello-wasmtime/host)
Finished dev [unoptimized + debuginfo] target(s) in 0.88s
Running `target/debug/host ./target/wasm32-wasi/debug/guest.wasm`
Error: error while executing at wasm backtrace:
0: 0x9b66 - <unknown>!__wasi_environ_get
1: 0x9b24 - <unknown>!__wasilibc_initialize_environ
2: 0x9a96 - <unknown>!__wasilibc_ensure_environ
3: 0x9c4d - <unknown>!getenv
4: 0x3b2a - <unknown>!std::env::_var_os::ha6d753b2be7fe17f
5: 0x3c99 - <unknown>!std::env::_var::hd7c38fcd0ed4a72d
6: 0x1829 - <unknown>!std::env::var::hd342252c7c9996f3
7: 0x14f4 - <unknown>!guest::main::hb3d545227282da60
8: 0xac4 - <unknown>!core::ops::function::FnOnce::call_once::hdf6e14f7ebde9f53
9: 0xf43 - <unknown>!std::sys_common::backtrace::__rust_begin_short_backtrace::ha23fe57107e63372
10: 0xecb - <unknown>!std::rt::lang_start::{{closure}}::h919e8dda9de0d648
11: 0x3807 - <unknown>!std::rt::lang_start_internal::h409072ad2c29d9a2
12: 0xe68 - <unknown>!std::rt::lang_start::ha5e1690a3bb184c3
13: 0x160e - <unknown>!__main_void
14: 0x2f6 - <unknown>!_start
15: 0x9b66 - <unknown>!__wasi_environ_get
16: 0x9b24 - <unknown>!__wasilibc_initialize_environ
17: 0x9a96 - <unknown>!__wasilibc_ensure_environ
18: 0x9c4d - <unknown>!getenv
19: 0x3b2a - <unknown>!std::env::_var_os::ha6d753b2be7fe17f
20: 0x3c99 - <unknown>!std::env::_var::hd7c38fcd0ed4a72d
21: 0x1829 - <unknown>!std::env::var::hd342252c7c9996f3
22: 0x14f4 - <unknown>!guest::main::hb3d545227282da60
23: 0xac4 - <unknown>!core::ops::function::FnOnce::call_once::hdf6e14f7ebde9f53
24: 0xf43 - <unknown>!std::sys_common::backtrace::__rust_begin_short_backtrace::ha23fe57107e63372
25: 0xecb - <unknown>!std::rt::lang_start::{{closure}}::h919e8dda9de0d648
26: 0x3807 - <unknown>!std::rt::lang_start_internal::h409072ad2c29d9a2
27: 0xe68 - <unknown>!std::rt::lang_start::ha5e1690a3bb184c3
28: 0x160e - <unknown>!__main_void
29: 0x2f6 - <unknown>!_start
note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information
Caused by:
missing required memory export
the original env get function is accessing the wasm module's memory, which is required to be an export named memory
Pat Hickey (Jan 23 2024 at 17:17):so, whatever module you are running needs to provide one of those
Ben Linsay (Jan 23 2024 at 17:20):I think it does?
println!("exports:");
for export in module.exports() {
println!(" {}", export.name());
}
exports:
memory
_start
__main_void
do I need to redeclare that the wrapped function needs memory somehow?
Ben Linsay (Jan 23 2024 at 17:22):my guest program is pretty close to hello world:
use std::env;
fn main() {
let term = env::var("TERM").unwrap_or_else(|_| "unknown".to_string());
println!("hello from wasm guest! your terminal is: {term}");
}
built with cargo build --target wasm32-wasi
Pat Hickey (Jan 23 2024 at 17:22):oh, hmm
Pat Hickey (Jan 23 2024 at 17:23):instead of passing env_get.typed(...) a &mut Store, can you pass it the Caller?
Ben Linsay (Jan 23 2024 at 17:24):oh, generate the typed wrapper on every call?
Ben Linsay (Jan 23 2024 at 17:24):not sure how else to get a Caller
Ben Linsay (Jan 23 2024 at 17:26):looks like the same error/stack trace:
let original_env_get = {
let Some(wasmtime::Extern::Func(env_get)) =
linker.get(&mut store, "wasi_snapshot_preview1", "environ_get")
else {
anyhow::bail!("missing environ_get");
};
env_get
};
linker.allow_shadowing(true);
linker.func_wrap(
"wasi_snapshot_preview1",
"environ_get",
move |mut caller: Caller<'_, _>, a: i32, b: i32| {
let typed: TypedFunc<(i32, i32), i32> = original_env_get.typed(&mut caller).unwrap();
typed.call(caller, (a, b))
},
)?;
yeah what if you make original_env_get bind to jut env_get instead of env_get.typed(store).unwrap
Pat Hickey (Jan 23 2024 at 17:26):hmm
Ben Linsay (Jan 23 2024 at 17:26):$ cargo run -p host -- ./target/wasm32-wasi/debug/guest.wasm
Compiling host v0.1.0 (/Users/benl/src/hello-wasmtime/host)
Finished dev [unoptimized + debuginfo] target(s) in 1.53s
Running `target/debug/host ./target/wasm32-wasi/debug/guest.wasm`
exports:
memory
_start
__main_void
Error: error while executing at wasm backtrace:
0: 0x9b66 - <unknown>!__wasi_environ_get
1: 0x9b24 - <unknown>!__wasilibc_initialize_environ
2: 0x9a96 - <unknown>!__wasilibc_ensure_environ
3: 0x9c4d - <unknown>!getenv
4: 0x3b2a - <unknown>!std::env::_var_os::ha6d753b2be7fe17f
5: 0x3c99 - <unknown>!std::env::_var::hd7c38fcd0ed4a72d
6: 0x1829 - <unknown>!std::env::var::hd342252c7c9996f3
7: 0x14f4 - <unknown>!guest::main::hb3d545227282da60
8: 0xac4 - <unknown>!core::ops::function::FnOnce::call_once::hdf6e14f7ebde9f53
9: 0xf43 - <unknown>!std::sys_common::backtrace::__rust_begin_short_backtrace::ha23fe57107e63372
10: 0xecb - <unknown>!std::rt::lang_start::{{closure}}::h919e8dda9de0d648
11: 0x3807 - <unknown>!std::rt::lang_start_internal::h409072ad2c29d9a2
12: 0xe68 - <unknown>!std::rt::lang_start::ha5e1690a3bb184c3
13: 0x160e - <unknown>!__main_void
14: 0x2f6 - <unknown>!_start
15: 0x9b66 - <unknown>!__wasi_environ_get
16: 0x9b24 - <unknown>!__wasilibc_initialize_environ
17: 0x9a96 - <unknown>!__wasilibc_ensure_environ
18: 0x9c4d - <unknown>!getenv
19: 0x3b2a - <unknown>!std::env::_var_os::ha6d753b2be7fe17f
20: 0x3c99 - <unknown>!std::env::_var::hd7c38fcd0ed4a72d
21: 0x1829 - <unknown>!std::env::var::hd342252c7c9996f3
22: 0x14f4 - <unknown>!guest::main::hb3d545227282da60
23: 0xac4 - <unknown>!core::ops::function::FnOnce::call_once::hdf6e14f7ebde9f53
24: 0xf43 - <unknown>!std::sys_common::backtrace::__rust_begin_short_backtrace::ha23fe57107e63372
25: 0xecb - <unknown>!std::rt::lang_start::{{closure}}::h919e8dda9de0d648
26: 0x3807 - <unknown>!std::rt::lang_start_internal::h409072ad2c29d9a2
27: 0xe68 - <unknown>!std::rt::lang_start::ha5e1690a3bb184c3
28: 0x160e - <unknown>!__main_void
29: 0x2f6 - <unknown>!_start
Caused by:
missing required memory export
just in case you can see something interesting in the stack trace, there it is :)
Pat Hickey (Jan 23 2024 at 17:28):would you mind adding the RUST_BACKTRACE=1? thats the wasm backtrace, which is helpful, but not the native frames
Ben Linsay (Jan 23 2024 at 17:29):Stack backtrace:
0: std::backtrace_rs::backtrace::libunwind::trace
at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
1: std::backtrace_rs::backtrace::trace_unsynchronized
at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
2: std::backtrace::Backtrace::create
at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/backtrace.rs:331:13
3: anyhow::error::<impl anyhow::Error>::msg
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.79/src/error.rs:83:36
4: anyhow::__private::format_err
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.79/src/lib.rs:684:13
5: wasmtime_wasi::sync::snapshots::preview_1::add_wasi_snapshot_preview1_to_linker::{{closure}}::{{closure}}
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-wasi-16.0.0/src/lib.rs:66:9
6: wiggle::run_in_dummy_executor
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wiggle-16.0.0/src/lib.rs:1171:11
7: wasmtime_wasi::sync::snapshots::preview_1::add_wasi_snapshot_preview1_to_linker::{{closure}}
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-wasi-16.0.0/src/lib.rs:66:9
8: <F as wasmtime::func::IntoFunc<T,(wasmtime::func::Caller<T>,A1,A2),R>>::into_func::native_call_shim::{{closure}}::{{closure}}
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-16.0.0/src/func.rs:1974:41
9: core::ops::function::FnOnce::call_once
at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/ops/function.rs:250:5
10: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/panic/unwind_safe.rs:271:9
11: std::panicking::try::do_call
at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/panicking.rs:504:40
12: ___rust_try
13: std::panicking::try
at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/panicking.rs:468:19
14: std::panic::catch_unwind
at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/panic.rs:142:14
15: <F as wasmtime::func::IntoFunc<T,(wasmtime::func::Caller<T>,A1,A2),R>>::into_func::native_call_shim::{{closure}}
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-16.0.0/src/func.rs:1969:29
16: wasmtime::func::Caller<T>::with::{{closure}}
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-16.0.0/src/func.rs:1786:13
17: wasmtime_runtime::instance::Instance::from_vmctx
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-runtime-16.0.0/src/instance.rs:240:9
18: wasmtime::func::Caller<T>::with
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-16.0.0/src/func.rs:1784:9
19: <F as wasmtime::func::IntoFunc<T,(wasmtime::func::Caller<T>,A1,A2),R>>::into_func::native_call_shim
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-16.0.0/src/func.rs:1958:34
20: <(A1,A2) as wasmtime::func::typed::WasmParams>::invoke::{{closure}}
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-16.0.0/src/func/typed.rs:587:21
21: <(A1,) as wasmtime::func::HostAbi>::call
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-16.0.0/src/func.rs:1693:18
22: <(A1,A2) as wasmtime::func::typed::WasmParams>::invoke
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-16.0.0/src/func/typed.rs:586:17
23: wasmtime::func::typed::TypedFunc<Params,Results>::call_raw::{{closure}}
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-16.0.0/src/func/typed.rs:181:17
24: wasmtime_runtime::traphandlers::catch_traps::call_closure
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-runtime-16.0.0/src/traphandlers.rs:241:18
25: wasmtime_setjmp_16_0_0
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-runtime-16.0.0/src/helpers.c:66:3
26: wasmtime_runtime::traphandlers::catch_traps::{{closure}}
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-runtime-16.0.0/src/traphandlers.rs:219:13
27: wasmtime_runtime::traphandlers::<impl wasmtime_runtime::traphandlers::call_thread_state::CallThreadState>::with::{{closure}}
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-runtime-16.0.0/src/traphandlers.rs:353:44
28: wasmtime_runtime::traphandlers::tls::set
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-runtime-16.0.0/src/traphandlers.rs:724:13
29: wasmtime_runtime::traphandlers::<impl wasmtime_runtime::traphandlers::call_thread_state::CallThreadState>::with
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-runtime-16.0.0/src/traphandlers.rs:353:19
30: wasmtime_runtime::traphandlers::catch_traps
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-runtime-16.0.0/src/traphandlers.rs:217:18
31: wasmtime::func::invoke_wasm_and_catch_traps
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-16.0.0/src/func.rs:1371:22
32: wasmtime::func::typed::TypedFunc<Params,Results>::call_raw
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-16.0.0/src/func/typed.rs:177:22
33: wasmtime::func::typed::TypedFunc<Params,Results>::call
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-16.0.0/src/func/typed.rs:88:18
34: host::main::{{closure}}
at ./host/src/main.rs:37:13
35: <F as wasmtime::func::IntoFunc<T,(wasmtime::func::Caller<T>,A1,A2),R>>::into_func::native_call_shim::{{closure}}::{{closure}}
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-16.0.0/src/func.rs:1974:41
36: core::ops::function::FnOnce::call_once
at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/ops/function.rs:250:5
37: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/panic/unwind_safe.rs:271:9
38: std::panicking::try::do_call
at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/panicking.rs:504:40
39: ___rust_try
40: std::panicking::try
at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/panicking.rs:468:19
41: std::panic::catch_unwind
at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/panic.rs:142:14
42: <F as wasmtime::func::IntoFunc<T,(wasmtime::func::Caller<T>,A1,A2),R>>::into_func::native_call_shim::{{closure}}
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-16.0.0/src/func.rs:1969:29
43: wasmtime::func::Caller<T>::with::{{closure}}
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-16.0.0/src/func.rs:1786:13
44: wasmtime_runtime::instance::Instance::from_vmctx
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-runtime-16.0.0/src/instance.rs:240:9
45: wasmtime::func::Caller<T>::with
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-16.0.0/src/func.rs:1784:9
46: <F as wasmtime::func::IntoFunc<T,(wasmtime::func::Caller<T>,A1,A2),R>>::into_func::native_call_shim
at /Users/benl/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wasmtime-16.0.0/src/func.rs:1958:34
47: <unknown>
hmm, yeah, thats where id expect it to come from, so im stumped at the moment and i need to run. ill see if i can get back to this later
Ben Linsay (Jan 23 2024 at 17:30):appreciate it! thanks!
Ben Linsay (Jan 25 2024 at 19:07):@Pat Hickey hey, have you had a chance to take another look? no rush, and if you want to just drop it, nbd :)
Pat Hickey (Jan 26 2024 at 01:51):sorry, been busy with getting preview 2 out the door. if you can upload a reasonably small reproduction, i'll take a look tomorrow
Ben Linsay (Jan 26 2024 at 02:23):congrats and no worries!
here's a gist: https://gist.github.com/blinsay/e9571a15fc1f59a355858cdba307946f
Pat Hickey (Jan 29 2024 at 19:37):hi, i finally got time to look at the reproduction, thanks for waiting
Pat Hickey (Jan 29 2024 at 19:38):i believe what is going on here is that you have called a method in your linker without first creating an instance of the module
Pat Hickey (Jan 29 2024 at 19:39):or creating any instance at all, basically
Pat Hickey (Jan 29 2024 at 19:40):you still need to call linker.instantiate somewhere
Pat Hickey (Jan 29 2024 at 19:40):otherwise, wasmtime tries to be helpful about calling Funcs from the "host context" where there is essentially just an empty instance
Pat Hickey (Jan 29 2024 at 19:40):and that empty instance doesnt have a memory
Pat Hickey (Jan 29 2024 at 19:46):actually, nevermind that explanation, I changed it to how it "should" work and its still broken
Pat Hickey (Jan 29 2024 at 19:46):which was, for reference,
// shadowing stuff ends
let module = Module::from_file(&engine, prog_path)?;
let inst = linker.instantiate(&mut store, &module)?;
inst.get_func(&mut store, "_start")
.ok_or_else(|| anyhow::anyhow!("_start export required"))?
.typed::<(), ()>(&store)?
.call(&mut store, ())?;
Ok(())
}
@Alex Crichton can you help me understand why this line uses the closure captured vmctx and not one belonging to the caller https://github.com/bytecodealliance/wasmtime/blob/ab5a4484ebac8d1f08f773d244baedb09b90a29b/crates/wasmtime/src/func.rs#L1959
Pat Hickey (Jan 29 2024 at 19:53):im trying to reason about why there are two different host states for those vmctxs
Pat Hickey (Jan 29 2024 at 19:55):basically, the behavior of the host-to-host call of func isnt retaining the exports id expect to be present in the store
Pat Hickey (Jan 29 2024 at 20:01):@Ben Linsay from the above questions to alex i am actually pretty stumped on whats causing this behavior. maybe alex will be able to help me understand whats up here
Alex Crichton (Jan 29 2024 at 20:04):hm the answer to that is basically "it'll be memory unsafe if we didn't" in the sense that each vmctx has slightly different host state, and the closure-captured one there has the closure specified to the linker as its state and that's what's being loaded and called. The caller's host_state I don't think corresponds to T in Store<T> and most of this level of detail is divorced from the overall embedding API.
Alex Crichton (Jan 29 2024 at 20:04):so I might need to understand more what's going on here to answer better
Alex Crichton (Jan 29 2024 at 20:17):oh I think I see what you're getting at, and this is indeed confusing to me as well. I'll need to puzzle this out a bit
Alex Crichton (Jan 29 2024 at 20:20):So I think it makes sense that this doesn't work right now. I think it's fine to consider that a bug, however.
The problem is that this has to do with how WASI finds the exported memory. When you work with WASI as it's set up by default with preview1 you never actually configure what memory to use, so each function has to look up on the caller what the memory export is. In this case though the "caller" is Rust host code itself, which means the "caller" doesn't have a memory export
Alex Crichton (Jan 29 2024 at 20:21):right now there's no way to fake "as if this was called by wasm", so I don't think that there's a way around this right now unfortunately.
Pat Hickey (Jan 29 2024 at 20:35):Yeah I thought the Caller would transfer that information but it is more subtle than that
Pat Hickey (Jan 29 2024 at 20:35):Anyway, ok, thanks for your help Alex
Pat Hickey (Jan 29 2024 at 20:36):@Ben Linsay I bet if you were using components and wasi preview 2 you wouldn’t have this problem. Want to try that instead? The distinction of being called from wasm goes away with components, and in general a lot of the complexity of an implementation of get environment is abstracted away
Pat Hickey (Jan 29 2024 at 20:38):If you do indeed need the wasi-common preview 1 implementation we could get you dispatching to the wiggle trait function in your func_wrap, which won’t have the same problems with caller as getting it from the linker
Pat Hickey (Jan 29 2024 at 20:39):But, I’m biased but if I had to recommend anything to authors of new systems, just use components! They solve tons of problems you may not even know you have yet :)
Ben Linsay (Jan 30 2024 at 14:18):thanks for taking a look!
Ben Linsay (Jan 30 2024 at 14:20):sure, components sound great. where do I find docs/examples?
Alex Crichton (Jan 30 2024 at 14:39):I'd recommend starting at https://component-model.bytecodealliance.org/ for components
Ben Linsay (Jan 30 2024 at 15:57):ty!
just took a quick skim and it's not immediately obvious to me how i'd use this to change the behavior of a specific wasi function. would I have to re-declare the entire wasi component and forward everything else along?
Pat Hickey (Jan 30 2024 at 16:01):There’s a lot of ways. You could do it entirely with component composition - make a component that imports get-environment and exports the same, and use wasm-tools compose to interpose it
Pat Hickey (Jan 30 2024 at 16:01):You can also use the host, like you were trying before, but with a wasmtime::component::Linker
Ben Linsay (Jan 30 2024 at 16:03):oh interesting
Pat Hickey (Jan 30 2024 at 16:03):If you do that you should consider using wasmtime::component::bindgen! to generate the bindings for the cli environment interface, and then add it to the linker
Ben Linsay (Jan 30 2024 at 16:03):which way would you recommend?
Pat Hickey (Jan 30 2024 at 16:04):Start with just the host, it’s easier I thibk
Ben Linsay (Jan 30 2024 at 16:04):that checks out
Ben Linsay (Jan 30 2024 at 16:05):do I still grab a reference to environ_get the same way I was doing before?
Ben Linsay (Jan 30 2024 at 16:06):or is that going to come through the bingen! thing you're suggesting?
Pat Hickey (Jan 30 2024 at 16:07):No, I don’t think so, you’ll call it by rust symbol instead of getting it from the Linker in components. We don’t have allow_shadowing on the component::Linker to my knowledge so, until we get around to adding it, copy the implementation of wasmtime_wasi::preview2::command::add_to_linker and then substitute just your cli::environment add_to_linker in there
Ben Linsay (Jan 30 2024 at 16:07):cool, will sit down this afternoon and give it a whirl. thanks dude!
Ben Linsay (Feb 03 2024 at 00:25):came back to this a few days later than I meant to, and I'm still a bit lost. I read through the docs for the bindgen! macro (which are nice and detailed, ty!) and tried to copy what was going on there just to get started.
I got bindings by copying the wit directory from wasi-cli into a local directory and I can get the bindgen macro making bindings for it with:
bindgen!({
world: "imports"
});
That seems really reasonable, and I can struct MyEnv; impl Host for MyEnv {...} now, which is great, but I have no idea how to find the symbol for the original implementation of get-environment. Any suggestions on where to find it?
I did go digging into the docs for wasmtime-wasi and found out that the Host trait is already generated there. Do I need to do the bindgen dance at all?
Ben Linsay (Feb 03 2024 at 00:27):I did notice that component::Linker has an allow_shadowing method btw :)
https://docs.rs/wasmtime/latest/wasmtime/component/struct.Linker.html#method.allow_shadowing
Ben Linsay (Feb 03 2024 at 00:28):kinda dead-ended on the host approach right now. I'll go try the component composition approach later this weekend too.
Alex Crichton (Feb 03 2024 at 18:02):If your goal is to override a small handful of WASI methods then you can probably skip the bindgen yeah. All the types you need are in the wasmtime-wasi crate and with allow_shadowing you can basically redefine an existing wasi function with your own custom closure (using the same signature as before). Accessing the previous implementation can be done by calling the Host trait method directly (e.g. wasmtime_wasi::preview2::bindings::wasi::cli::environment::Host::get_environment(...))
Ben Linsay (Feb 03 2024 at 20:38):@Alex Crichton dumb question, what do I pass as self if I call Host methods directly? I don't see anything that actually implements the trait
Alex Crichton (Feb 03 2024 at 21:11):The self argument is anything that implements the WasiView trait which is typically the T in Store<T>
Ben Linsay (Feb 03 2024 at 22:56):okay, I'm still completely lost. I'm trying to mirror what I had originally but I haven't gotten anything involving wasmtime::component to compile. :(
Definitely feels like I'd benefit from doing the basic stuff first, so I'm going to go play with components and composing them before coming back to this. Thanks for all your help so far.
Pat Hickey (Feb 05 2024 at 18:49):our tutorials and docs for this whole category are not as polished as they need to be, but we're going to work on it :)
Pat Hickey (Feb 05 2024 at 18:50):https://component-model.bytecodealliance.org/ is a great start. please do report here (or file at https://github.com/bytecodealliance/component-docs) if anything in there isnt correct / doesnt work
Ben Linsay (Feb 06 2024 at 17:03):the docs definitely got me started! I managed to understand what a component was, run through the examples, and get my own host program running a command component fairly easily. thanks :D
I'm back to trying to shadow get-environment. I can't figure out how to inspect the linker/components to see if I've actually gotten the right instance and redefined the right thing. any tips on where to go from here?
use anyhow::Context;
use std::env;
use wasmtime::component::Component;
use wasmtime::{Config, Engine, Store};
use wasmtime_wasi::preview2;
fn main() -> anyhow::Result<()> {
let path = env::args().skip(1).next().unwrap();
eprintln!("starting");
let mut config = Config::default();
config.wasm_component_model(true);
let engine = Engine::new(&config)?;
let wasi_view = ServerWasiView::new();
let mut store = Store::new(&engine, wasi_view);
let mut linker: wasmtime::component::Linker<ServerWasiView> =
wasmtime::component::Linker::new(&engine);
preview2::command::sync::add_to_linker(&mut linker).context("failed to link command world")?;
// --- shadowing stuff: nothing here seems to get called ----
linker.allow_shadowing(true);
let mut instance = linker
.instance("wasi/cli:environment")
.context("getting the wasi/cli:environment instance")?;
instance
.func_wrap("get-environment", |mut ctx, ()| {
eprintln!("called from the host wrapper");
// NOTE: this is terribly janky
let env = wasmtime_wasi::preview2::bindings::cli::environment::Host::get_environment(
ctx.data_mut(),
)?;
Ok(env.first().unwrap().clone())
})
.context("overriding get-environment")?;
// --- end shadowing stuff
eprintln!("loading component");
let component = Component::from_file(&engine, path).context("loading component failed")?;
let (instance, _) = wasmtime_wasi::preview2::command::sync::Command::instantiate(
&mut store,
&component,
&mut linker,
)
.context("failed to instantiate Command")?;
let res = instance.wasi_cli_run().call_run(&mut store);
println!("run -> {res:?}");
Ok(())
}
syntax is a little off and you'll need to add the version number to the package- so wasi:cli@0.2.0/environment
Pat Hickey (Feb 06 2024 at 17:59):we could add a getter to linker that fails if the thing isnt there
Ben Linsay (Feb 06 2024 at 18:14):ahhhh that'd be really nice :)
Ben Linsay (Feb 06 2024 at 18:15):it was really nice to be able to list imports/exports with modules. i don't see a way to do that with the component APIs. am I missing something or do those not exist (yet)?
Pat Hickey (Feb 06 2024 at 18:22):nope, doesnt exist
Pat Hickey (Feb 06 2024 at 18:25):agree that it should, we can add it
Last updated: Nov 22 2024 at 17:03 UTC