Richard Backhouse (Jul 23 2023 at 12:19): Richard Backhouse (Jul 23 2023 at 12:19):I'm having trouble setting up preopened dirs for go and c based components but not for a Rust based one. When they try and access file resources they get failures indicating that the capability has not actually been authorized.
For the go component I get back "errno 76" "capabilities insufficient"
Fof the c component I get back ENOENT No such file or directory
What's weird is that for the Rust based component the access is authorized fine.
This suggests that the problem lies with how the components are being built. The only different in building between the c/go components and the rust one is the additional "wasm-tools component embed" steps to add the world metadata.
I'm using
wasmtime ver 11.0.0
wasm-tools ver 1.0.37
wit-bindgen ver 0.9.0
Any thoughts one what the problem might be ?
Thanks
Pat Hickey (Jul 24 2023 at 03:03):i need more information to help debug. what is the wasm-tools component embed being used for? are you running this with the component adapter and preview2 impl (is it wasmtime_wasi::preview2::WasiCtxBuilder? or just wasmtime_wasi::WasiCtxBuilder?
Pat Hickey (Jul 24 2023 at 03:04):also, please share the rust and go toolchains used to compile these
Richard Backhouse (Jul 24 2023 at 09:15):I'm using wasmtime_wasi::preview2::WasiCtxBuilder
I'm using the wasm-tools component embed to add the world metadata to the wasm binaries for the imports and exports I have
The go toolchain is tinygo is 0.28.1
The rust toolchain is rustc 1.71.0 (8ede3aae2 2023-07-12)
Pat Hickey (Jul 24 2023 at 15:48):thanks. last question is which version of the wasi-preview1-component-adapter (i assume also from wasmtime 11.0.0, right?), then, with wasmtime running inside an executable with a tracing_subscriber env filter - can you set RUST_LOG=wasmtime_wasi=trace and collect those traces, that will let us see how each different implementation called into wasi, so we should be able to see why wasi returned the error it did
Richard Backhouse (Jul 24 2023 at 15:51):Yes the 11.0.0 version of wasi-preview1-component-adapter from wastime repo
I'll give the tracing a go. Thanks
Richard Backhouse (Jul 24 2023 at 17:09):Here is the trace for the Rust component writing to a file:
2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::cli_base::preopens] wit-bindgen import; module="preopens" function="get-directories"
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::cli_base::preopens] call
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::cli_base::preopens] return result=Ok([(6, "/")])
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::filesystem::filesystem] wit-bindgen import; module="filesystem" function="get-type"
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::filesystem::filesystem] call this=6
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::filesystem::filesystem] return result=Ok(DescriptorType::Directory)
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::filesystem::filesystem] wit-bindgen import; module="filesystem" function="open-at"
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::filesystem::filesystem] call this=6 path_flags=(SYMLINK_FOLLOW) path="test.log" open_flags=(CREATE|TRUNCATE) flags=(WRITE) modes=(READABLE|WRITABLE)
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::filesystem::filesystem] return result=Ok(7)
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::filesystem::filesystem] wit-bindgen import; module="filesystem" function="get-type"
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::filesystem::filesystem] call this=7
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::filesystem::filesystem] return result=Ok(DescriptorType::RegularFile)
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::filesystem::filesystem] wit-bindgen import; module="filesystem" function="write-via-stream"
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::filesystem::filesystem] call this=7 offset=0
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::filesystem::filesystem] return result=Ok(8)
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::io::streams] wit-bindgen import; module="streams" function="blocking-write"
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::io::streams] call this=8 buf=[83, 111, 109, 101, 32, 100, 97, 116, 97, 33]
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::io::streams] return result=Ok(10)
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::io::streams] wit-bindgen import; module="streams" function="drop-output-stream"
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::io::streams] call this=8
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::io::streams] return result=Ok(())
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::filesystem::filesystem] wit-bindgen import; module="filesystem" function="drop-descriptor"
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::filesystem::filesystem] call this=7
[2023-07-24T17:03:41Z TRACE wasmtime_wasi::preview2::wasi::wasi::filesystem::filesystem] return result=Ok(())
Here is the trace for the Go component using "os.Create" to open the file:
2023-07-24T17:03:56Z TRACE wasmtime_wasi::preview2::wasi::wasi::cli_base::preopens] wit-bindgen import; module="preopens" function="get-directories"
[2023-07-24T17:03:56Z TRACE wasmtime_wasi::preview2::wasi::wasi::cli_base::preopens] call
[2023-07-24T17:03:56Z TRACE wasmtime_wasi::preview2::wasi::wasi::cli_base::preopens] return result=Ok([(6, "/")])
[2023-07-24T17:03:56Z TRACE wasmtime_wasi::preview2::wasi::wasi::filesystem::filesystem] wit-bindgen import; module="filesystem" function="get-type"
[2023-07-24T17:03:56Z TRACE wasmtime_wasi::preview2::wasi::wasi::filesystem::filesystem] call this=6
[2023-07-24T17:03:56Z TRACE wasmtime_wasi::preview2::wasi::wasi::filesystem::filesystem] return result=Ok(DescriptorType::Directory)
[2023-07-24T17:03:56Z TRACE wasmtime_wasi::preview2::wasi::wasi::io::streams] wit-bindgen import; module="streams" function="write"
[2023-07-24T17:03:56Z TRACE wasmtime_wasi::preview2::wasi::wasi::io::streams] call this=4 buf=[70, 97, 105, 108, 101, 100, 32, 116, 111, 32, 111, 112, 101, 110, 32, 102, 105, 108, 101, 32, 111, 112, 101, 110, 32, 116, 101, 115, 116, 46, 108, 111, 103, 58, 32, 101, 114, 114, 110, 111, 32, 55, 54, 10]
Failed to open file open test.log: errno 76
[2023-07-24T17:03:56Z TRACE wasmtime_wasi::preview2::wasi::wasi::io::streams] return result=Ok(44)
Pat Hickey (Jul 24 2023 at 17:26):whatever go is doing wrong there isnt an issue with wasi preview 2- you can see that rust is calling "open-at" on fd 6 (the / preopen), with path "test.log" and the create oflag. for go to create a test.log file it would need to make an open-at call as well
Pat Hickey (Jul 24 2023 at 17:28):if your go program is calling the preview 1 path_open func in the adapter, the only reason it would get rejected before turning into a preview 2 open-at call is if it passed an invalid fd. since all of that happens in wasm, not in the host, we cant see it with this tracing.
Pat Hickey (Jul 24 2023 at 17:30):@Damian Gryski is my team's tinygo expert - maybe he has seen this sort of problem before and can help you know where to debug next? unfortunately I don't know anything about how tinygo hooks up to wasi.
Damian Gryski (Jul 24 2023 at 17:42):TinyGo uses wasi-libc under the hood, and our os package just calls those as standard libc functions -- we shouldn't be doing anything weird (at least not by default ... ).
Pat Hickey (Jul 24 2023 at 17:45):oh, theres wasi-libc under there. that could explain it. sit tight
Pat Hickey (Jul 24 2023 at 17:45):i was not expecting that :)
Damian Gryski (Jul 24 2023 at 17:47):Our vendor directory: https://github.com/tinygo-org/tinygo/tree/release/lib
Pat Hickey (Jul 24 2023 at 17:48):ok, the preview1-component-adapter has to lie about rights in the same manner as https://github.com/bytecodealliance/wasmtime/pull/6463 did
Pat Hickey (Jul 24 2023 at 17:48):ill get that fix in sometime this week
Damian Gryski (Jul 24 2023 at 17:49):The new wasip1 Go port does everything by hand, but tinygo went the easy route and offloaded everything to wasi-libc.
Richard Backhouse (Jul 24 2023 at 17:54):Thank you both for your help
Richard Backhouse (Aug 23 2023 at 09:30):I tried this out in wasmtime 12.0.0 (I'm assuming the fix went into this release) and am seeing the same failure. wasmtime_wasi=trace shows the same details as before.
Pat Hickey (Aug 23 2023 at 16:32):the change you need isnt in wasmtime, but in the wasi preview1 component adapter
Richard Backhouse (Aug 23 2023 at 16:55):You mean the wasi_snapshot_preview1.command.wasm and wasi_snapshot_preview1.reactor.wasm from the release right ? I updated both of those too and rebuilt the component with them
Richard Backhouse (Aug 25 2023 at 10:44):Any suggestions on debugging this further? I also tried with my own build of wasi_snapshot_preview1.wasm from the wasmtime 12.0.0 source with the same results
Pat Hickey (Aug 25 2023 at 16:06):sorry, I forgot to get back to this. Can you please file an issue on wasmtime with instructions on how to reproduce? providing the core wasm module created by tinygo
Richard Backhouse (Aug 25 2023 at 16:29):Ok wil do
Pat Hickey (Aug 25 2023 at 16:44):the wasmtime cli (in main) has support for running components directly, if that helps
Pat Hickey (Aug 25 2023 at 16:44):i dont expect theres any difference in this code between 12.0 and main.
Richard Backhouse (Aug 26 2023 at 12:56):I opened https://github.com/bytecodealliance/wasmtime/issues/6914 for this. It includes a testcase repo I put together
Richard Backhouse (Aug 26 2023 at 13:07):On a side note I see that Tinygo has released version 0.29 which provides support for GOOS=wasip1
Richard Backhouse (Aug 29 2023 at 13:02):Alex Crichton took a look at the PR I created and determined that my problem appears to be that the required was-libc explicit initialization is not getting called. This is handled by the _start function which as this is a component does not get called.
I've attempted to call _start myself and while it appears to be present in the WASM file the call to instantiate does not produce an Instance that includes it.
let instance = Testfs::instantiate_async(&mut store, &component, &linker).await?;
let start = instance.1.get_typed_func::<(), ()>(&mut store, "_start")?;
start.call_async(&mut store, ()).await?;
This results in "Error: failed to find function export _start"
So how can I get the _start code executed ?
Alex Crichton (Aug 29 2023 at 13:50):IMO this is mostly a set of details and things for the Go toolchain here, primarily around the "reactor vs command" concept. WASI historically would generate _start for commands which is like the main function sort of, and it's probably not suitable for use here as-is other than proving out it's what's required to fix the issue. For reactors the C/Rust toolchain generate an _initialize function which is called before other imports which is where initialization happens.
All that being said my main recommendation would be to update wasi-libc as-used by tinygo. If that is done then I believe the issue goes away because wasi-libc no longer requires explicit initialization.
Alex Crichton (Aug 29 2023 at 13:51):But you won't be able to solve this at the embedding/wasmtime layer beecause as you've found _start isn't exported from the component, just the core module. It's also not the right function to call since it's supposed to be main sort of.
I should also mention that updating wasi-libc may not be enough in the long run if the Go runtime itself requires explicit initialization. If everything there is lazily initialized though then it should be fine to not use _start or _initialize
Richard Backhouse (Aug 29 2023 at 14:19):I just tried a tinygo 0.29 (which still does not appear to updated its wasi-libc version :-( ) with a wasi-sdk v20 version of wasi-libc and it now works. I can at least move forward for now with a hacked version of tinygo.
Thanks Alex
Richard Backhouse (Aug 29 2023 at 14:33):Actually will have to walk back on the 20 version of wasi-libc. I had actually used the WASI-SDK version from Joel Dice's component-py repo which is newer. The 20 version does not work. I can still move forward though
Last updated: Oct 23 2024 at 20:03 UTC