wasmtime / issue #9194 wasmtime serve seems to ignore --d... · git-wasmtime

Stream: git-wasmtime

Topic: wasmtime / issue #9194 wasmtime serve seems to ignore --d...

Wasmtime GitHub notifications bot (Sep 03 2024 at 09:01):

javorszky added the bug label to Issue #9194.

Wasmtime GitHub notifications bot (Sep 03 2024 at 09:01):

javorszky opened issue #9194:

We've been trying to create a wasi:http world file to have access to the filesystem. We couldn't get it working.

Test Case

I've put up a repository here: https://github.com/javorszky/wasmtime-fscheck-serve which has the source code, a dockerfile, a makefile to make sure everything is consistent across different machines and host triplets.

Steps to Reproduce

clone the repository

build the docker container with make bd

run the docker container with make dr

send an empty get request to localhost:8080

Expected Results

I expect the wasm component to respond with a 200 OK with the body being the list of files and directories in the opened directory.

Actual Results

Response is a panicked crash with the following error log in docker:
stderr [3] :: thread '<unnamed>' panicked at src/lib.rs:28:53:
stderr [3] :: called `Result::unwrap()` on an `Err` value: Custom { kind: Uncategorized, error: "failed to find a pre-opened file descriptor through which \"/shenanigans\" could be opened" }
stderr [3] :: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2024-09-03T08:48:29.977839Z ERROR wasmtime_cli::commands::serve: [3] :: Error {
    context: "error while executing at wasm backtrace:\n    0: 0x258f2 - wasmtime_filesystem_debug.wasm!__rust_start_panic\n    1: 0x2546e - wasmtime_filesystem_debug.wasm!rust_panic\n    2: 0x25224 - wasmtime_filesystem_debug.wasm!std::panicking::rust_panic_with_hook::h8bb4f94cf9e68e20\n    3: 0x24646 - wasmtime_filesystem_debug.wasm!std::panicking::begin_panic_handler::{{closure}}::h2c354a3372fcca00\n    4: 0x245ad - wasmtime_filesystem_debug.wasm!std::sys_common::backtrace::__rust_end_short_backtrace::hf791a0fba63af5e9\n    5: 0x24dbc - wasmtime_filesystem_debug.wasm!rust_begin_unwind\n    6: 0x2a66b - wasmtime_filesystem_debug.wasm!core::panicking::panic_fmt::haadcabb094f4c85b\n    7: 0x2c2de - wasmtime_filesystem_debug.wasm!core::result::unwrap_failed::hd09ff05c9c2fb25f\n    8: 0x6a1d - wasmtime_filesystem_debug.wasm!<wasmtime_filesystem_debug::Component as wasi::proxy::exports::wasi::http::incoming_handler::Guest>::handle::hd44cd139d21c109c\n    9: 0x7853 - wasmtime_filesystem_debug.wasm!wasi::proxy::exports::wasi::http::incoming_handler::_export_handle_cabi::he753d69b8503d63c\n   10: 0x7798 - wasmtime_filesystem_debug.wasm!wasi:http/incoming-handler@0.2.1#handle\nnote: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information",
    source: UnreachableCodeReached,
}
error: hyper::Error(User(Service), guest never invoked `response-outparam::set` method: error while executing at wasm backtrace:
    0: 0x258f2 - wasmtime_filesystem_debug.wasm!__rust_start_panic
    1: 0x2546e - wasmtime_filesystem_debug.wasm!rust_panic
    2: 0x25224 - wasmtime_filesystem_debug.wasm!std::panicking::rust_panic_with_hook::h8bb4f94cf9e68e20
    3: 0x24646 - wasmtime_filesystem_debug.wasm!std::panicking::begin_panic_handler::{{closure}}::h2c354a3372fcca00
    4: 0x245ad - wasmtime_filesystem_debug.wasm!std::sys_common::backtrace::__rust_end_short_backtrace::hf791a0fba63af5e9
    5: 0x24dbc - wasmtime_filesystem_debug.wasm!rust_begin_unwind
    6: 0x2a66b - wasmtime_filesystem_debug.wasm!core::panicking::panic_fmt::haadcabb094f4c85b
    7: 0x2c2de - wasmtime_filesystem_debug.wasm!core::result::unwrap_failed::hd09ff05c9c2fb25f
    8: 0x6a1d - wasmtime_filesystem_debug.wasm!<wasmtime_filesystem_debug::Component as wasi::proxy::exports::wasi::http::incoming_handler::Guest>::handle::hd44cd139d21c109c
    9: 0x7853 - wasmtime_filesystem_debug.wasm!wasi::proxy::exports::wasi::http::incoming_handler::_export_handle_cabi::he753d69b8503d63c
   10: 0x7798 - wasmtime_filesystem_debug.wasm!wasi:http/incoming-handler@0.2.1#handle
note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information

Caused by:
    wasm trap: wasm `unreachable` instruction executed)
Versions and Environment

Wasmtime version or commit: wasmtime-cli 24.0.0

Operating system: linux bullseye (docker base image is rust:bullseye)

Architecture: aarch64-unknown-linux-gnu

Extra Info

It looks like preopen doesn't happen with wasmtime serve, as there's no openat syscall to the directory we're trying to preopen. For comparison I also have a cli version of the wasm with a preopened directory where I can see an openat call to the directory I specified in strace, and the component prints out the contents of the directory as expected.

I also tried many variations of passing -Scli=y -Spreview2=y with no fix to directory access.

Adding -Dlogging=y does not yield anything useful.

My assumption that wasmtime serve should honour the --dir flag is based on the following:

PR 8279: https://github.com/bytecodealliance/wasmtime/pull/8279

conversation in issue 8086: https://github.com/bytecodealliance/wasmtime/issues/8086

this source code dive:

serve command's handle_request calls new_store

serve command's new_store calls configure_wasip2

common's configure_wasip2 iterates over the directories to open

It looks like common's self.dirs is not getting populated from the command line argument.

Wasmtime GitHub notifications bot (Sep 03 2024 at 10:09):

javorszky edited issue #9194:

We've been trying to create a wasi:http world file to have access to the filesystem. We couldn't get it working.

Test Case

I've put up a repository here: https://github.com/javorszky/wasmtime-fscheck-serve which has the source code, a dockerfile, a makefile to make sure everything is consistent across different machines and host triplets.

Steps to Reproduce

clone the repository

build the docker container with make bd

run the docker container with make dr

send an empty get request to localhost:8080

Expected Results

I expect the wasm component to respond with a 200 OK with the body being the list of files and directories in the opened directory.

Actual Results

Response is a panicked crash with the following error log in docker:
stderr [3] :: thread '<unnamed>' panicked at src/lib.rs:28:53:
stderr [3] :: called `Result::unwrap()` on an `Err` value: Custom { kind: Uncategorized, error: "failed to find a pre-opened file descriptor through which \"/shenanigans\" could be opened" }
stderr [3] :: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2024-09-03T08:48:29.977839Z ERROR wasmtime_cli::commands::serve: [3] :: Error {
    context: "error while executing at wasm backtrace:\n    0: 0x258f2 - wasmtime_filesystem_debug.wasm!__rust_start_panic\n    1: 0x2546e - wasmtime_filesystem_debug.wasm!rust_panic\n    2: 0x25224 - wasmtime_filesystem_debug.wasm!std::panicking::rust_panic_with_hook::h8bb4f94cf9e68e20\n    3: 0x24646 - wasmtime_filesystem_debug.wasm!std::panicking::begin_panic_handler::{{closure}}::h2c354a3372fcca00\n    4: 0x245ad - wasmtime_filesystem_debug.wasm!std::sys_common::backtrace::__rust_end_short_backtrace::hf791a0fba63af5e9\n    5: 0x24dbc - wasmtime_filesystem_debug.wasm!rust_begin_unwind\n    6: 0x2a66b - wasmtime_filesystem_debug.wasm!core::panicking::panic_fmt::haadcabb094f4c85b\n    7: 0x2c2de - wasmtime_filesystem_debug.wasm!core::result::unwrap_failed::hd09ff05c9c2fb25f\n    8: 0x6a1d - wasmtime_filesystem_debug.wasm!<wasmtime_filesystem_debug::Component as wasi::proxy::exports::wasi::http::incoming_handler::Guest>::handle::hd44cd139d21c109c\n    9: 0x7853 - wasmtime_filesystem_debug.wasm!wasi::proxy::exports::wasi::http::incoming_handler::_export_handle_cabi::he753d69b8503d63c\n   10: 0x7798 - wasmtime_filesystem_debug.wasm!wasi:http/incoming-handler@0.2.1#handle\nnote: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information",
    source: UnreachableCodeReached,
}
error: hyper::Error(User(Service), guest never invoked `response-outparam::set` method: error while executing at wasm backtrace:
    0: 0x258f2 - wasmtime_filesystem_debug.wasm!__rust_start_panic
    1: 0x2546e - wasmtime_filesystem_debug.wasm!rust_panic
    2: 0x25224 - wasmtime_filesystem_debug.wasm!std::panicking::rust_panic_with_hook::h8bb4f94cf9e68e20
    3: 0x24646 - wasmtime_filesystem_debug.wasm!std::panicking::begin_panic_handler::{{closure}}::h2c354a3372fcca00
    4: 0x245ad - wasmtime_filesystem_debug.wasm!std::sys_common::backtrace::__rust_end_short_backtrace::hf791a0fba63af5e9
    5: 0x24dbc - wasmtime_filesystem_debug.wasm!rust_begin_unwind
    6: 0x2a66b - wasmtime_filesystem_debug.wasm!core::panicking::panic_fmt::haadcabb094f4c85b
    7: 0x2c2de - wasmtime_filesystem_debug.wasm!core::result::unwrap_failed::hd09ff05c9c2fb25f
    8: 0x6a1d - wasmtime_filesystem_debug.wasm!<wasmtime_filesystem_debug::Component as wasi::proxy::exports::wasi::http::incoming_handler::Guest>::handle::hd44cd139d21c109c
    9: 0x7853 - wasmtime_filesystem_debug.wasm!wasi::proxy::exports::wasi::http::incoming_handler::_export_handle_cabi::he753d69b8503d63c
   10: 0x7798 - wasmtime_filesystem_debug.wasm!wasi:http/incoming-handler@0.2.1#handle
note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information

Caused by:
    wasm trap: wasm `unreachable` instruction executed)
The important error is this one:
failed to find a pre-opened file descriptor through which \"/shenanigans\" could be opened
coming from this source code:
fs::read_dir("/shenanigans").unwrap()
where the command to start it was
/root/.wasmtime/bin/wasmtime serve --dir=/shenanigans ./target/wasm32-wasip1/debug/wasmtime_filesystem_debug.wasm
Versions and Environment

Wasmtime version or commit: wasmtime-cli 24.0.0

Operating system: linux bullseye (docker base image is rust:bullseye)

Architecture: aarch64-unknown-linux-gnu

Extra Info

It looks like preopen doesn't happen with wasmtime serve, as there's no openat syscall to the directory we're trying to preopen. For comparison I also have a cli version of the wasm with a preopened directory where I can see an openat call to the directory I specified in strace, and the component prints out the contents of the directory as expected.

I also tried many variations of passing -Scli=y -Spreview2=y with no fix to directory access.

Adding -Dlogging=y does not yield anything useful.

My assumption that wasmtime serve should honour the --dir flag is based on the following:
* PR 8279: https://github.com/bytecodealliance/wasmtime/pull/8279
* conversation in issue 8086: https://github.com/bytecodealliance/wasmtime/issues/8086
* this source code dive:
1. serve command's handle_request calls new_store
2. serve command's new_store calls configure_wasip2
3. common's configure_wasip2 iterates over the directories to open

It looks like common's self.dirs is not getting populated from the command line argument.

Wasmtime GitHub notifications bot (Sep 03 2024 at 17:44):

alexcrichton commented on issue #9194:

Thanks for the detailed report! I think the main issue is with this line where that causes cargo component to use the "proxy adapter" when converting WASIp1 calls to WASIp2. The proxy version hardcodes that the filesystem is inaccessible and will always return an empty filesystem. If you want access to the filesystem I think you probably want to just remove that line. That should cause the "reactor adapter" to be used instead which forwards WASIp1 calls to WASIp2.

After doing that you'll want to pass -Scli to wasmtime serve, and it should work this time. It didn't work before because the proxy adapter meant that WASIp2 functions weren't ever called.

Does that help clarify what's happening?

Wasmtime GitHub notifications bot (Sep 03 2024 at 17:53):

javorszky commented on issue #9194:

@alexcrichton yes, that makes sense. I removed that line and now the component works as expected, thank you so much!

I went through the documentation for wasmtime, and couldn't find any mention of what proxy:true does. I'll keep digging, but is there info about this already published I'm not finding?

Again, much appreciated!

Wasmtime GitHub notifications bot (Sep 03 2024 at 17:54):

javorszky commented on issue #9194:

Solved by removing the proxy = true line from the Cargo.toml under the [package.metadata.component] section

Wasmtime GitHub notifications bot (Sep 03 2024 at 17:56):

alexcrichton commented on issue #9194:

Ah the reason you didn't find that in the wasmtime docs is that it's a feature of cargo component, not Wasmtime. Wasmtime doesn't currently house docs for each guest language that could compile to wasm, although https://component-model.bytecodealliance.org/ is a location where such docs could work. Alas though I don't believe the proxy = true part is documented all that well in either place right now either, but it'd be good to contribute!

Wasmtime GitHub notifications bot (Sep 03 2024 at 17:56):

javorszky closed issue #9194:

We've been trying to create a wasi:http world file to have access to the filesystem. We couldn't get it working.

Test Case

I've put up a repository here: https://github.com/javorszky/wasmtime-fscheck-serve which has the source code, a dockerfile, a makefile to make sure everything is consistent across different machines and host triplets.

Steps to Reproduce

clone the repository

build the docker container with make bd

run the docker container with make dr

send an empty get request to localhost:8080

Expected Results

I expect the wasm component to respond with a 200 OK with the body being the list of files and directories in the opened directory.

Actual Results

Response is a panicked crash with the following error log in docker:
stderr [3] :: thread '<unnamed>' panicked at src/lib.rs:28:53:
stderr [3] :: called `Result::unwrap()` on an `Err` value: Custom { kind: Uncategorized, error: "failed to find a pre-opened file descriptor through which \"/shenanigans\" could be opened" }
stderr [3] :: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2024-09-03T08:48:29.977839Z ERROR wasmtime_cli::commands::serve: [3] :: Error {
    context: "error while executing at wasm backtrace:\n    0: 0x258f2 - wasmtime_filesystem_debug.wasm!__rust_start_panic\n    1: 0x2546e - wasmtime_filesystem_debug.wasm!rust_panic\n    2: 0x25224 - wasmtime_filesystem_debug.wasm!std::panicking::rust_panic_with_hook::h8bb4f94cf9e68e20\n    3: 0x24646 - wasmtime_filesystem_debug.wasm!std::panicking::begin_panic_handler::{{closure}}::h2c354a3372fcca00\n    4: 0x245ad - wasmtime_filesystem_debug.wasm!std::sys_common::backtrace::__rust_end_short_backtrace::hf791a0fba63af5e9\n    5: 0x24dbc - wasmtime_filesystem_debug.wasm!rust_begin_unwind\n    6: 0x2a66b - wasmtime_filesystem_debug.wasm!core::panicking::panic_fmt::haadcabb094f4c85b\n    7: 0x2c2de - wasmtime_filesystem_debug.wasm!core::result::unwrap_failed::hd09ff05c9c2fb25f\n    8: 0x6a1d - wasmtime_filesystem_debug.wasm!<wasmtime_filesystem_debug::Component as wasi::proxy::exports::wasi::http::incoming_handler::Guest>::handle::hd44cd139d21c109c\n    9: 0x7853 - wasmtime_filesystem_debug.wasm!wasi::proxy::exports::wasi::http::incoming_handler::_export_handle_cabi::he753d69b8503d63c\n   10: 0x7798 - wasmtime_filesystem_debug.wasm!wasi:http/incoming-handler@0.2.1#handle\nnote: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information",
    source: UnreachableCodeReached,
}
error: hyper::Error(User(Service), guest never invoked `response-outparam::set` method: error while executing at wasm backtrace:
    0: 0x258f2 - wasmtime_filesystem_debug.wasm!__rust_start_panic
    1: 0x2546e - wasmtime_filesystem_debug.wasm!rust_panic
    2: 0x25224 - wasmtime_filesystem_debug.wasm!std::panicking::rust_panic_with_hook::h8bb4f94cf9e68e20
    3: 0x24646 - wasmtime_filesystem_debug.wasm!std::panicking::begin_panic_handler::{{closure}}::h2c354a3372fcca00
    4: 0x245ad - wasmtime_filesystem_debug.wasm!std::sys_common::backtrace::__rust_end_short_backtrace::hf791a0fba63af5e9
    5: 0x24dbc - wasmtime_filesystem_debug.wasm!rust_begin_unwind
    6: 0x2a66b - wasmtime_filesystem_debug.wasm!core::panicking::panic_fmt::haadcabb094f4c85b
    7: 0x2c2de - wasmtime_filesystem_debug.wasm!core::result::unwrap_failed::hd09ff05c9c2fb25f
    8: 0x6a1d - wasmtime_filesystem_debug.wasm!<wasmtime_filesystem_debug::Component as wasi::proxy::exports::wasi::http::incoming_handler::Guest>::handle::hd44cd139d21c109c
    9: 0x7853 - wasmtime_filesystem_debug.wasm!wasi::proxy::exports::wasi::http::incoming_handler::_export_handle_cabi::he753d69b8503d63c
   10: 0x7798 - wasmtime_filesystem_debug.wasm!wasi:http/incoming-handler@0.2.1#handle
note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information

Caused by:
    wasm trap: wasm `unreachable` instruction executed)
The important error is this one:
failed to find a pre-opened file descriptor through which \"/shenanigans\" could be opened
coming from this source code:
fs::read_dir("/shenanigans").unwrap()
where the command to start it was
/root/.wasmtime/bin/wasmtime serve --dir=/shenanigans ./target/wasm32-wasip1/debug/wasmtime_filesystem_debug.wasm
Versions and Environment

Wasmtime version or commit: wasmtime-cli 24.0.0

Operating system: linux bullseye (docker base image is rust:bullseye)

Architecture: aarch64-unknown-linux-gnu

Extra Info

It looks like preopen doesn't happen with wasmtime serve, as there's no openat syscall to the directory we're trying to preopen. For comparison I also have a cli version of the wasm with a preopened directory where I can see an openat call to the directory I specified in strace, and the component prints out the contents of the directory as expected.

I also tried many variations of passing -Scli=y -Spreview2=y with no fix to directory access.

Adding -Dlogging=y does not yield anything useful.

My assumption that wasmtime serve should honour the --dir flag is based on the following:
* PR 8279: https://github.com/bytecodealliance/wasmtime/pull/8279
* conversation in issue 8086: https://github.com/bytecodealliance/wasmtime/issues/8086
* this source code dive:
1. serve command's handle_request calls new_store
2. serve command's new_store calls configure_wasip2
3. common's configure_wasip2 iterates over the directories to open

It looks like common's self.dirs is not getting populated from the command line argument.

Wasmtime GitHub notifications bot (Sep 03 2024 at 17:57):

javorszky commented on issue #9194:

Excellent, thank you! I'll put together some documentation once I figure out what it does in detail :smile:

Last updated: Nov 22 2024 at 16:03 UTC