Can't seem to preopen files with --dir · wasmtime

Hello folks! I've got a tiny little Rust program that takes a file name as an argument, reads it and prints it out.

It works just fine when compiled natively, but when running on Wasmtime of course it needs to be given capabilities over that file.

I learned there's a --dir flag that I could use from playing around with wasmtime help run, but I'm not entirely sure how I should use it?

E.g, if I call my program as wasmtime run ./cat.wasm -- ./text then I'd expect to be able to add --dir=$(pwd) or --dir=. an have that work.

Unfortunately, I can't seem to give it a path that it likes for this and I'm not sure if I'm missing some consideration about how PATHs are handled in WASI.

Leandro Ostera (Nov 22 2020 at 12:26):

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Custom {
  kind: Other,
  error: "failed to find a preopened file descriptor through which \"test_a.txt\" could be opened"
}'

Till Schneidereit (Nov 22 2020 at 14:09):

Hi Leandro! That should indeed work, yes. Can you share what the exact command line looks like that fails for you? Based on your description it sounds like perhaps you're adding the --dir=. to the very end. In that case, that wouldn't work, since everything after the -- is passed as arguments to the content, not wasmtime. So this wouldn't work:

wasmtime run ./cat.wasm -- ./text

but (assuming you're trying to open the file text in the current working directory) this should:

wasmtime run --dir=. ./cat.wasm -- ./text

Notification Bot (Nov 22 2020 at 14:10):

Leandro Ostera (Nov 22 2020 at 14:23):

λ wasmtime run --dir=. ./cat.wasm -- ./test
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Custom { kind: Other, error: "failed to find a preopened file descriptor through which \"./test\" could be opened" }'

I also tried removing the call to fs::read_to_string that reads the file, and I'm still getting the same thing. Does wasmtime do anything with the arguments in case they are paths? (E.g, try to "preopen" them?)

Till Schneidereit (Nov 22 2020 at 15:50):

That looks like it should work, yes — though of course that depends on the actual code you're compiling :smile:

For reference, I just tested this code (in a project created with cargo new wasi-exp), and it successfully prints the project's Cargo.toml file:

use std::error::Error;
use std::{env, fs};

fn main() -> Result<(), Box<dyn Error>> {
    let args: Vec<String> = env::args().collect();
    let str = fs::read_to_string(&args[1])?;
    println!("Reading file '{}'. contents:\n{}", args[1], str);
    Ok(())
}

wasmtime run --dir=. target/wasm32-wasi/debug/wasi-exp.wasm ./Cargo.toml

Till Schneidereit (Nov 22 2020 at 15:51):

It doesn't, because it can't know that they're meant to be files. That's because there's no way to provide typed arguments to a WASI main function — yet :smile:

Leandro Ostera (Nov 24 2020 at 11:29):

let data = std::fs::read_to_string(&path)?.replace("\n", "");

where path is passed in after parsing the arguments. Printing it out it shows me the same thing I'm passing in.

This data is then printed out again with print! very similarly to how you showed.

However, in the interest of keeping this self-contained as an example, I omitted that I'm putting together this binary by building a WASM library and using Walrus to extend the .wasm to include the right boot code.

The reason for this is that I'm building a little virtual machine that builds targets into binaries that are self-contained (so you don't need to _also_ install the virtual machine, which wouldn't be possible in a browser).

Could this have anything to do with that? Is there any "seal" that I'm breaking that Wasmtime will look into?

Till Schneidereit (Nov 24 2020 at 13:25):

It shouldn't affect preopen in and of itself, and there's definitely not some kind of "seal" to be broken. but depending on the details of what you're doing, perhaps it has an impact on the Rust std library implementation of preopen support?

You should be able to see the details of which file operations are attempted by prepending RUST_LOG=wasi to your command line. That'll spew a pretty verbose wall of log info, so it might take some wading through, but it should in particular include these relevant ones:

 DEBUG wasi_common::old::snapshot_0::ctx                   > WasiCtx inserting

 TRACE wasi_common::wasi::wasi_snapshot_preview1           > wiggle abi; module="wasi_snapshot_preview1" function="path_open"

And following that TRACE message various lines about the path operations related to path_open, which should be the most interesting part for debugging this. For reference, here's what that section looks like for my test case posted yesterday:

 TRACE wasi_common::wasi::wasi_snapshot_preview1           > wiggle abi; module="wasi_snapshot_preview1" function="path_open"
 TRACE wasi_common::wasi::wasi_snapshot_preview1           > fd=Fd(3) dirflags=symlink_follow (0x1) path=*guest 0x1100f0/10 oflags=empty (0x0) fs_rights_base=fd_read|fd_seek|fd_fdstat_set_flags|fd_sync|fd_tell|fd_advise|path_create_directory|path_create_file|path_link_source|path_link_target|path_open|fd_readdir|path_readlink|path_rename_source|path_rename_target|path_filestat_get|fd_filestat_get|fd_filestat_set_times|path_symlink|path_remove_directory|path_unlink_file|poll_fd_readwrite (0xfa7febe) fs_rights_inherting=fd_read|fd_seek|fd_fdstat_set_flags|fd_sync|fd_tell|fd_advise|path_create_directory|path_create_file|path_link_source|path_link_target|path_open|fd_readdir|path_readlink|path_rename_source|path_rename_target|path_filestat_get|fd_filestat_get|fd_filestat_set_times|path_symlink|path_remove_directory|path_unlink_file|poll_fd_readwrite (0xfa7febe) fdflags=empty (0x0)
 TRACE wasi_common::snapshots::wasi_snapshot_preview1      >      | needed_rights=HandleRights { base: path_open (0x2000), inheriting: fd_read|fd_seek|fd_fdstat_set_flags|fd_sync|fd_tell|fd_advise|path_create_directory|path_create_file|path_link_source|path_link_target|path_open|fd_readdir|path_readlink|path_rename_source|path_rename_target|path_filestat_get|fd_filestat_get|fd_filestat_set_times|path_symlink|path_remove_directory|path_unlink_file|poll_fd_readwrite (0xfa7febe) }
 TRACE wasi_common::path                                   > path="Cargo.toml"
 DEBUG wasi_common::path                                   > path get cur_path=Cargo.toml
 DEBUG wasi_common::path                                   > path_get path_stack=[]
 DEBUG wasi_common::sys::unix::path                        > path_get readlinkat path="Cargo.toml"
 TRACE wasi_common::snapshots::wasi_snapshot_preview1      >      | calling path_open impl: read=true, write=false
 DEBUG wasi_common::sys::unix::path                        > path_open dirfd=OsDir { rights: Cell { value: HandleRights { base: Rights(264240792), inheriting: Rights(268435455) } }, handle: RawOsHandle(File { fd: 7, path: "/Users/till/exp/wasi-exp", read: true, write: false }), stream_ptr: RefCell { value: Dir(0x7faf99204190) } } path="Cargo.toml" oflags=NOFOLLOW
 DEBUG wasi_common::sys::unix::path                        > new_fd=9
 DEBUG wasi_common::sys::unix                              > Host fd 9 is a file
 DEBUG wasi_common::sys                                    > Created new instance of OsFile handle=OsFile { rights: Cell { value: HandleRights { base: Rights(148898239), inheriting: Rights(0) } }, handle: RawOsHandle(File { fd: 9, path: "/Users/till/exp/wasi-exp/Cargo.toml", read: true, write: false }) }
 TRACE wasi_common::wasi::wasi_snapshot_preview1           > opened_fd=Fd(4)
 TRACE wasi_common::wasi::wasi_snapshot_preview1           > success=No error occurred. System call completed successfully. (Errno::Success(0))

Leandro Ostera (Nov 24 2020 at 14:26):

Thanks! This is the output I've gotten with RUST_LOG=wasi set. I can't seem to find a TRACE call to path_open :/

λ RUST_LOG=wasi wasmtime run --dir=. cat.wasm test
 DEBUG wasi_common::ctx > WasiCtx inserting entry pending=PendingEntry::Thunk(0x7fff947ae470)
 DEBUG wasi_common::sys::unix > Host fd 0 is a char device
 DEBUG wasi_common::ctx       > WasiCtx inserted fd=Fd(0)
 DEBUG wasi_common::ctx       > WasiCtx inserting entry pending=PendingEntry::Thunk(0x7fff947ae470)
 DEBUG wasi_common::sys::unix > Host fd 1 is a char device
 DEBUG wasi_common::ctx       > WasiCtx inserted fd=Fd(1)
 DEBUG wasi_common::ctx       > WasiCtx inserting entry pending=PendingEntry::Thunk(0x7fff947ae470)
 DEBUG wasi_common::sys::unix > Host fd 2 is a char device
 DEBUG wasi_common::ctx       > WasiCtx inserted fd=Fd(2)
 DEBUG wasi_common::ctx       > WasiCtx inserted fd=Fd(3)
 DEBUG wasi_common::old::snapshot_0::ctx > WasiCtx inserting (0, Some(PendingEntry::Thunk(0x7fff947ae918)))
 DEBUG wasi_common::old::snapshot_0::sys::unix::entry_impl > Host fd 0 is a char device
 DEBUG wasi_common::old::snapshot_0::ctx                   > WasiCtx inserting (1, Some(PendingEntry::Thunk(0x7fff947ae928)))
 DEBUG wasi_common::old::snapshot_0::sys::unix::entry_impl > Host fd 1 is a char device
 DEBUG wasi_common::old::snapshot_0::ctx                   > WasiCtx inserting (2, Some(PendingEntry::Thunk(0x7fff947ae938)))
 DEBUG wasi_common::old::snapshot_0::sys::unix::entry_impl > Host fd 2 is a char device
 DEBUG wasi_common::old::snapshot_0::sys::unix::entry_impl > Host fd 5 is a directory
 DEBUG wasi_common::old::snapshot_0::ctx                   > WasiCtx inserting (3, Entry { file_type: 3, descriptor: OsHandle(OsHandle(File { fd: 5, path: "/home/ostera/repos/github.com/AbstractMachinesLab/lam/examples/cat", read: t
rue, write: false })), rights_base: 264240792, rights_inheriting: 268435455, preopen_path: Some(".") })
 DEBUG wasi_common::old::snapshot_0::ctx                   > WasiCtx entries = {2: Entry { file_type: 2, descriptor: Stderr, rights_base: 136314954, rights_inheriting: 136314954, preopen_path: None }, 1: Entry { file_type: 2, descri
ptor: Stdout, rights_base: 136314954, rights_inheriting: 136314954, preopen_path: None }, 3: Entry { file_type: 3, descriptor: OsHandle(OsHandle(File { fd: 5, path: "/home/ostera/repos/github.com/AbstractMachinesLab/lam/examples/cat
", read: true, write: false })), rights_base: 264240792, rights_inheriting: 268435455, preopen_path: Some(".") }, 0: Entry { file_type: 2, descriptor: Stdin, rights_base: 136314954, rights_inheriting: 136314954, preopen_path: None }

Leandro Ostera (Nov 26 2020 at 13:30):

Till Schneidereit (Nov 30 2020 at 15:20):

@Leandro Ostera I don't know the details of how Rust's standard library handles preopened files. It's possible that there's a check happening inside the content code that already detects that no preopened directory can be found that'd make this path resolve.

Given that it seems like that should work — see my example above — perhaps the boot code you're injecting somehow breaks it? To test that, maybe you could use the reduced test case I posted above, verify that it works with the Wasmtime CLI as expected, and then see whether it also fails in your setup?

Stream: wasmtime

Topic: Can't seem to preopen files with --dir

Leandro Ostera (Nov 22 2020 at 12:24):