Colin D Murphy (Jan 27 2025 at 19:53): Colin D Murphy (Jan 27 2025 at 19:53):I'm having trouble with tempfile in WASI P2. I see that there is something in cap-std, but when I use tempfile::tempdir() in my code and run it with wasmtime, it uses https://github.com/rust-lang/rust/blob/master/library/std/src/sys/pal/wasi/os.rs#L266-L268 and panics. Does the cap-std work just need to be added to the wasmtime host code, do I need to use a wasi specific tempfile, or something else?
Alex Crichton (Jan 27 2025 at 20:02):I think that means that you're using a crate which doesn't fully support WASI yet in the guest itself, is this perhaps related to using the tempdir crate in the guest? If that's using std::env::temp_dir() you'll either need to use a different constructor or ask for WASI-specific handling
Joel Dice (Jan 27 2025 at 20:06):What would would be the "right thing" for tempdir to do when targeting WASI given that there's no official temporary directory? Would it make sense for std::env::temp_dir() to return "/tmp" and leave it up to the host embedder to make sure one of the preopens includes "/tmp"?
Alex Crichton (Jan 27 2025 at 20:33):Naively I'd probably say that tempdir::TempDir::new() (or w/e constructor is being used) should return an error (instead of panicking) and the user would have to use tempdir::TempDir::new_in(...) instead
Alex Crichton (Jan 27 2025 at 20:34):not a great experience but IMO is better than picking /tmp
Lann Martin (Jan 27 2025 at 20:42):Looks like its just following wasi-libc's example? https://github.com/WebAssembly/wasi-libc/blob/f1c557ce174b3ee2eff6122b2deebf637f6cdaee/libc-top-half/musl/include/stdio.h#L153
Alex Crichton (Jan 27 2025 at 22:00):I can probably clarify my position is that how best to fix this probably requires more context to the problem. Colin describe a panic but I don't know in what context a temporary directory was otherwise required. It's also a fact though that there's no canonical temporary directory with WASI, or at least not that I know of. How best to thread that needle is what requires context. One option is "just return /tmp". Another option is "return a dummy thing and issue a compiler warning". Another option is "return an error". Another is panic.
Lots that can be done here but balancing "this application requires a temporary directory" and "WASI fundamentally does not have a temporary directory" at least in my opinion has no easy answer and there's tradeoffs with whatever is chosen. The answer here probably relies on context at least I personally don't have and would require more understanding of the consequence sof a particular chosen solution in the context of the original application, language integration, etc.
Lann Martin (Jan 27 2025 at 22:18):The assumption of having a global tempdir is probably baked into quite a bit of code. I wonder if it would be a worthwhile extension of e.g. wasi:filesystem to provide an opt-in tempdir root
Colin D Murphy (Jan 28 2025 at 15:25):The context of the issue is random Rust code, often test code, that expects std::env::temp_dir()to return something.
Using #[cfg] directives to change the behavior to tempdir::TempDir::new_in("/") works. Maybe it would make sense to modify the tempfile crate so that this is done automatically? It's the ease adoption vs technical correctness question.
Alex Crichton (Jan 28 2025 at 15:54):Yeah I don't doubt that tempdirs are baked into places. If the goal is "everything should run super easily on WASI" (which I don't doubt as a goal), then IMO the answer here is to not try to patch around this somehow but it's to somehow get something official into wasi:filesystem around a temporary directory. I don't know what that would look like myself, though.
Alex Crichton (Jan 28 2025 at 15:55):Basically I don't think there's a "fix" for the standard library or the tempdir crate at this time, something needs to happen up a layer in wasi:filesystem itself IMO for this to get fixed. Anything else is at best a local patch or workaround for now
Colin D Murphy (Jan 28 2025 at 15:56):Sorry to ask a dumb question. But in what circumstances is the cap-std/cap-tempfile code used? I want to be sure we aren't already trying to solve this.
Alex Crichton (Jan 28 2025 at 15:59):Wasmtime's implementation of WASI uses cap-std but I don't think we use cap-tempfile. I'm not sure whether cap-std or cap-tempfile compiles to WASI itself to be a concern for guests using this (although if you're getting the panic through cap-tempfile it probably is a guest compiled to wasm)
Dave Bakker (badeend) (Jan 28 2025 at 16:22):One unique advantage of WASI is its virtualizability. If an arbitrary package beyond your control depends on an /tmp directory, it can be virtualized into existence by tools like WASI-Virt. Those should be able to virtualize the /tmp folder in-memory, and pass through other paths to the host. I don't know if WASI-Virt supports this specific use-case, but at least in theory it should be possible. Maybe this helps
Colin D Murphy (Jan 28 2025 at 16:26):I'm sure WASI-Virt is relevant to this discussion, but it isn't for my particular use case. The reason the crate I'm working on (c2pa) needs tempfiles is because the assets are too large to live in memory but at the same time require seekable streams. Therefore the filesystem is the only option.
Joel Dice (Jan 28 2025 at 16:52):We already have wasi:cli/environment/initial-cwd. Seems reasonable to add a temp-dir function to the same interface.
Colin D Murphy (Jan 30 2025 at 17:21):PR
https://github.com/WebAssembly/wasi-cli/pull/54
Last updated: Apr 09 2025 at 12:05 UTC