Andy Wingo (Oct 07 2025 at 14:36): Andy Wingo (Oct 07 2025 at 14:36):Hello dear wasmtime people :)
So, wasi-testsuite is of limited utility until its tests pass on all platforms. If anyone has some spare spoons, I have some concrete things that need fixing in order to have fully green tests when running with Wasmtime.
Highest priority things that would allow us to be green on Ubuntu:
Next highest that would allow us to be green on MacOS:
fadvise behaving differently on MacOS; weirdly Windows is fine, for once https://github.com/WebAssembly/wasi-testsuite/issues/178; I think that Wasmtime should shim this one.Next highest priority that would allow the WASIp3 test suite to be green on Windows:
WRITE flags: https://github.com/WebAssembly/wasi-testsuite/issues/176; ideally we make Windows behave as in the test, but if needed we can relax the spec.You can get the Wasm files mentioned in the bug reports here: https://github.com/WebAssembly/wasi-testsuite/tree/prod/testsuite-base/tests/rust/testsuite.
Finally there is an excruciating slog of around 30 failures for Rust WASIp1 tests in Windows: https://github.com/WebAssembly/wasi-testsuite/issues/175. I think it's mostly "expected an error but not that one" failures; given that it's WASIp1, I suggest that we just relax those tests, as a first approach. I.e. we don't assume they are wasmtime bugs.
Alex Crichton (Oct 07 2025 at 14:48):Need new wasmtime release
For this at least Wasmtime 38 will suffice which'll get released on Oct 20
Alex Crichton (Oct 07 2025 at 14:57):Thanks for writing this up though! I can try to poke at some of these things between builds this week
Alex Crichton (Oct 07 2025 at 16:22):Hm, actually, Andy would you be up for poking around in Wasmtime for this? Or would you prefer to stick to writing tests? I'm mostly fearful that this is going to create a very long backlog we don't have resources to apply at this time. The backlog is not itself the problem but having tests that don't pass anywhere isn't the best situation to be in either
Alex Crichton (Oct 07 2025 at 16:24):For example I just updated wasi-testsuite in Wasmtime and the http-response.rs test is failing because Wasmtime thinks 600 is a valid response status code but the test asserts that it isn't. Wasmtime's behavior comes from here and don't really have the energy myself to open up issues and figure out what the best course of action is here, but you probably have more knowledge about whether 600 should be invalid for example. It's trivial enough to add more validation to the implementation in Wasmtime, but basically I don't know what's the right thing to do here
Andy Wingo (Oct 07 2025 at 20:35):fwiw the test was passing on wasmtime built from git, at least a week ago; will check tomorrow. i wouldn't have sent it on if it didn't pass :sweat_smile:
Pat Hickey (Oct 07 2025 at 21:32):i hit merge on what i thought was a trivial PR of alex's and then CI failed on main
Pat Hickey (Oct 07 2025 at 21:32):https://github.com/WebAssembly/wasi-testsuite/actions/runs/18326423979
Alex Crichton (Oct 07 2025 at 22:09):Ah no worries! I find github actions' UI isn't great for "this job is allowed to fail" myself... Not sure if that's what happened here though, but should be easy-ish to fix either way
Alex Crichton (Oct 07 2025 at 22:10):er, this was in response to Andy not you Pat, that I have no idea lol
Andy Wingo (Oct 08 2025 at 12:35):yaargh, sorry about that CI thing, will fix
Last updated: Dec 06 2025 at 06:05 UTC