Stream: wasi

Topic: Basic WASI HTTP server

Sébastien Deleuze (Mar 22 2022 at 07:55):

@Nathaniel McCallum @Alex Crichton Hi, I try to leverage the brand new WASI sock_accept() and @Harald Hoyer related support in Mio to build a basic WASI HTTP server. I did that by forking and updating an existing one, see https://github.com/sdeleuze/mini_http.

I have reached a point where everything works fine with a regular native build (even while benchmarking with tools like hey) while the WASI version fails with a Bad file descriptor error at (I think) the third invocation of poll.poll(&mut events, None)? here.

I am still in the process of learning Rust, and I had to remove multithreading support to make it work with WASI, so I may have missed something obvious, but since the native version works fine, I would be interested by your feedback. I am wondering if my code need some fix/special handling for WASI or if I should raise a bug somewhere (Rust, Mio, Wasmtime, Wasi ?). Logs and repro steps are described here.

Alex Crichton (Mar 22 2022 at 14:11):

Sorry I haven't been following the socket-related work in WASI so I fear I may not be able to help much here

Dan Gohman (Mar 22 2022 at 15:33):

Would you be able to run it with WASI tracing enabled, by setting the env var RUST_LOG=wasi_common=trace, and post the log somewhere?

Sébastien Deleuze (Mar 22 2022 at 15:55):

Sure, and thanks for the hint. The logs are available at https://gist.github.com/sdeleuze/758cd2959613952c3fb1aafe80f1b42c. I see a message Error: key not in table.

Another information in case that could help: I had a similar error on a simpler example available here, and fixed it by disabling HTTP 1.1 keep-alive. In practice, I changed HTTP headers from Connection: keep-alive to Connection: close and closed the connection with Mio instead of the original code which was reusing the existing connection.

Dan Gohman (Mar 22 2022 at 16:19):

Thanks! I'll take a look in a few minutes here.

Dan Gohman (Mar 22 2022 at 16:57):

One thing that looks suspicious here is that there's a fd_close a little before the "key not in table" on Fd(3), which appears to be the socket fd. Is it intended to close the socket fd?

Sébastien Deleuze (Mar 22 2022 at 17:20):

No, I don't think that's intended since we are in the middle of processing the HTTP request/response.

Based on my 2 examples, I suspect that with WASI some Mio calls to poll.registry().reregister() don't prevent the socket fd to be closed, while on regular native build it keeps it opened as expected. Maybe by comparing the native logs with the WASI ones, it would give you some useful insights. Please let me know if/how I should provide similar logs for the native variant.

Dan Gohman (Mar 22 2022 at 18:36):

When I run the mini_http hello_wasi example, it panics on stdlistener.set_nonblocking(true).unwrap();

Dan Gohman (Mar 22 2022 at 19:04):

Ah, that was because I was building the wasi program with stable rust.

Dan Gohman (Mar 22 2022 at 22:04):

I've debugged this some more; one difference between the native and wasi versions is that the wasi accept fails with EWOULDBLOCK, which sends the mini_http server code down a different path

Dan Gohman (Mar 22 2022 at 23:32):

Wasmtime's poll_oneoff implementation appears to be returning spurious events, which leads mini_http to attempt to accept on a socket which isn't ready, which gets EGAIN/EWOULDBLOCK, which sends the server down this path.

Sébastien Deleuze (Mar 23 2022 at 09:42):

Thanks for digging into this. Please let me know if you would like I create a related issue on Wasmtime side, if a minimal repro would be useful, etc.

Dan Gohman (Mar 23 2022 at 17:47):

Ok, it looks like there are multiple issues here. One appears to be that wasmtime's poll is returning spurious events; I have an experimental patch which fixes this here.

Dan Gohman (Mar 23 2022 at 17:48):

Another is that WASI's poll_oneoff returns separate events for the "read" and "write" subscriptions, where in the native version, they get combined into a single event.

Dan Gohman (Mar 23 2022 at 17:48):

That may be something that needs to be fixed in the mio code.

Dan Gohman (Mar 23 2022 at 17:49):

I have to set this aside for now and work on other things, but I'll be around and can answer questions.

Sébastien Deleuze (Mar 23 2022 at 18:47):

Thanks, I will try to see if I can fix the mio code.

Harald Hoyer (Mar 28 2022 at 15:29):

https://github.com/sdeleuze/mini_http/pull/1

Harald Hoyer (Mar 28 2022 at 15:31):

As I wrote in email, the server should be fixed to handle readable and writeable events separately, because that can happen in real life, too.

Harald Hoyer (Mar 28 2022 at 15:31):

and so the patching of mio would become obsolete.

Sébastien Deleuze (Mar 28 2022 at 15:41):

Thanks @Harald Hoyer I will update the server as suggested.

Last updated: Dec 23 2024 at 12:05 UTC