Joel Dice (Dec 01 2023 at 15:11): Joel Dice (Dec 01 2023 at 15:11):So it turns out the wild success I reported earlier porting mio and tokio to WASI Preview 2 wasn't so wildly successful after all. Simple examples work, but things break when they get more complicated. After an evening of debugging, I learned a few things:
mio really wants everything to be edge-triggered. Native unix functions like epoll and kqueue can be configured for either edge- or level-triggering, and of course mio uses the edge option. It also supports the more traditional POSIX poll function, which only supports level-triggering, but mio apparently converts that to edge-triggering with some fancy juggling that I don't completely understand yet.mio already supports WASI Preview 1's poll_oneoff, but it apparently assumes it is edge-triggered. I don't think that's true, though, is it? This is a source of great confusion for me, because I was modeling the Preview 2 support on the poll_oneoff support.wasi:io/poll#poll function is definitely level-triggered, so it needs the same treatment that POSIX poll gets. Without it, mio will keep polling for both write- and read-ability and wasi:io/poll#poll will keep returning immediately because "yes, the handle is still ready for writing like I just told you 1000 times already".I'm going to keep cranking on this, but I could use some help answering this question: Is it correct to say that both Preview 1's poll_oneoff and Preview 2's wasi:io/poll#poll are level-triggered, i.e. if you ask either of them whether a socket is ready for writing, it will keep saying "yes" no matter how many times you ask it unless you actually do enough writing to trigger backpressure?
Alex Crichton (Dec 01 2023 at 15:13):Is it correct to say that both Preview 1's poll_oneoff and Preview 2's wasi:io/poll#poll are level-triggered
Yes
Joel Dice (Dec 01 2023 at 15:14):Ok, so mio's existing WASI support is buggy and I should just ignore it, sounds like.
Alex Crichton (Dec 01 2023 at 15:15):Trying to think more on this, but yeah if mio expects edge triggered and preview1 "just" calls poll then sounds like you'll need to ignore it
Alex Crichton (Dec 01 2023 at 15:15):I'm not sure how widely used the preview1 integration was to flesh out bugs like this
Alex Crichton (Dec 01 2023 at 15:16):I'd have to dig more into exactly what mio wants here to help more though, I would not have expected that level-triggered would break things
Joel Dice (Dec 01 2023 at 15:35):Alex, Lann, and I did a quick call to dig into this. It looks like mio's support for both poll_oneoff and POSIX poll are broken in subtle ways, which is what was causing my confusion. I'm going to try to fix the POSIX bits and totally replace the WASI bits.
Joel Dice (Dec 01 2023 at 16:12):Correction: looks like mio's POSIX poll support is not broken after all. I just didn't understand how it worked in the context of how it's used. Looks like it will be a good pattern to follow for WASI Preview 2 support.
Alex Crichton (Dec 01 2023 at 16:16):Oh? How was the interest between read/write split given what we were looking at?
Joel Dice (Dec 01 2023 at 16:22):initially, the mio code registers both read and write interest. Then, when one of those triggers (say write readiness, for example), the selector internally deregisters write interest while keeping read interest enabled. Then it unblocks tokio to do its write, if applicable, which will cause the IoSourceState interceptor to re-register both interests, which will cause the next poll to wake immediately, at which point the selector will deregister write interest again, which gives tokio a chance to write again, but this time it doesn't need to, so we go back and poll again, this time with only read interest, and we stay there blocked until there's something to read.
Joel Dice (Dec 01 2023 at 16:23):so technically there's still a very minor inefficiency where we poll one extra time redundantly, but only once per useful I/O operation; there's no unbounded busy waiting
Alex Crichton (Dec 01 2023 at 16:33):ah ok I can sort of see that
Joel Dice (Dec 01 2023 at 21:20):I've updated my mio fork with a rewrite of the WASI selector per the above discussion, and things are working nicely now. I've also added a somewhat more complicated test using tokio-postgres to the test suite.
Last updated: Nov 22 2024 at 16:03 UTC