Hi, I'd like to implement a waker for async IO, but there's nothing similar to pipe(2) or eventfd(2) defined in preview2/wasi-io. I wonder if I missed anything.
nope, no such thing exists in WASI. could you describe more what problem you are trying to solve? maybe there is another way to go about it
just to expand a little: one way of thinking about this is, there is not (yet) a way to have more than one stack of control (thread, coroutine, whatever) per store with wasi. so there are no facilities for waking another stack/thread, since there can only be one
when that restriction is lifted by landing threads and/or async (multiple stacks) in the component model in the 0.3 timeframe (roughly a year out, maybe longer), it will come with facilities for waking another thread/stack
Hi Pat, I want to make a async runtime on the guest side based on tokio/mio. The problem is that no waker is implemented for WASI at the moment, meaning tasks won't be wake up. It is often implemented via eventfd(2)
on Linux, or pipe(2)
in some other situations.[1] I tried to come up with something[2], but the interface is awful. I wonder if there was a discussion regarding the usefulness of the interface, as I vaguely remember the pipe interface was in wasi-io in the early days.
[1]: https://github.com/tokio-rs/mio/blob/master/src/waker.rs
[2]: https://github.com/tokio-rs/mio/pull/1662
I'm not sure if I fully understand the multi-stacks, but seems async tasks with single thread works for me with my mio waker implementation(with wasmtime-9).
hmm. well, i havent tried to do that, but i suspect that the dependence on wakers is because tokio assumes a multi-threaded runtime, and wakers are the way to notify readiness between those
oh ok, so if you're on wasmtime-9 you're using wasi preview 1 stuff
have you looked at preview 2 yet?
we changed a number of things to make it possible to wait on more kinds of things, but fundamentally there is still just the equivalent of poll_oneoff
where the sole thread/stack of control is suspended until one of the arguments is ready
again, im not really in touch with tokio on a deep enough level to know this for sure, but my suspicion is that mio
is the wrong abstraction entirely for implementing a runtime on top of preview 2
In case it's of interest, I created experimental forks of Rust, mio
, tokio
, and tokio-postgres
based on wasi:sockets@0.2.0
, with a demo here: https://github.com/dicej/wasi-sockets-tests
its interesting to me! i should look how you handled this stuff joel
ah yeah so if you're using sockets on top of wasi-libc already then theyre back to being an fd so these mio changes make sense
https://github.com/tokio-rs/mio/compare/master...dicej:mio:wasi-sockets
It's been a while, and I'm just picking up this project again, only walked through the wasmtime's changelog, haven't upgraded to preview2 yet. Not able to use stock tokio was a problem for us, so I wanted to try if I can improve the waker implementation if there's any chance.
Even with poll_oneoff
, the a pair of fd could be used by tasks to "wakeup" each other because those fd are connected on the host side.
Thanks Joel, it looks interesting, I'll look into it in a while.
If you just need two async tasks within a guest to talk to each other, e.g. https://docs.rs/futures/latest/futures/channel/index.html plus either Tokio or a bare-bones executor should work fine. If you need multiple guests to talk to each other, a TCP connection to localhost could be an option.
Yeah, with customized hostcalls, it shouldn't be a problem. But I'd like to make something that fits mio's waker interface, e.g. Waker::new()
does some magic internally. Socket will be cumbersome for this task I'm afraid.
im not sure what you mean by "with customized hostcalls, it shouldnt be a problem" - the suggestion joel had, using futures, doesnt rely on the host at all. the futures
implementation is in terms of atomics and (synchronous) locks, which are implemented in std
and would boil down to regular memory and no-ops respectively because there arent multiple threads in wasi
i think waker is a red herring here - the docs here https://docs.rs/mio/latest/mio/struct.Waker.html say waker is for cross-thread waking of Poll, and there is no such thing as cross-thread waking in this context
Sorry, I was still thinking about fitting into mio's interface, so it's easy to port "normal" program onto WASI, instead of changing how tasks communicate. Not sure if the single thread assumption will break by wasi-threads in the future.
wasi-threads is a dead proposal, its not a design that will move forward. the folks who worked on that have changed their focus and are working on a new core wasm thread proposal that solves the problems that wasi-threads uncovered. the component model will get threads once that work is complete, and when it does this story will change. thats a year out, maybe more.
Oh that's good to know, I actually waited I could get wasi-threads working to reduce duplication with shared memory. I'll check the new component model later. For now I'm going to assume WASI is a single threaded env, and hope it won't shoot my foot in the future.
Thank you both!
anb has marked this topic as resolved.
yep, when that assumption about wasi changes you can count on having a big effort from wasi contributors bringing the ecosystem along. so even if you do end up encountering it one day, it wont be alone!
Last updated: Dec 23 2024 at 12:05 UTC