Stream: wasi

Topic: wasi-sockets

Daniel Blignaut (Apr 14 2023 at 14:55):

Hi, seems like there's been quite a lot of movement on the wasi-sockets standard. Is there any public facing work done on implementing this in wasmtime? I'm interested in understanding how it's being implemented and /or contributing towards this.

Dan Gohman (Apr 14 2023 at 15:17):

https://github.com/bytecodealliance/preview2-prototyping is where we're working on a host implementation for wasi-sockets. It's early, but it's complete enough to accept and connect and send and receive

Dan Gohman (Apr 14 2023 at 15:20):

In particular, the full socket+bind+listen+accept sequence works.

Daniel Blignaut (Apr 14 2023 at 16:32):

thank you. I'll start reading through and getting to grips with it. I haven't read through the full spec but given that there's deviations from "native" socket libs (e.g. getaddrinfo not implemented)... do you see a path where rust libraries like tokio, h2, etc. can be ported to wasm32-wasi targets or do you think there'll need to be major changes to add wasm target support?

Dan Gohman (Apr 14 2023 at 16:36):

Yes, the vision is that wasi-sockets should be able to look like "normal" sockets to Rust and other languages. Polling/async will need some special considerations, but that's already the case for all platforms out there.

Dan Gohman (Apr 14 2023 at 16:41):

And while wasi-sockets doesn't have a full getaddrinfo, it does have name resolution, which should support an implementation of getaddrinfo which should look pretty normal to applications.

Daniel Blignaut (Apr 14 2023 at 16:43):

yes, I did read and watch your last youtube video talking through the spec so that makes sense to me just wasn't sure how hard the "adapter" layer would be to implement to support popular crates currently used

Dan Gohman (Apr 14 2023 at 16:45):

I'm working on getting a preview2-enabled libc up and running, and then I plan to add sockets support there, which will mean implementing the actual POSIX APIs on top of wasi-sockets. Once we have that, I expect a lot of stuff will Just Work.

Notification Bot (Apr 14 2023 at 16:49):

This topic was moved here from #general > wasi-sockets by fitzgen (he/him).

Hongyu Li (Apr 19 2023 at 05:45):

I noticed that APIs such as sock-bind and sock-listen are imported in the preview-2. I thought that the reason why wasmtime use the pre-open sockets is to decrease the Nondeterminism. How is Nondeterminism considered in these APIs?

I'm also curious about the Nondeterminism in the sock-accept API. Is it handled by the wasm runtime or it is unavoidable and we just allow it?

Dan Gohman (Apr 19 2023 at 11:38):

Historically, it turns out that the lack of bind/connect in WASI is more a result of non-technical circumstances than technical constraints. Some host environments don't need or want bind/connect, but others do, so it's desirable to have them as an option for those that do.

Dan Gohman (Apr 19 2023 at 11:39):

The "just accept" API does have less nondeterminism than full bind/connect, though in use cases that want just accept, I'd say it's more about giving the host more control over the listening socket than about nondeterminism per se.

Hongyu Li (Apr 25 2023 at 15:31):

Thanks for the explanation! BTW, I want to add some RDMA APIs to the wasi, they will bring some nondeterminism. Is there a way to verify that the nondeterminism brought by my APIs is unavoidable or acceptable?

Hongyu Li (Apr 25 2023 at 15:40):

The nondeterminism brought by my APIs can be divided into two parts:

1. Some of the wasi-rdma APIs are also network related. If the wasi-sock APIs which bring nondeterminism are accepted by the community, does it means I don't need to worry about the nondeterminism brought by the wasi-rdma APIs?
2. Meanwhile, the wasi-rdma will also bring nondeterminism due to the shared-memory model and the client-server model. Is this acceptable for the wasi?

Dan Gohman (Apr 25 2023 at 16:18):

For WASI, I think it works best to think about nondeterminism from the perspective of specific use cases.

Dan Gohman (Apr 25 2023 at 16:20):

wasi-sockets has a lot of well-understood use cases which are ok with the nondeterminism inherent to sockets. They can fail, they can take variable amounts of time, they can have varying metadata, and so on, and sockets applications are prepared to accept this.

Dan Gohman (Apr 25 2023 at 16:22):

WASI as a whole has other use cases which are not ok with that kind of nondeterminism, but what we say is, those use cases should avoid using wasi-sockets. The Worlds mechanism lets people define their own virtual environments with just the APIs they need, and exclude those that don't make sense for their use cases.

Dan Gohman (Apr 25 2023 at 16:26):

So if you have a new API, instead of asking "is this level of nondeterminism ok with everyone?", we can ask "do you have enough use cases that are ok with this level of nondeterminism?".

Dan Gohman (Apr 25 2023 at 16:31):

Now, all that said, with any kind of shared memory, the concern I expect WASI as a whole will have is virtualizability, rather than determinism. Shared memory means that messages can be sent via loads and stores which can't be intercepted by an intermediary, so it's not possible to virtualize/wrap/interpose/polyfill/etc. an API that includes exposed shared memory.

Last updated: Nov 22 2024 at 16:03 UTC