Wasmtime GitHub notifications bot (May 13 2020 at 15:16): Wasmtime GitHub notifications bot (May 13 2020 at 15:16):alexhultman commented on Issue #70:
HTTP features in a low-level POSIX-like API seems highly out of place - let third-parties implement that stuff on top of the, low-level BSD-like socket syscalls from WASI instead (assuming they are added at some point).
An HTTP server or client is really high-level and can be implemented in a million different ways with tons of different goals and opinions - BSD sockets on the other hand are quite standardized and efficient and a much more logical fit for a low level base WASM API like WASI.
There are about a million web servers and clients ready to compile for WASI as soon as networking syscalls are added - let them figure out how to bring HTTP to WASM.
If you need an HTTP client, wouldn't it makes sense to let the market self-organize and let things like curl or wget come out the victor? I sure would prefer curl-on-WASI, rather than some new HTTP-client from WASI itself.
Wasmtime GitHub notifications bot (May 13 2020 at 17:17):svsintel commented on Issue #70:
Once low level sockets are implemented, then any upper level protocol can work. I can use "fetch" to access external data, but I want sockets to be able to listen.
Wasmtime GitHub notifications bot (May 13 2020 at 17:32):pchickey commented on Issue #70:
We agree that low level BSD-like socket functionality should be part of WASI. The HTTP interfaces we are working on are motivated by two environments (Fastly's Compute@Edge product, and Istio/ATS's proxy-wasm environment https://github.com/WebAssembly/wasi-http-proxy) where low-level socket access is inappropriate or impossible. These interfaces are not intended to replace socket interfaces or displace use of curl/wget in those environments.
Wasmtime GitHub notifications bot (May 13 2020 at 17:44):alexhultman commented on Issue #70:
The fact that we're considering an HTTP API now doesn't preclude doing a plain sockets style API as well.
An HTTP API does not preclude a low level API, _but_ whatever magic you decide on for the high level HTTP API _must_ map logically to the low level BSD-socket APIs (coming later on?).
So if you imagine some _magic_ high level fetch-like async HTTP call, then that _magic_ must eventually map down to FDs and pollsets perfectly, otherwise you're going to get a low level platform that does not logically come together with the high level parts, causing havoc and serious incompatibilities within the WASI universe itself.
A web browser running JavaScript has an implicit event-loop that is used to handle Promises, async/await and async fetches, etc. Here it is easy to add new "magic" because you never have to perfectly define the lower level counterparts. So magic stays magic.
Creating an HTTP API before you know exactly how the low level parts should look like is dangerous. What event-loop are you settling for? How should that event-loop map to polling syscalls? How should it integrate with co-routines, etc. If these things don't come together logically at the low level then you're going to end up with two entirely incompatible ways of doing async stuff.
So for instance, we need to be able to get the FD of that HTTP connection and get the FD of the event-loop that you poll with, otherwise we as low level developers cannot seamlessly integrate with high level async networking.
Starting with the low level first, nailing how the universe should work is a lot more sane than first deciding on _magic_ then retrofitting that magic to low level parts.
Wasmtime GitHub notifications bot (May 13 2020 at 18:10):alexhultman deleted a comment on Issue #70:
The fact that we're considering an HTTP API now doesn't preclude doing a plain sockets style API as well.
An HTTP API does not preclude a low level API, _but_ whatever magic you decide on for the high level HTTP API _must_ map logically to the low level BSD-socket APIs (coming later on?).
So if you imagine some _magic_ high level fetch-like async HTTP call, then that _magic_ must eventually map down to FDs and pollsets perfectly, otherwise you're going to get a low level platform that does not logically come together with the high level parts, causing havoc and serious incompatibilities within the WASI universe itself.
A web browser running JavaScript has an implicit event-loop that is used to handle Promises, async/await and async fetches, etc. Here it is easy to add new "magic" because you never have to perfectly define the lower level counterparts. So magic stays magic.
Creating an HTTP API before you know exactly how the low level parts should look like is dangerous. What event-loop are you settling for? How should that event-loop map to polling syscalls? How should it integrate with co-routines, etc. If these things don't come together logically at the low level then you're going to end up with two entirely incompatible ways of doing async stuff.
So for instance, we need to be able to get the FD of that HTTP connection and get the FD of the event-loop that you poll with, otherwise we as low level developers cannot seamlessly integrate with high level async networking.
Starting with the low level first, nailing how the universe should work is a lot more sane than first deciding on _magic_ then retrofitting that magic to low level parts.
Wasmtime GitHub notifications bot (May 14 2020 at 22:20):alexhultman commented on Issue #70:
Alright I misunderstood what WASI was about. Apparently it's not just low level.
But we still want BSD sockets :)
Wasmtime GitHub notifications bot (May 15 2020 at 09:47):tschneidereit commented on Issue #70:
But we still want BSD sockets :)
As @pchickey said, low level sockets do make sense. I'm not aware of anyone currently driving the process of working out a proposal for a WASI module. If that's something you'd be interested in helping drive, opening an issue in the WASI spec repository might make sense. See the recent issue on WASI-nn as an example. (Though of course something not as fully fleshed out definitely works as a starting point!)
We'd be happy to provide help with implementations in Wasmtime, be they experimental to support the standards process, or production-quality to already serve use cases!
Wasmtime GitHub notifications bot (Jun 02 2020 at 13:46):MikeCamel commented on Issue #70:
Suggest adding labels "wasi-network-sockets" and "feature-requests".
Wasmtime GitHub notifications bot (Jun 08 2020 at 10:56):tschneidereit labeled Issue #70:
We need a full fledged networking API. This includes functions like
socket(),connect(),bind(),listen(),accept(),getsockopt(),setsockopt(), etc.
Wasmtime GitHub notifications bot (Jun 08 2020 at 10:58):tschneidereit commented on Issue #70:
Suggest adding labels "wasi-network-sockets" and "feature-requests".
@MikeCamel we don't have those, and should be careful about adding too many labels. I added the
enhancementlabel and hope at least for now the combination of that andwasi:apishould be enough. If and when thewasilabels become too crowded, we can look into adding a more fine-grained taxonomy.
Wasmtime GitHub notifications bot (Jun 09 2020 at 11:29):MikeCamel commented on Issue #70:
Sounds good, thanks.
Wasmtime GitHub notifications bot (Aug 04 2020 at 12:11):ShadowJonathan commented on Issue #70:
Is there any progress on this front? I was surveying krustlet for usage, but i quickly realized that quite core functionality like networking wasn't implemented all the way back here in
wasmtime, and by extension, WASI.Are there any proposals/issues/requests/PRs that I can follow on WASI's side to see progress regarding this?
Wasmtime GitHub notifications bot (Aug 07 2020 at 13:21):About how BSD sockets capabilites looks like, BASH creates a socket using:
/dev/tcp/www.example.com/80:exec 3<>/dev/tcp/www.example.com/80 echo -e "GET / HTTP/1.1\r\nhost: http://www.example.com\r\nConnection: close\r\n\r\n" >&3 cat <&3
Wasmtime GitHub notifications bot (Aug 07 2020 at 13:29):afsec deleted a comment on Issue #70:
About how BSD sockets capabilites looks like, BASH creates a socket using:
/dev/tcp/www.example.com/80:exec 3<>/dev/tcp/www.example.com/80 echo -e "GET / HTTP/1.1\r\nhost: http://www.example.com\r\nConnection: close\r\n\r\n" >&3 cat <&3
Wasmtime GitHub notifications bot (Aug 07 2020 at 17:01):3miliano commented on Issue #70:
We at Kong we are working on full socket implementation for WASI. We included connect, bind, listen, accept, etc. We also added some security features so the host does have some sort of control about what the WASM code can connect to, listen to, and so forth.
I think it aligns with everything I am reading on this thread.
You can find the proposal here:
https://github.com/Kong/WASI/tree/feat/wasi-sockets
and a potential implementation here:
https://github.com/Kong/wasmtime/tree/feat/wasi-socketsI would love to hear some feedback on it.
Wasmtime GitHub notifications bot (Aug 08 2020 at 12:03):@3miliano that looks great! A couple of random comments below:
- IMO it would be nice if TLS and proxies are handled transparently by the host, rather than the Wasm guest, because those involve host dependent network configuration (such as TLS certificates handling and address translation) and may affect portability of Wasm application. If TLS is supported by the host, address resolution would also need to be done by the host, because the original host name is embedded and checked in the handshake (as SNI).
- Support for multiplexing might also be desirable, for protocols like QUIC.
I was sketching a draft proposal mostly focusing on those aspects, though the other POSIX related stuff is similar to yours.
Wasmtime GitHub notifications bot (Aug 08 2020 at 12:13):ShadowJonathan commented on Issue #70:
@3miliano can we maybe see the link to the pull request into WASI proper once it's up? Interesting proposal, I'll be looking out for it
Wasmtime GitHub notifications bot (Aug 11 2020 at 16:56):3miliano commented on Issue #70:
@ueno On the topic of TLS. Yes, that would be interesting. One thing to consider is that TLS doesn't exactly require system calls, and therefore you can of course compile a TLS library such as WoflSSL, OpenSSL, etc in WASM itself. Having said that, I hear you about the TLS. That might be something to look out for in the future.
I'll take a look at your proposal, maybe we can collaborate on a single solution.
Wasmtime GitHub notifications bot (Feb 03 2021 at 17:11):bjorn3 commented on Issue #70:
Should this #71 and #73 be moved to https://github.com/webassembly/wasi?
Last updated: Oct 23 2024 at 20:03 UTC