Stream: wamr

Topic: WAMR socket API

Marcin Kolny (Jul 11 2022 at 09:38):

Hi all,
My team at Prime Video is looking to use WAMR sockets; I noticed there's already a decent support for IPv4 TCP socket, but we identified a few gaps:

• missing implementation for NS-lookup
• lack of ipV6 support
• no implementation for some of the UDP sockets (mainly - missing sendmsg/recvmsg and/or sendto/recvfrom)

I'd like to ask if there's any plan to implement those features any time soon? And if not, would you be open for PRs from our team with those features implemented? Many thanks

Wang Xin (Jul 11 2022 at 13:09):

Marcin Kolny said:

Hi all,
My team at Prime Video is looking to use WAMR sockets; I noticed there's already a decent support for IPv4 TCP socket, but we identified a few gaps:

• missing implementation for NS-lookup
• lack of ipV6 support
• no implementation for some of the UDP sockets (mainly - missing sendmsg/recvmsg and/or sendto/recvfrom)

I'd like to ask if there's any plan to implement those features any time soon? And if not, would you be open for PRs from our team with those features implemented? Many thanks

Hi Marcin, it would be excellent if your team have a plan to submit patches for the missing parts! :smiley:

Marcin Kolny (Jul 11 2022 at 14:30):

Thanks for the reply @Wang Xin , do you already have any early design documents for the APIs or would you rather stick to POSIX as closely as it's possible?

lum1n0us (Jul 12 2022 at 01:19):

There is an unimplemented APIs for NS-lookup. https://github.com/bytecodealliance/wasm-micro-runtime/blob/0f6e5a55a4ec053a185fb877ab1fc304b3882c57/core/iwasm/libraries/lib-socket/inc/wasi_socket_ext.h#L163

recvmsg/sendmsg (https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/core/iwasm/libraries/lib-socket/inc/wasi_socket_ext.h#L87) will be mapped to Wasm functions "sock_send" and "sock_recv" of "wasi_snapshot_preview1"

Marcin Kolny (Jul 12 2022 at 08:58):

thanks @liam for the reply. I already have seen the code you shared, and I wasn't sure if that approach will give us sufficient functionality, therefore asked for some details. Let me clarify that below:

addr_resolve

current signature is __wasi_addr_resolve(__wasi_fd_t fd, const char *host, __wasi_ip_port_t port, uint8_t *buf, __wasi_size_t size):

• port - is it intentional to not to have it as a service (and therefore, use string as a type), just like we have in getaddrinfo? Would that block some of the usecases?
• I can see why we don't want to have hints - in many cases it's probably not necessary, and we can always apply filtering on top of it, while keeping the API simple; one disadvantage is that we might be passing a bit too much data around when it's not necessary
• buf is actually not clear to me - how do you see this implemented? Should we rather try to return an array of datastructures? One approach I considered is that we could ask the caller to allocate the memory for us and pass the size, we fill the memory, and if we have more data to return, we indicate it with additional flag, so the API would be: addr_resolve(_wasi_fd_t fd, const char *host, __wasi_ip_port_t port, addrinfo *buf, __wasi_size_t in_size, __wasi_size_t *out_size, __wasi_size_t *max_size) where:
  • buf is an array of addrinfo objects allocated by user
  • in_size is the size of the buf array
  • out_size is the number of elements inserted to the array
  • max_size in case we can't fit all the values into the buf, we tell user there's more, but they need to provide bigger buffer if they want to get all of that

One thing I'm not sure though is whether the number of addrinfo objects can change between the calls - if so, max_size should be treated more like a hint rather than the source of truth.

mapping sendmsg/recvmsg to sock_send/sock_recv

I think that works very well for TCP where the connection is estabilished between two entities, but it doesn't allow us e.g. to send UDP packets to a specific address. Therefore we need at least send_to / recv_from functions so we can specify the destination; we can also just implement sendmsg/recvmsg as it provides a superset functionality already (e.g linux kernel uses sendmsg to implement sendto) but I don't know if we want to expose such a low-level functionality through WASM.

Please let me know your thoughts on that, I'm really keen on discussing that further with you and work on a solution.

Marcin Kolny (Jul 13 2022 at 12:52):

@liam @Wang Xin if that works better for you, could we setup a call to discuss the interfaces? Thanks

Wang Xin (Jul 13 2022 at 22:41):

Sure, it is great to talk about it over a call. What is your time zone? Liam and I are UTC+8. @Marcin Kolny

lum1n0us (Jul 14 2022 at 03:15):

About addr_resolve:

• port is optional. It is OK to pass 0(zero) in most cases. It is used to handle something like http://example:8080/path. Maybe it is OK to remove it.
• hints is a good one. If targeting to UDP and IPv6, we should provide that kind of information.
• buf could be in heap(malloc()) or in stack(local variables). Wasm runtime or Wasm compiler will keep buf in linear memory and treat it like a i32[].
  Wasm code actually passes buf[0] in functions. Since the current implementation only covers TCP and IPv4, as you suggested, used_buf_size will be helped.

As you see, the current implementation of socket APIs is limited to TCP + IPv4, it will need an expansion about UDP and IPv6. like send_toand recv_from.

Btw, the Wasm community is working on a new proposal about wasi-socket. It will be a good reference for us.

Marcin Kolny (Jul 14 2022 at 07:35):

@Wang Xin @liam thanks both. I've seen the proposal and I think we could borrow from that some ideas, however I think the current implementation in WAMR is a bit different to a proposal; because the proposal is not accepted yet, I'd suggest filling the gaps for now following the current structure, and once the proposal is accepted, we could possibly implement those new APIs and start deprecating the existing ones, but I think it'd be best to discuss it in the meeting.
I'm in the UTC + 1 timezone; would Monday, 18th July at 5pm UTC+8 (which is 10am for me) work for you?

Wang Xin (Jul 14 2022 at 07:41):

18th July at 5pm UTC+8 works for me. @Marcin Kolny

Wang Xin (Jul 14 2022 at 07:41):

could you please send an invitation email to xin.wang@intel.com?

Marcin Kolny (Jul 14 2022 at 07:48):

@Wang Xin I just sent an invite, please let me know when you get it. Also, feel free to forward it to anyone from your team you's interested in.

Wang Xin (Jul 16 2022 at 23:34):

@Marcin Kolny yes, I have forwarded the invitation.

Chris Woods (Feb 16 2024 at 16:18):

Hi Folks, I've been playing around with trying to compile Mongoose (https://github.com/cesanta/mongoose) for WAMR, been running into an issue where select() always returns a -1 (error). Just wondering if this is something anyone else has encountered? Does the WAMR socket API play nice with the WASI SDK's select ?

lum1n0us (May 19 2024 at 07:33):

@Marcin Kolny do u think it is a good idea to upstream socket APIs to WASI (as a part of P1)?

Marcin Kolny (May 19 2024 at 09:12):

I'd be up for upstreaming it, however, it uses wamr-specific interface so I'm not sure if that will be accepted by maintainers. I know e.g. wasmedge has slightly different socket API too

Marcin Kolny (May 19 2024 at 09:18):

@Chris Woods I think the select should work with wamr sockets, but if you have minimal reproducible example, I'd be happy to have a look

Chris Woods (May 20 2024 at 12:14):

100% - I was working on the Mongoose embedded HTTP server, Which I did actually get working, you can see that here : https://github.com/woodsmc/wamr_with_mongoose However it required swapping select for poll which does work with WAMR, at least on Linux. I have to admit I've not tied to reproduce this on Windows / Mac or a RTOS (yet).

Chris Woods (May 20 2024 at 12:19):

@lum1n0us wasn't there a P2 socket proposal too ? I haven't had my coffee yet, but I thought sure there was one?

Marcin Kolny (May 20 2024 at 13:00):

p2 has WASI sockets defined (https://github.com/WebAssembly/wasi-sockets) and it's already implemented in wasi-libc.

lum1n0us (May 29 2024 at 01:12):

it uses wamr-specific interface

@Marcin Kolny I guess you are talking about --addr-pool, --allow-resolve and it is descriptor_table_get_ref() in wasi-p2. Is it?

Marcin Kolny (May 29 2024 at 08:12):

I meant syscalls defined in wasi_socket_ext, e.g. __wasi_sock_accept

lum1n0us (Jun 06 2024 at 01:50):

@Marcin Kolny there is a user case involves sockets APIs. It is using wasi-sdk to generate Wasm modules. Since the case will be published as an official product, the toolchain becomes a considerable concern.

WAMR posix socket support requires compile additional files(like wasi_socket_ext.c) with user's guest language code(has to be C) together. In our case, the user code is written in not-C guest language. So, we need to pack the additional file(wasi_socket_ext.c) into the toolchain, like wasi-sdk, and publish the modified toolchain to customers in this case. It seems like that we have to maintain a toolchain

On the other hand, as we know, wasi-sdk-p2 has the socket feature. And with the p2-p1-adaptor, it is able to run wasm-wasip2 module success.

For wamr, there are ~3 choices to enable socket API in wasm module

• maintain a private toolchain based on original wasi-sdk-p1 but add wamr socket
• implement socket api in wasi-p1 based on wamr socket. may need redesign, since wasi-socket exists.
• use wasi-p2 and p2-p1-adaptor.
• others ...

here is my question, in your opinion, from users' angle, which one of above option is better for a long-term maintain product?

Marcin Kolny (Jun 06 2024 at 06:40):

@lum1n0us based on your options I assume you're in a similar position as my team and you can't make any updates to the native runtime and you have to make sure the wasm file is preview1-compatible?

• In that case, the adapter might be the best option here. However, keep in mind that the support for p2 in wasi-sdk is still in progress; I think sockets are the only module implemented natively in wasi libc, and all other interfaces require p1->p2 adapter. So basically the process is: 1) compile wasi libc p2 target (which in fact will have a bunch of p1 imports) 2) run the p1->p2 adapter to have a CM module with p2 imports only. I don't think there's official way for p1 -> p2 adapter to not generate a CM module, but IIRC @Christof Petig had some prototype working. Once you have that, then you can link the P2 Core module with p2->p1 adapter and have p1-compatible wasm binary. When wasi-libc has a native support for p2, it'd be a bit easier because you can just compile wasi-libc with your program and use wasm-ld (not wasm-component-ld) as linker to get P2 WASM core module directly.
• I'd love to have a socket support in wasi-sdk-p1 based on WAMR sockets API, but I'm not sure if it will be accepted by maintainers (maybe worth having a discussion though) for two reasons: 1) sockets aren't part of p1, so it'd be a bit like wasi threads (which in fact is included in wasi-sdk) 2) there are other runtimes (e.g. wasmedge) that implement non-standard socket APIs and IIRC it's a bit different to WAMR, so the change will be very WAMR specific. As I said, I'd love to have it and perhaps it's good to have a discussion with the others about it (IIRC @Chris Woods wanted to have sockets api in P1 too).
• I think maintaining a private toolchain will likely come with maintenance overhead, however, one thing you could do to reduce it is perhaps leaving the symbols unresolved, but share with your customer a pre-compiled (to wasm) a version of wasi_socket_ext so they can use it at link time

Please let me know what are your thoughts and whether you'd be interested in collaborating on the adapter so I can share a bit more details and some of the challenges that we'd need to overcome.

lum1n0us (Jun 07 2024 at 01:54):

Thanks for your helpful opinions.

Lucky me, the project is in its early stage. I have my chance to make the best choice.

I agree that a private toolchain will likely come with maintenance overhead. So I guess our choices are only wasip1 or wasip2.

About wasi-sdk-p1 socket support, since there is wasi-socket proposal and it's been in p2, does it mean we have to make a similar implementation or else we have to create/pass a new proposal(for p1 socket, because every functionality should have a relative wasi-proposal )?

Marcin Kolny (Jun 07 2024 at 08:00):

I think wasip1 is considered frozen right now and any new proposals are likely to be rejected. Having said that, I'd be supportive for having sockets API as part of p1 if you'd like to give it a try and propose it to the community.

lum1n0us (Jun 11 2024 at 01:54):

Once the adapter is in usable state, we'll likely abandon the support of preview1 in the tooling.

@Marcin Kolny Since :up: , would u mind sharing the status of the adapter?

Marcin Kolny (Jun 11 2024 at 08:27):

@lum1n0us there's not much beyond just a proof of concept shared on github. We didn't invest much in that yet for two main reasons:

1. p1 is going to be in wasi libc for some time
2. there isn't even an offical support for p2 core modules in wasi sdk (and I guess there won't be one for a while, until wasi libc is fully based on p2 interfaces)

I think sooner or later we'll need to get back to it, but it's not that urgent for us at that stage. Having said that, if you and your team are willing to work on the adapter, we can certainly collaborate.

Last updated: Nov 22 2024 at 16:03 UTC