Stream: wasi

Topic: WASI spec changes via wasmtime


view this post on Zulip Dave Bakker (badeend) (Sep 17 2023 at 08:47):

I would like to make some changes to the error handling code in wasmtime's wasi-sockets implementation. Mainly to simplify mapping back-and-forth between the BSD error codes. As part of this change I would also like to change some error codes defined in the wasi-sockets proposal (which I'm the champion of).

I've noticed with other WASI proposals, that sometimes changes are first made in the wasmtime repo with updates to both the implementation _and_ their copy of the spec (/crates/wasi/wit/...) within the same PR. And then later the proposal repo is updated, retroactively.

With the changes I have in mind, this seems the most pragmatic approach because they're implementation driven.
Are there any guidelines on this practice for me to follow?

view this post on Zulip Pat Hickey (Sep 17 2023 at 20:09):

for a while we let the wit in wasmtime and the spec diverge pretty far, because we were in a "prototype" stage, but since we are now getting close to shipping p2, ive been trying to make a PR to wasmtime with local spec + implementation changes, and simultaneously make a PR to the spec repos upstream with whatever changes, and I leave links between the two issues

view this post on Zulip Pat Hickey (Sep 17 2023 at 20:11):

imo, specs that haven't yet had implementation-driven changes are not yet mature specs. its only by implementing and testing the specs, and iterating based on what we learn there, that specs become mature. so, be glad we are at this stage!

view this post on Zulip Dave Bakker (badeend) (Sep 17 2023 at 20:28):

Alright, thanks! The past few days I've been working on a test suite for the TCP socket implementation, along with some associated tweaks to the implementation. I hope have a PR ready somewhere the coming week.

view this post on Zulip Pat Hickey (Sep 17 2023 at 20:29):

oh, wrt errors, i have mentioned this noodling to dan and others but i dont think to you

view this post on Zulip Pat Hickey (Sep 17 2023 at 20:29):

so, first off, we redesigned output-streams to have last-operation-failed and closed as distinct errors

view this post on Zulip Pat Hickey (Sep 17 2023 at 20:30):

i want to do the same thing to input-stream, especially this is informed by implementing http stuff

view this post on Zulip Pat Hickey (Sep 17 2023 at 20:30):

the other thing i want to do, after we switch over to resources, is to make last-operation-failed have a payload stream-error-cause resource

view this post on Zulip Pat Hickey (Sep 17 2023 at 20:31):

and that has a method describe() -> option<string> that gives debug info, and the docs say you have to promise to not use that for control flow, just for displaying debug info, so that we dont get into the bad game of strings needing to be identical across engines/implementations

view this post on Zulip Pat Hickey (Sep 17 2023 at 20:31):

and then we can also have sockets be able to elaborate it into some more particular error

view this post on Zulip Pat Hickey (Sep 17 2023 at 20:32):

e.g. a resource tcp { elaborate(stream-error-cause) -> option<more-specific-error-enum> }

view this post on Zulip Dave Bakker (badeend) (Sep 17 2023 at 20:32):

Good to know! The changes I have so far don't impact streams, but rather the methods on the socket resource themselves (bind, accept, listen, connect, etc...)

view this post on Zulip Pat Hickey (Sep 17 2023 at 20:33):

returning the option because its ok for the tcp to say "i dont know anything about that error" because perhaps it came from some other place

view this post on Zulip Pat Hickey (Sep 17 2023 at 20:33):

ok! good to know

view this post on Zulip Pat Hickey (Sep 17 2023 at 20:33):

i think for http we want this to be able to elaborate on what sort of protocol errors happened

view this post on Zulip Pat Hickey (Sep 17 2023 at 20:33):

we havent worked out all the details yet

view this post on Zulip Pat Hickey (Sep 17 2023 at 20:34):

it seems more useful when you have some sort of more complex protocol that is being used via a stream, vs files and sockets which are low level enough where theres really not many reasons a stream can go wrong

view this post on Zulip Dave Bakker (badeend) (Sep 17 2023 at 20:35):

Just for my information, is this an inbetween solution until proper Preview3 streams are a thing?

view this post on Zulip Pat Hickey (Sep 17 2023 at 20:35):

i think so, yeah

view this post on Zulip Pat Hickey (Sep 17 2023 at 20:36):

the way i see preview3 streams right now is that theyre something we need one day, but we cant really design them until we see how the preview 2 streams end up getting used in implementations

view this post on Zulip Pat Hickey (Sep 17 2023 at 20:37):

so rather than anticipate what theyre going to look like, we're going to try to make preview 2 streams work as nicely as we can with all the things we need to implement right now, and then figure out the path to preview 3 when the time comes.

view this post on Zulip Dave Bakker (badeend) (Sep 17 2023 at 20:37):

make sense

view this post on Zulip Pat Hickey (Sep 17 2023 at 20:37):

same basic attitude as i said above, that a spec is only mature as you iterate on it with the things that come up in implementation.

view this post on Zulip Dave Bakker (badeend) (Sep 17 2023 at 20:42):

Can there be multiple last-operation-faileds on a single stream without having to terminate the stream?

view this post on Zulip Pat Hickey (Sep 17 2023 at 20:42):

nope - we're defining streams that, once theres any failure, theyre closed forever

view this post on Zulip Pat Hickey (Sep 17 2023 at 20:42):

we havent found any cases where that didnt work

view this post on Zulip Dave Bakker (badeend) (Sep 17 2023 at 20:43):

I have two :P

view this post on Zulip Dave Bakker (badeend) (Sep 17 2023 at 20:52):

Instead of having an accept method you'd need to continuously poll, I'd love to have a way to (somehow) have listen return a stream of newly accepted client socket resources. The problem is that accept can return transient errors like ECONNABORTED. This can be an indicator to the application that the listen queue is filling up. At the moment the spec dictates that these temporary error must be skipped over, precisely because of the envisioned fail-once/closed-forever semantics.

view this post on Zulip Dave Bakker (badeend) (Sep 17 2023 at 20:53):

An other is UDP send&receive. Those are currently methods sending individual datagrams, but "ideally" I would like to have a stream of datagrams in the future

view this post on Zulip Dave Bakker (badeend) (Sep 17 2023 at 20:58):

I understand that those use cases are not relevant at the moment, because we're only dealing with byte streams. But they might become relevant in the future?

view this post on Zulip Dave Bakker (badeend) (Sep 17 2023 at 21:01):

Alternative for the accept stream specifically could be to explicitly model errors as messages in the stream. Eg. listen() -> stream<variant { temp-error(code), new-client(sock-resource) }>

view this post on Zulip Dave Bakker (badeend) (Sep 17 2023 at 21:04):

But I don't know if that alternative generalizes well for cases where the error may occur while writing/pushing items to a stream. (The UDP Send stream example)

view this post on Zulip Pat Hickey (Sep 18 2023 at 01:19):

ah, yes, i see what you mean now. i think as we generalize streams to arbitrary datatypes we'll be able to support what you describe

view this post on Zulip Pat Hickey (Sep 18 2023 at 01:20):

we definitely have places where that sort of semantics is required, e.g. readdir's pseudo-stream

view this post on Zulip Pat Hickey (Sep 18 2023 at 01:21):

but for bytestreams, we havent found that yet.

view this post on Zulip Dan Gohman (Sep 18 2023 at 15:51):

@Badeend With Preview 3's typed streams, we can have the stream element type be a result, which would be a way to report accept errors without having the stream as a whole fail.

view this post on Zulip Dan Gohman (Sep 18 2023 at 15:52):

The same applies to modelling UDP datagrams as streams.

view this post on Zulip Dave Bakker (badeend) (Sep 18 2023 at 18:53):

Yeah, that's more or less what I thought too (above)

view this post on Zulip Dave Bakker (badeend) (Sep 18 2023 at 18:54):

I'll change the implementation & spec to allow for these errors


Last updated: Dec 23 2024 at 12:05 UTC