Yoshua Wuyts (May 27 2024 at 15:49): Yoshua Wuyts (May 27 2024 at 15:49):@Luke Wagner @Alex Crichton question: should individual members on variants/enums be tagged with a @since gate? - If this was Rust I'd say: "yes", since we have the #[non_exhaustive] attribute which sets enums up to be forward-compatible. But I'm not sure what the thinking is for WIT around this. I don't think we have an equivalent attribute right now?
Yoshua Wuyts (May 27 2024 at 15:50):I can see this being relevant for things like error cases which we may want to extend later. Or for example in the case of HTTP there have been a number of new verbs added via RFCs over the years.
Yoshua Wuyts (May 27 2024 at 15:51):Right now the wit-tools patch adding the @since gates does not seem to support it, so I guess I was wondering what our thinking here was.
Yoshua Wuyts (May 27 2024 at 16:18):Oh I guess @since also can't be added to individual struct fields right now either
Luke Wagner (May 27 2024 at 16:19):Yep, that's right
Yoshua Wuyts (May 27 2024 at 16:20):Uh oh, I should have re-read the doc you wrote, shouldn't I?
Yoshua Wuyts (May 27 2024 at 16:20):(I'll go do that now haha)
Alex Crichton (May 28 2024 at 16:16):I think the difference between Rust and WIT types here means that it doesn't make sense to put individual annotations on variants/fields because subtyping in the component model considers two structurally-the-same types equivalent, meaning that you don't have the same level of privacy in WIT that you can get in Rust
Lann Martin (May 28 2024 at 16:20):In other words, adding a variant is never backward compatible today, right?
Alex Crichton (May 28 2024 at 16:20):correct, yeah
Yoshua Wuyts (May 28 2024 at 16:45):okk, I see! ty
Yoshua Wuyts (May 28 2024 at 16:49):that does seem tough though - If I'm understanding it correctly, that would then mean that post-WASI 1.0, if a new HTTP verb would be published, we wouldn't have a way to add it to wasi:http without breaking the interface. I guess I was expecting at least some form of forward-compatibility?
Alex Crichton (May 28 2024 at 16:49):that's correct, and that's more-or-less just how the component model works
Alex Crichton (May 28 2024 at 16:50):we'd have to add new types for extenable-enums or extenable-structs to the component model to allow for such a use case
Alex Crichton (May 28 2024 at 16:50):and/or a different versioning story
Yoshua Wuyts (May 28 2024 at 16:53):Alex Crichton said:
and/or a different versioning story
by that you mean: "WASI 1.0 doesn't mean no breaking changes forever after"? - e.g. folks would expect to work with potentially a 2.0, 3.0, etc.
Alex Crichton (May 28 2024 at 16:55):IMO we're not far enough along to really answer that
Alex Crichton (May 28 2024 at 16:55):no one wants continual breaking changes after a "one dot oh" no matter what
Alex Crichton (May 28 2024 at 16:55):so that's at least the guiding light to me
Yoshua Wuyts (May 28 2024 at 16:55):yeah yeah yeah, that was my expectation too - I'm glad we're on the same page ^^
Yoshua Wuyts (May 28 2024 at 16:56):I was mostly trying to understand what you were saying - when you said: "different versioning story" my mind immediately went to: "oh, so.. breaking changes?"
Dave Bakker (badeend) (May 28 2024 at 16:56):The component-model repo used to have a "subtyping" section. The idea is/was that new fields (structs), cases (variants) and parameters (functions) could be added in a backwards-compatible way. I don't know where that section went, though.
Yoshua Wuyts (May 28 2024 at 16:58):oh interesting!
Dave Bakker (badeend) (May 28 2024 at 16:58):Found it. It was removed in Clarify what's in Preview 2. So I don't know if it was just temporarily removed for the preview 2 launch, or if it is a permanent change.
Lann Martin (May 28 2024 at 16:59):a new HTTP verb
There is a solution for this specific problem in place (along with any nonstandard verbs): https://github.com/WebAssembly/wasi-http/blob/a81c61fc584278f801393b22289840a439e51f50/wit/types.wit#L21
other(string)
Alex Crichton (May 28 2024 at 17:04):yeah I can expand by saying that there's a few different ways we could go about making things extensible
Alex Crichton (May 28 2024 at 17:04):e.g. a different way to update versions, a subtyping story, something more general like other(string), all of these are possible in theory
Alex Crichton (May 28 2024 at 17:05):it's moreso that right now no such story exists and one needs to be created to enable a story for backwards-compatibly adding variants/fields
Yoshua Wuyts (May 28 2024 at 17:06):ok, yeah that makes sense to me
Yoshua Wuyts (May 28 2024 at 17:07):knowing that we've got options we can explore in the future is probably the higher-order bit in my question
Yoshua Wuyts (May 28 2024 at 17:07):ty for elaborating ^^
Luke Wagner (May 28 2024 at 17:10):I still very much want to re-add subtyping so that it's possible to, in particular, add cases to variants in a non-breaking fashion. I initially added the Subtyping section because it seemed easy enough at the component-type level. But the problem is that just because a component-function-type f1 is a subtype of f2, that doesn't mean that the source-level bindings are a subtype (or the Canonical ABI which sometimes shows through directly in the source-leve bindings), and thus if we say that a WIT-level change is semver-compatible, it may actually produce source-incompatible, and thus breaking, changes. So in the short-term we just disabled subtyping so we could sort it out more carefully later.
Ralph (May 28 2024 at 17:13):what's the upshot of the current situation? right now, we just need to define a new type?
Luke Wagner (May 28 2024 at 17:15):Yeah, new type used by new function (or make a breaking version change and change the type/function in-place)
Luke Wagner (May 28 2024 at 17:33):Part of the reason to take some time to figure out the right solution is that there are a couple of different paths:
There's one path where we try to support maximal subtyping (and thus API Evolution flexibility) by doing magic in the toolchain to capture the version you compiled against so that from "inside" the component, your bindings stay with a fixed type (even if the interface itself changes) until you intentionally bump it (and then opt into any source-level breaking changes). This seems like a cool sci-fi thing we could do, but it adds a new dimension of versioning and complexity to think about and so perhaps is too magic.
There's a more conservative path of only allowing subtyping when we have a reasonably-thought-out answer for how this form of subtyping can be implemented source-compatibly in all the languages we can think of, so that if source bindings always use the latest minor version, client source code never breaks. But this is a pretty stringent requirement when you consider, e.g., C, which is going to pretty much lay bare the whole Canonical ABI and thus the size of variants/records is an observable part of the interface; in the limit this rules out almost all subtyping. But perhaps there's some smart things we can do here to carve out just enough wiggle room for the cases we need.
Also, a built-in error type (which might be a necessary part of the overall stream feature) that supports a more dynamically-typed form of error case analysis (like anyhow::Error) could subsume a bunch (but definitely not all all) use cases for extensibility in the 0.3 timeframe.
bjorn3 (May 28 2024 at 17:36):But this is a pretty stringent requirement when you consider, e.g., C, which is going to pretty much lay bare the whole Canonical ABI and thus the size of variants/records is an observable part of the interface
If a function returns a struct with 3 fields and you are subtyping this function to one that returns only two fields, as far as the canonical ABI is concerned on the callee side only those two fields really exist and thus contribute to the observable size, right? Both the caller and callee can have a different view of what fields exist. The glue code between components is responsible for adding/removing fields as necessary when translating.
Last updated: Dec 23 2024 at 12:05 UTC