Yoshua Wuyts (Nov 19 2023 at 11:15): Yoshua Wuyts (Nov 19 2023 at 11:15):Hey all, I wrote a (short) post today about WIT, and how a change in how we expect people to interact with it may change the way we evaluate design tradeoffs. I've talked with some folks about this over the past few weeks, and I figured it might be interesting to write my thoughts down for others to read too.
Rethinking WIT as (primarily) a machine format
Yoshua Wuyts (Nov 19 2023 at 11:18):(In case anyone is worried about the post prior to reading it: I'm not pressing for any immediate calls to action in this post. It's mostly musings about perspectives and how a reframing might lead us to evaluate decisions differently. None of this should be directly relevant to our ongoing Preview 2 sprint.)
Yoshua Wuyts (Nov 19 2023 at 11:19):On a meta-note: is this a good channel to post things like this? I figured this was directly relevant to the component model and this felt like the best channel for that?
Till Schneidereit (Nov 19 2023 at 12:43):@Yoshua Wuyts thank you for writing this, and sharing it here! :heart:
For some historical context, I originally started pushing for a richer syntax for WIT because I came to the conclusion that it'd be too highly visible for any foreseeable future for s-expressions to be sufficient. I strongly believe that structured syntax is important for usability of any text format that is read by humans to a meaningful degree—in particular if the people producing and consuming it aren't necessarily deep experts in the system behind the format.
Based on that, I don't agree with your framing of WIT as (primarily) a machine format. However! I entirely agree with everything you say in the post about how the ecosystem should ideally develop. In particular, deep integration of Components into existing workflows, ecosystems, and platforms are without a doubt crucial to the component model being successful. For that reason I very much like the term "meta-ecosystem" for what we're building with Components and component tooling: we should have a layer of tooling that is low-level and language-/platform-agnostic enough to layer on top of in ways that work well for as many language and platform workflows as possible. Producing and consuming components should indeed feel no different from producing and consuming artifacts, modules, etc that are native to the respective ecosystem.
In practice that means when using Rust, having as workspace of components that can be linked together should feel the same as having a workspace of (non-component) crates as much as possible. Same for JS and npm, Python and PyPI, Ruby and Gems, etc. And yes, I agree that that should as much as possible mean that when in Rust, you should write Rust and a WIT file should be derived from that, and same for JS and d.ts files, etc.
Your example of auto-generating interfaces for different languages as part of component docs is something we've long wanted to have, and I strongly believe it should be a key part of warg registries: you should be able to set as a global flag what language to show all interface docs in, and what package manager to use in showing instructions for how to add a component as a dependency, etc.
And I think we're in pretty good shape for all of this. Restrictions like kebap-case-plus-acronyms are meant just for this, as is the requirement to name return values if there's more than one of them. I think the one key thing we're missing from WIT right now to make this work as well as possible is to define a structure for doc comments that will enable us to convert them into language-native doc comments with high quality.
Till Schneidereit (Nov 19 2023 at 12:47):so why am I saying that I disagree with the title? Because of what I mentioned at the beginning: I believe that for any foreseeable future too many people will be exposed to WIT directly for us to not treat human producers/consumers as a first-class concern for the format. I think it needs to be both: a human-readable format, but with enough structure enforced to derive other formats with highest fidelity. And I also don't think these requirements are at odds with each other. A key reason is that we don't have to do things like optimize for size or processing speed. And we can afford to make it a little more complicated to write good tooling, if that's what's required to retain some property that's only useful for humans. We always have the isomorphic binary encoding that can be fully optimized for machines
Yoshua Wuyts (Nov 19 2023 at 13:16):(reading what you wrote here)
Yoshua Wuyts (Nov 19 2023 at 13:17):heh, yeah I guess the title was the part of the post I was least sure about
Yoshua Wuyts (Nov 19 2023 at 13:58):(got a phone call mid-reading the comments; continuing now)
Yoshua Wuyts (Nov 19 2023 at 14:00):Heh, okay read it all
Yoshua Wuyts (Nov 19 2023 at 14:00):Yeah I basically agree with everything you wrote
Yoshua Wuyts (Nov 19 2023 at 14:00):I feel like maybe the title of my post is somewhat at odds with the actual content of it?
Yoshua Wuyts (Nov 19 2023 at 14:05):I strongly believe that structured syntax is important for usability of any text format that is read by humans to a meaningful degree—in particular if the people producing and consuming it aren't necessarily deep experts in the system behind the format.
This makes a lot of sense to me. Having used WinMD for the windows-rs project; I'm so glad WIT is actually easy to understand by humans. Needing to load ILSpy to do even the most basic introspection into WinMD definitions really didn't feel good. And I'm glad we didn't take that path!
Yoshua Wuyts (Nov 19 2023 at 14:12):Or wait, I guess I wasn't decompiling winmd definitions directly - but using it to extract the definitions from shared libraries. That's not quite the same. Though I'm happy WIT is written with humans in mind nevertheless :sweat_smile:
Yoshua Wuyts (Nov 19 2023 at 14:16):Anyway, changed the title from "rethinking WIT" (implying a potential big change) to "reframing WIT" (merely a change of perspective). And I've added a clarifying note at the start of the post. I hope that will make the title be more in line with the actual post.
Till Schneidereit (Nov 19 2023 at 15:48):To emphasize that again: I also basically agree with everything you wrote! I wanted to give some context on where WIT came from, why it has the shape it has, and what I personally believe is critically important about its function and how it functions. I guess to me the key part is that people generally associate "machine format" with "prioritizes ease of parsing and semantic (machine) analysis over human consumption", and that's the part I feel strongly mustn't be the case for WIT.
A recent example of where this comes into play is adding required semicolons. The initial motivation was to make parsing easier, and I was against this change based on that motivation. However, @Luke Wagner made the great point that semicolons also make it much much easier to extend the syntax later on—which makes them directly supportive of what I think should be the most important property
Yoshua Wuyts (Nov 19 2023 at 17:49):I appreciate you adding context!
Victor Adossi (Nov 22 2023 at 02:45):Awesome blog post -- it was a great read, I really like the idea of bridging that UX gap between WIT and the code by making WIT code-driven. It's really exciting to think about and I think it maps pretty cleanly in Rust to one more more proc macros +/- build.rs stuff.
Maybe a reasonable first step to thinking about how this would work is gathering places where other IDLs went code-first, and trying to map the feature sets/matrix
The other thing I think we should explore is a native wit-doc tool which can be used to render WIT definitions to HTML. My thinking is that this should be modeled loosely after rustdoc or Swift's DocC.
Are you aware of wit-bindgen's markdown generator?
Victor Adossi (Nov 22 2023 at 03:07):.75 seconds of googling and there's ASP.NET's code-first gRPC. I also remember some Java Spring stuff that was "code-first" (and painful) from a past life, but there are probably a bunch of other reasonable examples
Yoshua Wuyts (Nov 22 2023 at 08:18):Oh I had not seen the markdown generator. That’s useful, thank you!
Last updated: Nov 22 2024 at 16:03 UTC