Currently, all the wit-bindgen-<language>
are written in Rust to emit language code, given that the parsed WIT AST (in particular, Resolve
struct) is in Rust. This poses some challenges to language experts to develop and maintain bindgen because they will have to know Rust well. This motivates the idea of developing bindgen in their own languages consuming resolved WIT packages. The question is how and in what format can we supply resolved WIT packages to bindgen written in different languages.
wasm-tools
emits a serialization format (like JSON) for fully-resolved WIT packages as a typed tree structure, so that each bindgen can deserialize to language-specific data structures for consuming. What do you all think?
Should generators be...components?
wait but in order to get components, we will need WIT repr of the WIT AST :exploding_head:
https://github.com/protocolbuffers/protobuf/blob/v21.3/src/google/protobuf/descriptor.proto#L62 :smile:
Proposing that the component generator be a component feels appropriate for someone having the "Untitled Goose Game" goose as a user-pic :laughing:
I actually have a use-case for this in the warg registry: JIT generation of bindings for interface packages
@Alex Crichton how difficult do you think this would be? ^ "This" being the introduction of a component interface for wit-bindgen generators
Oh definitely doable IMO, and I think it'd be pretty neat to have something like that combined with an otherwise language-native wasm-dep-management solution or something like that
@Lann Martin could you please elaborate more about how a component interface for generators would help to solve the challenge mentioned at the top? Isn't this a circular problem that in order to have component in language A, you will need a binding generator, but in order to have binding generator in language A, you will need to have a component?
You can either bootstrap from another language or write the first bindings manually. In practice I think you'll end up writing a lot of initial codegen samples manually anyway.
Anyway these aren't mutually exclusive ideas; a codegen component could emit JSON as easily as anything else.
:wave: I was chatting with @Mossaka (Joe) in the Go Slack about this topic, so he raised it here. I’m interested in helping out with the Go interfaces generated by wit-bindgen.
It struck me as somewhat similar to the Protobuf tool protoc, which can use external tools to generate language-specific bindings for a Protobuf description. The Protobuf tooling to generate Go bindings is written in Go, so it can use the Go types and AST packages, as well as code formatting and verification.
As Joe pointed out in the OP, it means that a Go developer can (more?) easily contribute, as the tooling to emit Go is written in Go. I wondered if the same would be possible for wit-bindgen.
Reading the source code to wit-bindgen, I noticed a Markdown emitter. Guessing here, would it be possible to emit a JSON description of the WIT that another program could consume (e.g. wit-bindgen-go)? I’m not very familiar with Rust, but is there an idiomatic way to emit JSON from a data structure?
Thanks!
Wit-Bindgen consists of an input parser (or several if you also want to support binary representations of wit), an abstract machine to lift and lower arguments (wasm-tools' cabi create, soon to be moved to wit-bindgen repo) and a code generator with multiple layers.
I recommend to share the first two across languages, even though the canonical ABI crate's API is quite complex, but sufficiently documented in rustdoc (it took me some time to find this).
(I am working on creating c++ bindings for both sides and had to dig deeply to learn to value this architecture, but to me switching to c++ was never an option :grinning_face_with_smiling_eyes:)
Oh, I should probably mention that Alex plans to rewrite the cabi code to generate lifting and lowering functions (there is an issue in wasm tools and wit-bindgen describing this plan), this should make the generated code more easy to understand (also by formatting tools).
https://github.com/bytecodealliance/wasm-tools/pull/1149
What if we followed the LSP approach, allowing (bootstrap) bindings generators to be written as WASI commands that speak JSON-RPC over stdio? This would allow reuse of generation code via method call while reducing the bootstrapping requirements to "just" supporting WASI.
(this could also run native binaries without WASI, but for my pet use-cases I'd like them to use wasm :smile:)
Hey there, I've raised an issue on the wit-bindgen repo as I think this is valuable to all. https://github.com/bytecodealliance/wit-bindgen/issues/640
What if we followed the LSP approach
I like this idea!
Follow up on this thread: @Mossaka (Joe) and I have made some progress: JSON serialization support was merged into the wit-parser
crate, and we have a Go implementation that can ingest the WIT JSON and re-emit WIT as a proof of concept: https://github.com/ydnar/wasm-tools-go
Last updated: Dec 23 2024 at 12:05 UTC