Mossaka (Joe) (Aug 08 2023 at 19:34): Mossaka (Joe) (Aug 08 2023 at 19:34):Currently, all the wit-bindgen-<language> are written in Rust to emit language code, given that the parsed WIT AST (in particular, Resolve struct) is in Rust. This poses some challenges to language experts to develop and maintain bindgen because they will have to know Rust well. This motivates the idea of developing bindgen in their own languages consuming resolved WIT packages. The question is how and in what format can we supply resolved WIT packages to bindgen written in different languages.
wasm-tools emits a serialization format (like JSON) for fully-resolved WIT packages as a typed tree structure, so that each bindgen can deserialize to language-specific data structures for consuming. What do you all think?
Lann Martin (Aug 08 2023 at 19:45):Should generators be...components?
Mossaka (Joe) (Aug 08 2023 at 19:49):wait but in order to get components, we will need WIT repr of the WIT AST :exploding_head:
Lann Martin (Aug 08 2023 at 19:50):https://github.com/protocolbuffers/protobuf/blob/v21.3/src/google/protobuf/descriptor.proto#L62 :smile:
Jamey Sharp (Aug 08 2023 at 19:51):Proposing that the component generator be a component feels appropriate for someone having the "Untitled Goose Game" goose as a user-pic :laughing:
Lann Martin (Aug 08 2023 at 19:54):I actually have a use-case for this in the warg registry: JIT generation of bindings for interface packages
Till Schneidereit (Aug 08 2023 at 22:29):@Alex Crichton how difficult do you think this would be? ^ "This" being the introduction of a component interface for wit-bindgen generators
Alex Crichton (Aug 08 2023 at 22:36):Oh definitely doable IMO, and I think it'd be pretty neat to have something like that combined with an otherwise language-native wasm-dep-management solution or something like that
Mossaka (Joe) (Aug 09 2023 at 01:29):@Lann Martin could you please elaborate more about how a component interface for generators would help to solve the challenge mentioned at the top? Isn't this a circular problem that in order to have component in language A, you will need a binding generator, but in order to have binding generator in language A, you will need to have a component?
Lann Martin (Aug 09 2023 at 01:36):You can either bootstrap from another language or write the first bindings manually. In practice I think you'll end up writing a lot of initial codegen samples manually anyway.
Lann Martin (Aug 09 2023 at 01:43):Anyway these aren't mutually exclusive ideas; a codegen component could emit JSON as easily as anything else.
Randy Reddig (Aug 09 2023 at 21:03)::wave: I was chatting with @Mossaka (Joe) in the Go Slack about this topic, so he raised it here. I’m interested in helping out with the Go interfaces generated by wit-bindgen.
It struck me as somewhat similar to the Protobuf tool protoc, which can use external tools to generate language-specific bindings for a Protobuf description. The Protobuf tooling to generate Go bindings is written in Go, so it can use the Go types and AST packages, as well as code formatting and verification.
As Joe pointed out in the OP, it means that a Go developer can (more?) easily contribute, as the tooling to emit Go is written in Go. I wondered if the same would be possible for wit-bindgen.
Reading the source code to wit-bindgen, I noticed a Markdown emitter. Guessing here, would it be possible to emit a JSON description of the WIT that another program could consume (e.g. wit-bindgen-go)? I’m not very familiar with Rust, but is there an idiomatic way to emit JSON from a data structure?
Thanks!
Christof Petig (Aug 10 2023 at 05:53):Wit-Bindgen consists of an input parser (or several if you also want to support binary representations of wit), an abstract machine to lift and lower arguments (wasm-tools' cabi create, soon to be moved to wit-bindgen repo) and a code generator with multiple layers.
I recommend to share the first two across languages, even though the canonical ABI crate's API is quite complex, but sufficiently documented in rustdoc (it took me some time to find this).
(I am working on creating c++ bindings for both sides and had to dig deeply to learn to value this architecture, but to me switching to c++ was never an option :grinning_face_with_smiling_eyes:)
Christof Petig (Aug 10 2023 at 05:59):Oh, I should probably mention that Alex plans to rewrite the cabi code to generate lifting and lowering functions (there is an issue in wasm tools and wit-bindgen describing this plan), this should make the generated code more easy to understand (also by formatting tools).
Christof Petig (Aug 10 2023 at 06:01):https://github.com/bytecodealliance/wasm-tools/pull/1149
Lann Martin (Aug 10 2023 at 19:47):What if we followed the LSP approach, allowing (bootstrap) bindings generators to be written as WASI commands that speak JSON-RPC over stdio? This would allow reuse of generation code via method call while reducing the bootstrapping requirements to "just" supporting WASI.
Lann Martin (Aug 10 2023 at 19:49):(this could also run native binaries without WASI, but for my pet use-cases I'd like them to use wasm :smile:)
Mossaka (Joe) (Aug 10 2023 at 21:50):Hey there, I've raised an issue on the wit-bindgen repo as I think this is valuable to all. https://github.com/bytecodealliance/wit-bindgen/issues/640
Mossaka (Joe) (Aug 10 2023 at 21:58):What if we followed the LSP approach
I like this idea!
Randy Reddig (Sep 26 2023 at 15:34):Follow up on this thread: @Mossaka (Joe) and I have made some progress: JSON serialization support was merged into the wit-parser crate, and we have a Go implementation that can ingest the WIT JSON and re-emit WIT as a proof of concept: https://github.com/ydnar/wasm-tools-go
Last updated: Nov 22 2024 at 17:03 UTC