Victor Adossi (Aug 03 2023 at 23:55): Victor Adossi (Aug 03 2023 at 23:55):Hi all, I'd like to actually extend the functionality performed by wit-bindgen -- i.e. manipulating the output of wit-bindgen.
Since Rust does not have eagerly-evaluated macros (at the end-user level anyway), this means that to actually get this to work (as far as I was able to try, a tt-muncher macro does not solve this), I need to actually get the TokenStreams from generate.
Rust also prevents you from using the function that represents proc macro functionality, so it seems like a another crate & re-export is necessary (I've seen some tricks here like use crate::* but they didn't work for me)
Would a patch to make the actual token transformation code and relevant structs like Config public be reasonable? I'd like to contribute that if so
Victor Adossi (Aug 03 2023 at 23:57):As an example for a concrete use-case -- injecting derived traits is a good one (::serde::Deserialize/::serde::Serialize) -- it becomes possible to write a macro that does wit-bindgen (without re-implementing parsing opts/doing wit-bindgen) and arbitrary extra functionality.
Another use case which is similar is writing an attribute macro that works with wit-bindgen -- I haven't tried this yet but I imagine something like:
#[my_wit_bindgen_customization(SomeArg)]
wit_bindgen::generate!("some-package")
In that way you could imagine generalizing lots of kinds of modifications/layers on top of the code generated by wit-bindgen
Alex Crichton (Aug 04 2023 at 07:28):I think it would be reasonable to make more things public, yeah, although I also think it's fine to add more features to the macro itself to avoid a second wrapper macro. Either way works
Karel Hrkal (kajacx) (Aug 04 2023 at 07:54):It is indeed possible to manipulate the outgoing (and even the incoming) TokenStream of the macro, but it's kind of wonky. I am actually doing just that here, but it requires "forking" the entire wasmtime-component-macro crate.
The best solution would be to split the wasmtime macro crate into two, so this would become slightly less painful to do.
Another solution that I am using is defining custom types and mappings between these custom types and the wit types. This has the advantage that I can derive any traits I want and implement any additional functions I want as well.
Adding "custom wrappers" to the bindgen macro is possible, but it would just make the macro more complicated, it would make the code using the macro more complicated, and it would most likely not cover all use cases anyway. I think time would be better spent on supporting more guest languages.
Victor Adossi (Aug 04 2023 at 13:55):Alex Crichton said:
I think it would be reasonable to make more things public, yeah, although I also think it's fine to add more features to the macro itself to avoid a second wrapper macro. Either way works
Thanks for the feedback Alex -- so I think in a general case like Serialize/Deserialize it makes sense to upstream (kind of like the ownership option, I dug around there a bunch trying to see if something like this existed), but there's definitely a case for stuff that shouldn't be upstreamed either (project-specific stuff)...
I'd like to pursue both so I'll try and get an official issue & PR up.
Victor Adossi (Aug 04 2023 at 14:00):Karel Hrkal (kajacx) said:
It is indeed possible to manipulate the outgoing (and even the incoming)
TokenStreamof the macro, but it's kind of wonky. I am actually doing just that here, but it requires "forking" the entirewasmtime-component-macrocrate.The best solution would be to split the wasmtime macro crate into two, so this would become slightly less painful to do.
Another solution that I am using is defining custom types and mappings between these custom types and the wit types. This has the advantage that I can derive any traits I want and implement any additional functions I want as well.
Adding "custom wrappers" to the bindgen macro is possible, but it would just make the macro more complicated, it would make the code using the macro more complicated, and it would most likely not cover all use cases anyway. I think time would be better spent on supporting more guest languages.
Thanks for the input! I agree with the approach here.
Concretely I was figuring that the simplest way would be to have some sort of wit-bindgen-rust-macro-utils (-core? -common?) and a re-export of the token-manipulating functions in there (and some other stuff) under wit-bindgen-rust-macro.
Defining custom types is definitely the easiest way (i.e. you could use serde's remote deriving features), but that's a slightly separate goal -- I really want to maintain how hands-off wit-bindgen is for the user, and minimize the differences & code the user has to write.
I agree that adding custom wrappers to the macro definitely makes it more complicated, but I'd like for it to be an option at least -- don't think it would take much work to make it possible, and after that we'll see what the community comes up with. I'm also comfortable making the issue + PR myself as to not burden the team -- much more important work to be done (and my hack is working for me anyway).
Victor Adossi (Aug 22 2023 at 13:47):Just a quick update on this, I made an issue here:
https://github.com/bytecodealliance/wasmtime/issues/6881
If anyone has some input that would be a great place to put it -- including if you think it isn't necessary/it's fine to do the sort of partial vendoring.
Last updated: Oct 23 2024 at 20:03 UTC