Stream: git-wasmtime

Topic: wasmtime / issue #6881 Externally reusable proc macro exp...

Wasmtime GitHub notifications bot (Aug 22 2023 at 12:19):

vados-cosmonic opened issue #6881:

Feature

I'd like to be able to reuse and modify the output of macros like wit-bindgen::generate and wasmtime::component::bindgen

Benefit

Users will be able to build on the code generated by *bindgen crates.

This is a somewhat "backdoor" way of solving the following other issues:

https://github.com/bytecodealliance/wit-bindgen/issues/554
https://github.com/WebAssembly/component-model/issues/58 (it probably does not make sense to make annotations, but language specific bindgen modification probably makes more sense)

Implementation

To make that happen, it should be possible to move top level (and possibly lower) macro functionality (i.e. fn (input: InputStream) -> proc_macro::InputStream) to a separate crate that exports the transformations (and configuration/Parse impls necessary), and publish those separately, with a re-export under the original packages.

Alternatives

Alternatively, people can make partial copies of the current existing crates in order to use the functionality therein.

I've done this, and while it does work, it introduces an unnecessary partial vendoring of wasmtime in the downstream crate. Certainly a reasonable stop gap solution, but it would be much preferred if the import could boil down to mostly one dependency on a crate like wit-bindgen-macros/wit-bindgen-macro-util.

Wasmtime GitHub notifications bot (Aug 22 2023 at 12:20):

vados-cosmonic edited issue #6881:

Feature

I'd like to be able to reuse and modify the output of macros like wit-bindgen::generate and wasmtime::component::bindgen

Benefit

Users will be able to build on the code generated by *bindgen crates.

This is a somewhat "backdoor" way of solving the following other issues:

https://github.com/bytecodealliance/wit-bindgen/issues/554
https://github.com/WebAssembly/component-model/issues/58 (it probably does not make sense to make annotations, but language specific bindgen modification probably makes more sense)

Implementation

To make that happen, I'd like to move top level (and possibly lower) macro functionality (ex. functions with the signature (input: proc_macro::InputStream) -> proc_macro::InputStream) to a separate crate(s) that exports the transformations (and configuration/Parse impls necessary), and publish those separately, with a re-export under the original packages.

Alternatives

Alternatively, people can make partial copies of the current existing crates in order to use the functionality therein.

I've done this, and while it does work, it introduces an unnecessary partial vendoring of wasmtime in the downstream crate. Certainly a reasonable stop gap solution, but it would be much preferred if the import could boil down to mostly one dependency on a crate like wit-bindgen-macros/wit-bindgen-macro-util.

Wasmtime GitHub notifications bot (Aug 22 2023 at 16:54):

kajacx commented on issue #6881:

This would certainly be great. I am currently also using the workaround of manually copying the macro crate's code so that I can modify the outcoming TokenStream, for example here.

I have already created this issue in the past, but was closed. I think @alexcrichton didn't understand what I wanted, so let me re-iterate:

All that we are asking for is that the wasmtime_component_macro (and the wit-bindgen equivalent for the guest) is split into two: one that exports the macros as normal functions with TokenStream -> TokenStream signatures, and a second crate that just re-exports those functions as proc macros.

That's all. I don't expect any support in terms of "stability" of the resulting code from the macro or anything like that.

Wasmtime GitHub notifications bot (Aug 22 2023 at 19:24):

alexcrichton commented on issue #6881:

At least personally I don't have much more to add over what I already mentioned. I apologize if you felt misunderstood, but I feel like I understand what you're saying and asking for and what I'm trying to explain is why it may seem like a simple ask it's not as simple as it appears on the surface and I don't think is something we should take on maintaining.

The more "public surface area" of Wasmtime that folks use the more we inevitably get pushback when making changes or issues whenever changes are made. These aren't something we are really planning on maintaining beyond the interfaces that are intended.

My recommendation is still to either (a) copy and vendor the sources yourself to take on the maintenance burden yourselves or (b) work on integrating these features upstream here in Wasmtime.

Wasmtime GitHub notifications bot (Aug 23 2023 at 01:47):

vados-cosmonic commented on issue #6881:

Hey @alexcrichton apologies I searched but somehow didn't see that other thread! I made this thread thinking I was late to making one at all in the official repo after asking on Zulip.

My apologies -- the workaround as noted is fine (as is trying to integrate the features in jiving with Wasmtime proper).

Wasmtime GitHub notifications bot (Aug 23 2023 at 01:47):

vados-cosmonic closed issue #6881:

Feature

I'd like to be able to reuse and modify the output of macros like wit-bindgen::generate and wasmtime::component::bindgen

Benefit

Users will be able to build on the code generated by *bindgen crates.

This is a somewhat "backdoor" way of solving the following other issues:

https://github.com/bytecodealliance/wit-bindgen/issues/554
https://github.com/WebAssembly/component-model/issues/58 (it probably does not make sense to make annotations, but language specific bindgen modification probably makes more sense)

Implementation

To make that happen, I'd like to move top level (and possibly lower) macro functionality (ex. functions with the signature (input: proc_macro::InputStream) -> proc_macro::InputStream) to a separate crate(s) that exports the transformations (and configuration/Parse impls necessary), and publish those separately, with a re-export under the original packages.

Alternatives

Alternatively, people can make partial copies of the current existing crates in order to use the functionality therein.

I've done this, and while it does work, it introduces an unnecessary partial vendoring of wasmtime in the downstream crate. Certainly a reasonable stop gap solution, but it would be much preferred if the import could boil down to mostly one dependency on a crate like wit-bindgen-macros/wit-bindgen-macro-util.

Wasmtime GitHub notifications bot (Aug 23 2023 at 08:42):

kajacx commented on issue #6881:

@alexcrichton I understand what you're saying and asking for ... I don't think is something we should take on maintaining. I am not asking you to maintain the output of the proc macro any more than you do already. Let me explain it this way:

Wasmtime version 12 released, so to update my code, I have to:

1) Find the proc macro repository on github
2) Manually update the Cargo.toml dependencies, some of which are workspace dependencies, so this includes sifting through the code and finding the actual versions
3) Manually copy the new source code
4) Fix any problems that are created by using a new (and not necessarily compatible) version of the proc macro

Now let's compare this to how this would look like if the proc macro crates were split in two:

1) Update the version of the dependent proc macro crate to 12.something
4) Fix any problems that are created by using a new (and not necessarily compatible) version of the proc macro

The point 4 is exactly the same, because it's exactly the same code. So how is the first approach better?

I do not understand what "maintaining" are you talking about, I am not asking you to maintain anything, just split the crate in two and do everything else the same way you are doing.

Wasmtime GitHub notifications bot (Aug 23 2023 at 19:52):

alexcrichton commented on issue #6881:

To expand on this, I can say yes I understand what you're saying @kajacx and I additionally understand how that is appealing from your perspective. From your perspective I agree that it's less work for you and doesn't seem too add much maintenance on the project.

From Wasmtime's perspective, however, there's a few issues in my opinion:

• There's no reason for this split in the sense that if someone changes it there's no test that will fail or reason to keep it the way it is. This means that it can change at any time, and a request in the future to change it back to the way it was is, in my opinion, maintenance.
• You may not be the only user of this feature. While you may be willing to update when Wasmtime changes others may be less willing and may open issues here asking for guidance in updating, request that changes be backed out, or something like that.
• More generally even if users use a feature like this and update just fine there may be subtle runtime or other issues which aren't predicted since we, the maintainers of wasmtime, aren't necessarily going into this eyes open. This means that subtle issues may arise into our issue queue which we then debug and spend time on which end up bottoming out in code elsewhere we can't do much about.

Personally I'd prefer to avoid these situations. I understand that the distinctions I'm making are subtle and somewhat hand-wavy as well. To be clear I'm just one maintainer here and there are many others, so if others are willing to help you carry through a change like this I'm happy to hand it off to them.

Last updated: Dec 23 2024 at 13:07 UTC