Joel Dice (Jun 12 2023 at 14:28): Joel Dice (Jun 12 2023 at 14:28):In my work to support shared-everything linking, I've been wrestling with the problem of Wizer-style pre-initialization for components. I've spoken with various folks about it, suggesting and discarding a few approaches. I had a new idea over the weekend, which I'd like to get feedback on, particularly from @fitzgen (he/him) and @Alex Crichton if/when you have time: https://hackmd.io/rZ2_yvhRRsqjJiKoj27MCA
Guy Bedford (Jun 12 2023 at 18:23):@Joel Dice am I reading this right that it would be a sort of "component wizer" project? I really like that you're considering making it possible to be a component itself as well. While I understand you say it could apply to many component graphs, how much of the approach do you see as tied to the needs of the dynamic linking work specifically? Would you really be able to generalize enough of the work to make the effort worthwhile? Would be great to discuss this topic further in the dynamic languages meeting as well.
Jamey Sharp (Jun 12 2023 at 18:31):The more I think about it, the more I like this idea! I've left a comment on the document suggesting a kind of strange way of looking at the middle steps.
@Guy Bedford, my take is that what Joel has proposed should in principle work for any component as long as the result of partial execution can be serialized as a component. (So I guess reference types, and at least some of the GC proposal, don't work.) The implementation complexity mostly comes from parsing enough of the component to find all the mutable state, then exporting the revised state in some way that the host can find it, and finally serializing the component again with those "holes" filled back in with the updated state. It sounds like a pretty straightforward (though not exactly easy) solution to me.
Joel Dice (Jun 12 2023 at 18:42):@Guy Bedford Yes, I spoke with @fitzgen (he/him) about adding component support to Wizer, but when we dove into the details of what that would entail for both Wizer and Wasmtime, it got pretty ugly. The proposal above avoids that ugliness by doing some strategic pre-processing of the input component.
For the time being, I'm focused on the shared-everything linking case (and more specifically, componentize-py), so I haven't thought deeply yet about the general case of pre-initializing arbitrary components, but what @Jamey Sharp described is what I have in mind.
Guy Bedford (Jun 12 2023 at 18:50):So this tool would still depend on Wizer? One issue we have in JS land is that we need binary builds maintained on npm of wizer (and also anything depending on wizer). hence my excitement for the possibility of supporting wasmable interpreters, over maintaining more binary builds on npm.
Joel Dice (Jun 12 2023 at 18:53):No, what I've described wouldn't depend on Wizer. And yes, I'm excited about that, too. I've added a note to the bottom of the doc:
Also note that, since there’s nothing runtime-specific about this approach, we could theoretically use a Wasm interpreter which itself compiles to Wasm and package the whole tool as a Wasm module or component.
Jamey Sharp (Jun 12 2023 at 18:58):This would do the work that wizer does in a different way. It might make sense for it to be "the new wizer" and live in the wizer repo.
It could be a multi-stage series of components: The "wizer" component reads the user's component and produces a new component that, when run, emits the wizened component. (This is basically my weird suggestion in Joel's document.) Then you don't need a wasm interpreter running inside wasm; whichever wasm runtime you use runs the user's code as well as the code that writes the final output.
Guy Bedford (Jun 12 2023 at 19:01):If there's more iterative paths as well, that wouldn't be a problem, but stating that end goal seems worthwhile to me! I am all for such a Component Wizer / Wizer v2 and the overall approach, count me on board :)
fitzgen (he/him) (Jun 13 2023 at 18:38):Seems reasonable to me.
Soft preference for adding this to Wizer than creating a whole new tool specifically for components of just exactly that particular shape.
Joel Dice (Jun 13 2023 at 19:44):Thanks for taking a look, @fitzgen (he/him). I'm not sure when I'll get around to implementing this, but I'll look at how it might fit into Wizer when I do.
Last updated: Nov 22 2024 at 17:03 UTC