Stream: cargo-component

Topic: Composition from multiple base component crates

Nuke 🌄 (Mar 08 2024 at 23:05):

I am in the middle of trying to get https://github.com/NukeManDan/wasi-demo/commit/f2959bb03bba0aa0cdfbfb52071698b91a7cf1a1 to work, the idea here is I would like to have something like https://github.com/bytecodealliance/component-docs/pull/100 working here for an MVP node and web application that uses a _composed_ wasm including two components. As a demo I want to import and export those components.

Nuke 🌄 (Mar 08 2024 at 23:09):

I was hoping that a "dummy" crate would not be needed, if I can get wasm-tools component wit to generate a base Wasm file and then wasm-tools compose from that and the generated per-crate wasm

But whatever I do here I seem to get

error: component `target/wasm32-wasi/release/responder.wasm` does not import any instances

I am confused as my wit is:

world responder {
    import base64;
    import ckcompact-dr-kdf;
    include base64-world;
    include ckcompact-dr-kdf-world;
}

Nuke 🌄 (Mar 08 2024 at 23:10):

I bet I am missing something silly... any tips here would be awesome :pray:

Peter Huene (Mar 08 2024 at 23:25):

You might be running into the issue that imports are elided from the output component if there's no use of them in the source.

Peter Huene (Mar 08 2024 at 23:25):

so there's nothing to compose it with, as it has no imported instances

Nuke 🌄 (Mar 08 2024 at 23:27):

sounds like that would be the culprit... But how would I go about wiring up a dummy crate to output this correctly?

Nuke 🌄 (Mar 08 2024 at 23:28):

or if not possible, what about using wasm-tools component wit to generate a base Wasm file and then wasm-tools compose from that and the generated per-crate wasm?
Would that elide things as well?

Peter Huene (Mar 08 2024 at 23:31):

you could use a wasm-tools component embed --dummy command to create a dummy core module that imports everything and then create a component from that with wasm-tools component new, but the component will trap for any of its exports

Nuke 🌄 (Mar 08 2024 at 23:57):

Is there a way to express "I want to re-export this component"? Essentially I just want a passthrough

Peter Huene (Mar 09 2024 at 00:28):

not that i'm aware of; probably your best bet to show a composition here would be to call some function on the import, even if it's just to force the import to appear in the output component

Nuke 🌄 (Mar 09 2024 at 00:48):

https://github.com/NukeManDan/wasi-demo/blob/68c6ae692c3df29df89e9c7e11f1f869f4fa01b7/src/lib.rs endeed up working, but man that is really not a good DX way to re-export already implimented and compiled components :grimacing:

I am guessing there is not many use cases of this outside examples...

Nuke 🌄 (Mar 09 2024 at 00:52):

but perhaps wanting to expose the interface of an included world / external component in your final wasm would be desired. In my mind that would be simple to do to bundle various worlds together... I was under the impression the https://component-model.bytecodealliance.org/design/wit.html#including-other-worlds keyword would make this easy , but in my case the worlds I am including only export, and no matter what I try I get that imported instances elided issue IIUC

Nuke 🌄 (Apr 12 2024 at 21:29):

Seems like WAC will address some of the issues I was facing "re-exporting"

In the simplest terms, a composition is a collection of components that are instantiated in a topological order and certain exports from those instances are made available as exports of the composition itself.

And export statements to expose some or all exports of included components/worlds

Would that be the best path forward to replace what I did in this demo ?

Nuke 🌄 (Apr 12 2024 at 21:31):

related to https://bytecodealliance.zulipchat.com/#narrow/stream/206238-general/topic/.E2.9C.94.20Tool.20to.20verify.20WIT.2E/near/431380267 - would it be appropriate to make a WAC stream you think @Peter Huene ? I would be a follower!

Further, if my previous demo would be a good candidate as an example of using WAC to "re-export" from components, I might be game to make one as an example with stepwise README - as I see there are none yet in the WAC repo. Related to https://github.com/bytecodealliance/wac/issues/51

Nuke 🌄 (Apr 12 2024 at 21:54):

maybe https://github.com/bytecodealliance/wac/issues/80 is closer to what I was looking for wrt "re-export" :thinking: WDYT @Ryan Levick (rylev) ?

Ryan Levick (rylev) (Apr 15 2024 at 08:59):

I'm not sure what exactly you mean by "re-exporting". The issue you linked to is a way to "auto-discover" how two components should be composed together where the exports of one component are used as the imports of another. Likely the default behavior is that the exports of the "socket" component (i.e., the component whose imports we're satisfying) are exported and all of the exports of the "plug" component (i.e., the component whose exports were using to satisfy those imports) that are not used to satisfy an import are also exported.

Nuke 🌄 (Apr 15 2024 at 16:24):

I think why I I ask about that issue is I want a way to explicily set specific exports of an inner component are exported by the overall composed object. I believe that all not explicitly exported interfaces of the overall composition are elided away. That makes sense to me, as I would not expect inner parts of the composition to bubble up and be callable directly unexpectedly.

But I hope that there would be a way in wit or WAC to override that behavior without needing to write a wrapper component with pass through of an interface, as I think is required now.

Lann Martin (Apr 15 2024 at 16:25):

You'd need to define that behavior somewhere; why not in a (wrapper) component?

Nuke 🌄 (Apr 15 2024 at 16:33):

As to me it would hope the chore of that is not needed (in cargo component, a crate with impls of all exports that simply import and call exports, multiple files and perhaps ~30 loc) , if wit or wac can express what exports in a composition you want exported overall unchanged (perhaps a single loc to express unmodified export of an import).

I would also expect overhead in the wasm generated in that wrapping, rather than a direct call (I would need to introspect to see for sure)

Nuke 🌄 (Apr 15 2024 at 16:40):

As an example: say I have a compiled component model wasm with many exports (a library) and I want to compose that into a strict subset (and prune the new composition of all unneeded lib code generated completely).

As it stands now I think this is only possible by writing a wrapper crate, where I would expect only to need to specify in wit (or in composition tooling if impossible in wit/wac) what I want to export overall and composition tooling to "re-export" parts of that library, discarding the unneeded parts.

Lann Martin (Apr 15 2024 at 16:58):

Ah I see; I think what you're asking for is already a feature of WAC but I admittedly haven't used it too much yet.

Ryan Levick (rylev) (Apr 15 2024 at 17:04):

Yes, if I'm understanding your use case correctly, wac handles this case. You can pick exactly which exports to have from the composition regardless of which component they originally come from

Last updated: Nov 25 2024 at 19:03 UTC