Paul Scheduikat (Feb 07 2024 at 12:39): Paul Scheduikat (Feb 07 2024 at 12:39):Do I understand correctly that is expected for each component implementing a world from a wit file to be it's own package in warg, and the wit file also being published to a separate package encoded in it's own wasm file?
If that's the case, is there a standard way in warg to describe the relationship between the implementation components and the wit component?
Or is there a way to merge the implementation .wasm components with the wasm encoded wit?
Or can a package consist of multiple wasm files?
Paul Scheduikat (Feb 07 2024 at 13:05):Has been helpful writing this down.
Library users only ever import the interface definitions from the wit component.
Implementations are only specified when composing components.
Correct?
Paul Scheduikat (Feb 07 2024 at 13:27):What happens when specifying conflicting implementations i.e. two components each specify an implementation for a subset of the interfaces, but these subsets intersect?
Calvin Prewitt (Feb 07 2024 at 18:12):Great questions! Some of these answers are still being figured out. Here are some partial answers.
A single Wasm binary is what you publish with Warg. However, that binary could be composed of other components that either are bundled in the binary or just referenced with an import statement that needs to be resolved by the registry. So a single Wasm binary could be a composition of multiple Wasm binaries.
A Wasm component binary could be just the binary representation of a WIT text file. Or it could have an implementation of those interfaces. Both are publishable components.
You might want to publish interfaces by themselves if you want them to be reused and targeted by multiple component implementations.
Lann Martin (Feb 08 2024 at 13:59):If a wit file is intended to have multiple implementations (like the WASI specs) then you would publish it as a standalone binary-encoded wit package. If you just want a representation of a particular component's imports/exports then you don't need a separate package; components are self-describing in that way.
Ralph (Feb 08 2024 at 14:16):I think I missed this, but do we have a way to create a "binary-encoded wit package" right now? with no implementations, just the module with the world(s)?
Paul Scheduikat (Feb 08 2024 at 14:19):@Lann Martin
How would I import a "implementation" without a separate wit component defining it's interfaces?
package example:add;
interface add {
add: func(a: u32, b: u32) -> u32;
}
world adder {
export add;
}
When I create a component with cargo component which confines to the adder world, I then get this fairly unhelpful wit back.
$ wasm-tools component wit target/wasm32-wasi/release/adder.wasm
package root:component;
world root {
export example:add/add;
}
Ralph said:
I think I missed this, but do we have a way to create a "binary-encoded wit package" right now? with no implementations, just the module with the world(s)?
Paul Scheduikat (Feb 08 2024 at 14:21):@Ralph
wasm-tools component wit wit/ -w -o out.wasm
@Paul Scheduikat
There's some rough edges importing implementations right now. I know @Peter Huene and @Daniel Macovei are working through the issues.
Ralph (Feb 08 2024 at 14:27):no, this is great! I had missed that you could create an interface-only module
Ralph (Feb 08 2024 at 14:27):That's wonderful!!!!
Paul Scheduikat (Feb 08 2024 at 14:29):@Calvin Prewitt The interface types are probably already contained in the binary is there a way to convert them to wit?
Ralph (Feb 08 2024 at 14:29):I do know you can extract them easily enough, but it's .wit-to-module that I'd missed.....
Ralph (Feb 08 2024 at 14:32):wasm-tools component wit HelloWorld.wasm
package root:root;
world root {
}
Ralph (Feb 08 2024 at 14:33):but THAT has an implementation.....
Lann Martin (Feb 08 2024 at 14:35):@Paul Scheduikat I believe this prints a component's interface as wit: wasm-tools component wit component.wasm
Calvin Prewitt (Feb 08 2024 at 14:36):@Paul Scheduikat The tooling will get more integrated and easier for workflows. But wasm-tools component wit filepath/filename.wasm will print to to stdoout.
Calvin Prewitt (Feb 08 2024 at 14:37):I just ran it on wasi:http encoded as binary and outputted to .wit text file. Works as expected.
Calvin Prewitt (Feb 08 2024 at 14:40):Doc comments will survive the roundtrip to binary and back to .wit text file. But you will end up with a single .wit file and lose the original directory structure of separate .wit files.
Paul Scheduikat (Feb 08 2024 at 14:40):Only for interface components for implementations it doesn't:
$ wasm-tools component wit adder.wasm
package root:component;
world root {
export example:calculate/add;
}
It is contained in the binary tho:
$ wasm-tools component wit adder.wasm -t
(component
(type (;0;)
(component
(type (;0;)
(component
(type (;0;)
(instance
(type (;0;) (func (param "a" u32) (param "b" u32) (result u32)))
(export (;0;) "add" (func (type 0)))
)
)
(export (;0;) "example:calculate/add" (instance (type 0)))
)
)
(export (;0;) "root:component/root" (component (type 0)))
)
)
(export (;1;) "root" (type 0))
(@producers
(processed-by "wit-component" "0.20.1")
)
)
AH, yes, is there an issue for that?
Daniel Macovei (Feb 08 2024 at 14:44):Sorry in your example what does your original component's wit look like?
Paul Scheduikat (Feb 08 2024 at 14:46):@Daniel Macovei
package example:calculate;
interface add {
add: func(a: u32, b: u32) -> u32;
}
world adder {
export add;
}
It's not that it "only works for interface components". It's that the binary you printed has a different wit than the wit you used to create the component.
Ralph (Feb 08 2024 at 14:52):Paul, are you testing out the component book example?
Paul Scheduikat (Feb 08 2024 at 14:52):Yes rust component example
Daniel Macovei (Feb 08 2024 at 14:52):Sorry I know that's confusing. But if you look at the wat you shared, that is a component that exports a component type that is described by the wit you shared, which is a different component than the one described by the wit
Lann Martin (Feb 08 2024 at 14:56):The tool could recurse into the nested interfaces. You wouldn't always want to do that unconditionally but it is functionality that could be added.
Daniel Macovei (Feb 08 2024 at 14:58):Are you using cargo-component? It seems like your wat was handwritten
Ralph (Feb 08 2024 at 14:58):Lann Martin said:
The tool could recurse into the nested interfaces. You wouldn't always want to do that unconditionally but it is functionality that could be added.
it would be nice to be able to pass a --tree option to recurse, though in many cases those interfaces are not exposed to the outside caller.
Paul Scheduikat (Feb 08 2024 at 14:58):@Daniel Macovei
No cargo component.
Ralph (Feb 08 2024 at 14:59):Daniel Macovei said:
Are you using cargo-component? It seems like your
watwas handwritten
he's doing the book's wat example: https://component-model.bytecodealliance.org/tutorial.html
Ralph (Feb 08 2024 at 14:59):Correct, @Paul Scheduikat ?
Daniel Macovei (Feb 08 2024 at 14:59):Was surprised there aren't any wasi interfaces in your output
Ralph (Feb 08 2024 at 15:00):Basically, Paul is user testing the tutorial for us :-) we need to take notes and file prs and issues :-)
Paul Scheduikat (Feb 08 2024 at 15:01):@Daniel Macovei You only get those if your code is directly runnable.
Paul Scheduikat (Feb 08 2024 at 15:04):@Ralph The tutorial works, there is just just no example of composing multiple self written packages together.
Daniel Macovei (Feb 08 2024 at 15:04):^^
Ralph (Feb 08 2024 at 15:04):right, this is interesting!
Ralph (Feb 08 2024 at 15:04):great!
Daniel Macovei (Feb 08 2024 at 15:06):It's a little tricky because composition is the intersection of using several tools together here. You're asking about warg, cargo-component, wasm-tools component wit, and then there are also a variety of ways to perform the composition.
Ralph (Feb 08 2024 at 15:06):I'll see about doing that later today and submit a pr to the doc repo
Ralph (Feb 08 2024 at 15:07):@Daniel Macovei is right, but that shouldn't bother us too much at this point methinhks. once language tooling kicks in, that will all change anyway....
Daniel Macovei (Feb 08 2024 at 15:07):Yes this use case definitely is something we care about making more intuitive, it's just going to take some time I think. Folks are aware it's not intuitive
Daniel Macovei (Feb 08 2024 at 15:09):But specifically I think one of the take aways is that while wasm-tools could support recursion when it prints wit, probably a common point of confusion is that the top level wit for a binary generated by cargo-component exports the wit that you use to create the binary, meaning it is not going to have the same wit you used generate the component
Daniel Macovei (Feb 08 2024 at 15:09):One needs to understand that to start playing around with composition
Daniel Macovei (Feb 08 2024 at 15:10):And that probably isn't clear from the component book
Ralph (Feb 08 2024 at 15:12):squillace@idiopath:~/work/hello-wasi-http/target/wasm32-wasi/debug$ wasm-tools component wit hello_wasi_http.wasm
package root:component;
world root {
import wasi:io/error@0.2.0;
import wasi:io/streams@0.2.0;
import wasi:cli/stdout@0.2.0;
import wasi:cli/stderr@0.2.0;
import wasi:cli/stdin@0.2.0;
import wasi:http/types@0.2.0;
export wasi:http/incoming-handler@0.2.0;
}
Ralph (Feb 08 2024 at 15:13):if it were a composed component, you'd be right, only the top level things would be visible; I like the tree idea, though....
Ralph (Feb 08 2024 at 15:13):that's Dan's rust wasi:http hello world
Ralph (Feb 08 2024 at 15:14):lemme go find the wit:
image.png
Daniel Macovei (Feb 08 2024 at 15:16):hmm if you have the code available it looks like that one might actually have several wit files
Ralph (Feb 08 2024 at 15:17):Dan had this one running throughout the RC flow, so early on you had to have the wit to compile, and it hasn't been rearranged: https://github.com/sunfishcode/hello-wasi-http
Ralph (Feb 08 2024 at 15:18):which was why I was asking whether we even could now build only wit files!
Ralph (Feb 08 2024 at 15:18):I was "used to" having to have the physical .wit to build
Paul Scheduikat (Feb 08 2024 at 15:20):@Daniel Macovei
I still don't really understand how I would do this. The following is loosely contrived from the tutorial, but simplified and I now want to split it up in two packages.
package docs:add;
interface add {
add: func(a: u32, b: u32) -> u32;
}
world adder {
export add;
}
The adder world is then built to a component with cargo component.
This results in the adder.wasm file implementing the adder world.
I then create a new project (component) with cargo component new and only copy over the adder.wasm file.
package docs:calculator;
world calculator {
import docs:add:add;
}
What are the steps to implement the calculator world?
Daniel Macovei (Feb 08 2024 at 15:22):So that's the part where you write rust code in your cargo component project
Daniel Macovei (Feb 08 2024 at 15:24):The rust code isn't included in the docs, but you would import the bindings in your lib.rs and reference the interfaces you're implementing from bindings. This is the part where the cargo-component docs are your friend https://github.com/bytecodealliance/cargo-component?tab=readme-ov-file#getting-started
Lann Martin (Feb 08 2024 at 15:24):As a meta-point, this thread might get more relevant eyeballs in #wit-bindgen
Ralph (Feb 08 2024 at 15:24):agreed; can you move it? I'm not a zulip master, I confess.
Ralph (Feb 08 2024 at 15:25):#bleg
Paul Scheduikat (Feb 08 2024 at 15:25):@Daniel Macovei But I need the type information for docs:add:add;to be able to use it in rust. I now only have adder.wasm which although it contains the type information isn't easily usable.
Daniel Macovei (Feb 08 2024 at 15:26):cargo-component leverages wit-bindgen which creates the rust types you need from your wit and exposes them in the bindings module it generates for you
Daniel Macovei (Feb 08 2024 at 15:27):in the link i shared there's this snippet
Paul Scheduikat (Feb 08 2024 at 15:27):I do get that I can copy over the wit interface declaration from the other project, but as I have learned now I should be able to import it with just the adder.wasm file.
Daniel Macovei (Feb 08 2024 at 15:27):mod bindings;
use bindings::Guest;
struct Component;
impl Guest for Component {
/// Say hello!
fn hello_world() -> String {
"Hello, World!".to_string()
}
}
@Daniel Macovei I do understand how to implement components in rust.
Daniel Macovei (Feb 08 2024 at 15:31):Sorry it's a little confusing what you're trying to import where. A little later on I can try and push an example that recreates the example in the book. Without having looked too closely at it though you need to create two different cargo-component projects, one for the calculator, and one for adder, and then compose the output binaries
Daniel Macovei (Feb 08 2024 at 15:31):Paul Scheduikat said:
Daniel Macovei I do understand how to implement components in rust.
Sure thing, sorry for mistaking that as the point of confusion
Ralph (Feb 08 2024 at 15:32):I think the point is the book doesn't discuss the "composing" or "binding together" of the components, am I correct, Paul?
Paul Scheduikat (Feb 08 2024 at 15:36):I know how I would compose these two packages together.
If a wit file is intended to have multiple implementations (like the WASI specs) then you would publish it as a standalone binary-encoded wit package. If you just want a representation of a particular component's imports/exports then you don't need a separate package; components are self-describing in that way.
I don't understand how I would go about it when downloading just a implementation wasm component from a registry, without a separate wit package defining the interfaces.
Ralph (Feb 08 2024 at 15:37):EXACTLY
Ralph (Feb 08 2024 at 15:38):up to now, I've been copying around the wit files for implementations; the question is how to reference those types by pointing at an interface-only component module.
Ralph (Feb 08 2024 at 15:39):Let's ignore the registry part for a moment, because that is just a location. First we should be able to replicate this locally and document that in the book.
Paul Scheduikat (Feb 08 2024 at 15:40):Agree registry is irrelevant here.
Last updated: Nov 22 2024 at 17:03 UTC