Component model imports/exports vs worlds · general

In the component model, am I right in understanding that a host's world only needs to expose a superset of the imports of a guest component? Is there any enforcement that a world's imports also be satisfied by guest components' exports?

Alex Crichton (Jan 29 2024 at 20:10):

you're correct in that a host needs to have at least the guests imports but is allowed to have more (it can have a superset)

There's no relation to a world's imports/exports in the sense that the guests exports are what's available for the embedding to call. The guest is allowed to export more than the host wants, however

Alex Crichton (Jan 29 2024 at 20:11):

Geoff Goodman (Jan 29 2024 at 20:12):

I think it's mostly clear. What I want to make sure to clear up is what the relationship is between a host's imports and a guest's exports.

Geoff Goodman (Jan 29 2024 at 20:15):

If a host world has an import of imported: func(); and the guest has an export of exported: func(); these two don't line up.

It's clear to me that the exported guest export can go unconsumed without issue.

What's not clear is whether the host's imported import must be fulfilled by the component.

Alex Crichton (Jan 29 2024 at 20:16):

No overlap is needed for imports/exports, no. The guest can do whatever it likes with the imports and the host can do whatever it wants with the exports, no need for either to be related

Alex Crichton (Jan 29 2024 at 20:17):

the host imports provided to the component may or may not be called if that's what you're asking though, just because a component imports something doesn't mean its needed for any particular export

Geoff Goodman (Jan 29 2024 at 20:19):

Can the host enforce that a guest component provide a specific export statically in the .wit definition?

Alex Crichton (Jan 29 2024 at 20:21):

world my-world {
    export foo: func();
}

if a host runs with that world then components must export a function called foo to be loaded correctly

Geoff Goodman (Jan 29 2024 at 20:23):

OK, that makes a ton of sense. I had a difficult time discovering that when reading WIT.md :sweat_smile:.

Geoff Goodman (Jan 29 2024 at 20:27):

I think what I've been finding difficult to internalize with the world concept is that it seems to be trying to describe two contracts in a single paradigm.

Geoff Goodman (Jan 29 2024 at 20:30):

It seems more like the world is more in tune with the host --> guest relationship; it describes the 'world' in which guests must operate.

On the other hand, it seems like a Component is really the set of imports it requires and the set of exports it provides.

Alex Crichton (Jan 29 2024 at 20:30):

indeed! A world itself can be viewed from both the perspective of the guest and the host as well. A host is "this is what you gotta support" and a guest is "this is what you can use", but it's definitely a bit of a mind-bender

Geoff Goodman (Jan 29 2024 at 20:31):

I understand that some Components might be designed to act as sorts of proxies (ie: wasm-virt) so perhaps that's where the 'bi-directional' nature of the world comes in...

Geoff Goodman (Jan 29 2024 at 20:33):

But it seems like a proxy component might want to define itself more in terms of two contracts: 1) the host side; and 2) the guest side.

It seems like the current wit spec tries to lump those into the same definition.

Alex Crichton (Jan 29 2024 at 20:34):

Definitely feel free to open issues on the component-model repo, in addition to improving the current documentation I'm sure there's changes we could consider at a deeper level too

Geoff Goodman (Jan 29 2024 at 20:34):

Geoff Goodman (Jan 29 2024 at 20:35):

Do you think this intuition about the host vs guest side contract makes any sense? I'm very new to this whole world and don't want to derail conversations with uninformed takes.

Alex Crichton (Jan 29 2024 at 20:38):

Heh to some degree first impressions always make sense to me, but in general it's definitely something we've struggled with in the past. There's a lot of different users viewing WIT from different angles and it's been tough to balance all the concerns. In that sense it's probably not a silver bullet of how to rewrite everything but at the same time I think it's important feedback of how to shape documentation. Much of it has been written by folks who have their own perspectives that don't match everyone else's, so I think it's valuable to shape it to be better understandable to newcomers

Geoff Goodman (Jan 29 2024 at 20:55):

Is it fair to say that a host Runtime might specify one or more worlds describing it's capabilities? I think it's intuitive to make sense of what a host _offers_ to guest components.

On the topic of a Runtime imports, I think _every_ component operating in that world must fulfill these, right? It is not enough for one guest Component among many to satisfy the runtime imports, is it?

Geoff Goodman (Jan 29 2024 at 20:56):

I'm trying to write down my thoughts and in doing so keep discovering gaps in my understanding. I really appreciate your help filling in these gaps synchronously like this! :smile:

Alex Crichton (Jan 29 2024 at 21:01):

You can sort of talk about what a host supports in two ways. One way is that it has multiple possible worlds it support, but you could also union all those worlds together and say that the host supports that one world. I suspect it's more common to say that hosts support multiple worlds, however. For example the wasmtime CLI supports the wasi:cli/command world via wasmtime run and the wasi:http/proxy world via wasmtime serve.

I think the answer your question is "no", but I'm going to say more words here as my understanding may be flawed. For the imports of a world those are items that are provided by the host. Components may or may not use those imports but components don't satisfy the imports by default. You can, however, link components together (e.g. via wasm-tools compose) in which case you would have to satisfy the imports, but that operation is more like connecting the exports of one component into the imports of another.

Geoff Goodman (Jan 29 2024 at 21:04):

I think I got this backwards :man_facepalming: so let me rephrase: "Is each guest Component that operates in a given world required to provide the exports defined by the union of the Runtime's worlds?

Geoff Goodman (Jan 29 2024 at 21:10):

I was writing out an example with wasi-http and realize that I probably have another gap in understanding. Does a component statically declare the worlds it targets? In other words, a component designed to handle http requests would say, "I target wasi:http" and therefore the runtime could then compare its expectations for such components agains the component's _actual_ exports?

Geoff Goodman (Jan 29 2024 at 21:11):

So failing to export a handle() function in the incoming-handler interface would be a link-time error.

Geoff Goodman (Jan 29 2024 at 21:12):

That suggests that runtimes would need _a priori_ knowledge of the worlds that guest components might say they target.

Geoff Goodman (Jan 29 2024 at 21:13):

Or have a mechanism for dynamically dereferencing a world name to that world's definition.

Geoff Goodman (Jan 29 2024 at 21:14):

If all that is true, it makes me think that the world names are significant. I thought it was their _shape_ that is significant but perhaps there is a sort of 'tagged identify' carried by interfaces and functions. Is that true?

Geoff Goodman (Jan 29 2024 at 21:16):

How to packages fit into this whole dynamic? Haha, this is really difficult to wrap my head around.

Alex Crichton (Jan 29 2024 at 21:23):

You were right until the word "union". For example wasi:http/proxy and wasi:cli/command have different exports. A command is a main function and a proxy is a per-http-request function. Wasmtime supports both so you could technically union the worlds together, but they're for different purposes so a component doesn't actually have to support both to run in wasmtime.

So I would rephrase what you said as something along the lines of: A component must provide all exports a host requires for a world that it's running. That way a host gets to define which world, if many are supported, is being used (e.g. run vs serve)

Sort of and sort of not. When you build a component all current build tooling works by you specify a world in the WIT format and then bindings are generated. The final component, however, does not have WIT embedded. Instead the final component simply has imports, exports, and the types of those imports/exports. This means that a component is self-describing in what it needs and provides (imports and exports) and that may not correspond to any precise world. For example components running in wasi:cli/command may not import all that it has to offer.

You can see the exact shape of a component in the WIT format with wasm-tools component wit ./my-component.wasm. That will print a WIT rendering of what a component requires. This is given a default name "root" but that doesn't actually correspond to what the component was generated with. That's also because as part of the build process worlds might be unioned (e.g. your app might use deps A and B both of which were built with slightly different worlds so your total app is the union of the two, e.g. the union of all that's imported).

This is the theory, yes. Component runtimes can statically inspect a component binary and say "nope I can't run that". A runtime doesn't have to instantiate the component to make this decision, only parse it. In practice I don't think Wasmtime does this yet with its bindgen! macro, but we should probably add something for that. (it's just an oversight, no fundamental reason for this)

This is again, sort of correct and sort of not. It depends! How fancy is your runtime? Practically, however, you are correct. The wasmtime::component::bindgen! macro for example takes a WIT world as input, which is the a-priori knowledge of what it's targetting.

I say sort of though because this is not necessarily required. You can technically load a component, reflect on its imports/exports, and then play around with the component anyway. That's more of a debugging-style situation or something like that, but it's not necessarily required. Not statically knowing the world, though, will result in a perf hit.

This I can safely say is incorrect, or at least in theory. A world's name should not matter, only its shape. This goes back to how components are self-describing in the types of their imports/exports but there's no name corresopnding to the world. We give worlds names to refer to them in bindings generators but that's basically a WIT-level phenomenon which is not reflected into component binaries.

WIT packages are units of distribution for managing a collection of WIT files. WIT is sort of a layer as a whole DSL in working with components in that sense. Components themselves simply are and there's no concept of a package for components (yet at least). WIT packages are a collection of interfaces and worlds which are given a name.

Basically a fancy way of saying that it's just a WIT-level construct to help manage the developer workflow of dealing with WIT dependencies, WIT interfaces, etc.

Geoff Goodman (Jan 29 2024 at 21:36):

So the package and world name is only significant at build time as a way of expressing a _target_ for a component or a sort of 'use-case' for a runtime. The target 'world' is a host-specific bit of business logic. For example, in the wasmtime CLI, the command you use comes with an implicit 'world'; the serve command implies the wasm:http reactor world.

Geoff Goodman (Jan 29 2024 at 21:43):

I've seen quite a few examples of vertical composition between components, where a wrapper or adapter component changes the surface area of a wrapped component. This allows a guest component to potentially operate in a very different world than the one that its own interface suggests.

But I wonder if there are other kinds of composition being pursued where a single component has different parts of its host expectations fulfilled by distinct host components.

| Runtime | Provider 1 | Guest |
|         |------------|       |
|         | Provider 2 |       |

Geoff Goodman (Jan 29 2024 at 21:56):

Anyway big thanks Alex. I'm going to digest all this and write down my understanding on GitHub.

Alex Crichton (Jan 29 2024 at 21:58):

Indeed! One example here would be that you could implement the wasi:http/proxy world in terms of wasi:cli/command technically (aka you write the http server yourself). You'd then take a serve-targetting-component and wrap it and become a command-targetting-component.

An interesting idea! While I believe the component model affords for this I don't think it's been explored all that much in tooling/runtimes yet. Could be a neat idea though!

Geoff Goodman (Jan 30 2024 at 16:15):

Hi folks, I've taken a stab at describing the WASM Component / Runtime composition model and how it aligns to the WIT IDL. I intend to post this to the discussion on GitHub about renaming the world paradigm but would really appreciate a pre-read.

I hope to get feedback about whether I'm making false assumptions or other factual inaccuracies. Opinions are of course my own and are _definitely_ coming from the noob perspective.

# The world of WIT

As a newcomer to WASM Components and the WIT IDL, I'm trying to understand how I
should be using `.wit` files. Without touching on how these are expressed in
WIT, I can think of several use-cases listed below. I'm trying to approach this
from a composition perspective.

1. As a host, what capabilities do I offer to guests?
2. As a host, what requirements do I impose on guests?
3. As a guest, what requirements must my host fulfill?
4. As a guest, what capabilities do I expose to the host (and possibly to other
   guests)?

Now, you'll note that I didn't mention Component or Runtime above. It's my
understanding that a Runtime will _always_ be acting as a host and will _never_
be a guest. On the other hand, Components _may_ act as a host, but will _always_
be guests.

I also didn't mention any particular syntactical constructs. I'm trying to
understand the use-cases in the abstract before looking at how they're expressed
in the WIT IDL.

## Component composition and ecosystems

Given what we've explored above, I think it is helpful to see that a Runtime
must express a **Host Contract** and that a Component must express a **Guest
Contract** as well as an optional **Host Contract**. These are described below.

But while the Host Contract and Guest Contract help organize our thoughts about
vertical composition, I think they are missing the ecosystem angle. What I think
we're still missing is (4) from the original list: _Guest Capabilities_.

I think it's reasonable to expect folks to start building out an ecosystem of
re-usable WASM Components that fulfill specific needs. In that way, we can think
of such Components as traditional libraries / modules / packages / etc. These
Components encapsulate some logic and expose that through their API as _Guest
Capabilities_. This is functionality that is being put out there for anyone to
consume, if they so chose. There is _no requirement_ made about how it is used.
They are provided as is and it is up to consumers to decide how they should be
wired up.

So with that, I think we've got our third required concept, bringing us to the
following:

- **Host Contract** The minimum set of exported capabilities that a host
  enforces upon its guests. In `wasi:http`, this would be
  `incoming-handler.handle()`. In `wasi:cli`, this would be `run.run()`. In the
  former case, `wasmtime serve` requires a guest Component to provide the
  `handle()` method. In the latter case, `wasmtime run` requires a guest
  Component to provide the `run()` method.
- **Guest Contract** The minimum set of importable capabilities that a guest
  requires from its host. In other words, a guest Component that needs to
  perform file I/O will require capabilities from its host or it just can't do
  its job.
- **Guest Capabilities** The set of capabilities a Component offers to consumers
  with no assertions made about how they are used. An example of this might be a
  pseudo-random number generator Component. It might expose a `seed()` method
  that may (or may not) be called and a `random()` method to get the next
  pseudo-random number.

## Wrap up

So with that huge wall of text out of the way, what does this say about the
current WIT IDL? I think my main take-away is that the `world` paradigm is
trying to express all three of the above concepts simultaneously and ends up not
quite nailing it on any of them.

Geoff Goodman (Jan 30 2024 at 17:53):

Alex Crichton (Jan 30 2024 at 19:23):

Ralph (Jan 31 2024 at 10:00):

@Geoff Goodman this is wonderful (also with Alex and everyone). I also can't make any sense of it, so I'll pick up the thread. The reason? "world" already makes sense to me, so likely I'm missing something.

Stream: general

Topic: Component model imports/exports vs worlds

Geoff Goodman (Jan 29 2024 at 19:54):