Stream: wasi

Topic: understanding component model

Merlijn Sebrechts (Aug 11 2023 at 15:35):

Hi all, I'm trying to better understand the component model and how it relates to "modules". I have some questions:

1. The docs sometimes talk about "webassembly core modules". Does this refer to "modules as described in the webassembly core specification"?
2. Will components typically have multiple modules inside of them; or do you expect all composition to happen on the component level instead of on the module level in the future?

I'm a bit confused by the statement of "modules are shared libraries, components are executables". Do we expect some runtimes to provide some shared functionality using modules, like how shared libraries work on Linux? Or do we expect them to only provide such functionality using components?

Peter Huene (Aug 11 2023 at 15:36):

Hi. @Merlijn Sebrechts.

1. Yes, that's exactly what is meant by "wasm core module".
2. There's nothing prohibiting having a "standalone" (my term meaning not composed with other components) component that is comprised of multiple modules, only that generally the language-specific tooling produces a single implementation module and thus is "componentized" with that single module. Technically, even these components have more than one module defined in them, but the extra modules are tooling-created to facilitate in the lowering of imports and contain no executable functions. But that's different from having multiple implementation modules.

Peter Huene (Aug 11 2023 at 15:37):

I'm not sure what describes modules as shared libraries and components as executables, as that's not quite apt

Merlijn Sebrechts (Aug 11 2023 at 15:37):

This blog post makes that analogy: https://www.fermyon.com/blog/webassembly-component-model

Joel Dice (Aug 11 2023 at 15:39):

Regarding #2: you can now also compose application-defined modules in a component:
https://github.com/WebAssembly/component-model/blob/main/design/mvp/examples/SharedEverythingDynamicLinking.md
https://github.com/bytecodealliance/wasm-tools/pull/1133

Lann Martin (Aug 11 2023 at 15:40):

I think the confusing term there is "shared"; unlike shared libraries, module instances can't be shared between components (though module code can be)

Joel Dice (Aug 11 2023 at 15:40):

Merlijn Sebrechts said:

This blog post makes that analogy: https://www.fermyon.com/blog/webassembly-component-model

Ha, I wrote that blog. Like all analogies, it's imprecise, but I _think_ it provides a decent intuition.

Merlijn Sebrechts (Aug 11 2023 at 15:40):

So does that mean that in the future, "modules" will become more or less a hidden implementation details for app developers (like object files are nowadays)? So devs in the future will mostly interact with components instead?

Peter Huene (Aug 11 2023 at 15:41):

Merlijn Sebrechts said:

So does that mean that in the future, "modules" will become more or less a hidden implementation details for app developers (like object files are nowadays)? So devs in the future will mostly interact with components instead?

I think that's the hope, at least; interacting with components provides interoperability and composability that's just not possible when dealing with core modules.

Joel Dice (Aug 11 2023 at 15:44):

Situations where module composition can be desirable are e.g. porting existing libraries and FFI systems. For example: supporting Python native extensions, which has been one of my recent projects.

Merlijn Sebrechts (Aug 11 2023 at 15:50):

Joel Dice said:

Regarding #2: you can now also compose application-defined modules in a component:
https://github.com/WebAssembly/component-model/blob/main/design/mvp/examples/SharedEverythingDynamicLinking.md
https://github.com/bytecodealliance/wasm-tools/pull/1133

I have to admit, I've already read through the shared everything dynamic linking doc, and I'm still having a hard time to understand it. Is it something like this?

• So by default, component model linking means shared-nothing linking: interaction between modules happens across component boundaries using the defined interfaces.
• This means that if multiple modules use common functionality like libc, and libc is a module instead of a component, it will be copied across multiple components (by the compiler).
• With shared-everything linking, the compiler doesn't actually add the libc module to the component. Instead, the runtime does this.

Am I understanding this correctly?

Joel Dice (Aug 11 2023 at 15:58):

That sounds right to me. When you link libc.a into your app when building a core module (which may or may not end up wrapped in a component), it can't be shared with other modules or components. When you link libc.so using shared-everything linking, it can be used by multiple modules and even multiple subcomponents within the resulting component, which results in less code duplication.

Joel Dice (Aug 11 2023 at 15:59):

That becomes a big deal when the library is big, like libpython3.11.so or the .NET runtime.

Merlijn Sebrechts (Aug 11 2023 at 16:00):

Oh, so the "shared" means "two components share the same module" instead of "two modules share the same address space"?

Joel Dice (Aug 11 2023 at 16:00):

yeah, kind of like how two native processes can use the same .so, but not share memory

Merlijn Sebrechts (Aug 11 2023 at 16:01):

ok, makes sense!

Joel Dice (Aug 11 2023 at 16:01):

but to be clear, you can _also_ have two modules share the same address space (i.e. when one imports a memory from the other)

Merlijn Sebrechts (Aug 11 2023 at 16:02):

Indeed

Merlijn Sebrechts (Aug 11 2023 at 16:03):

Right now, wasmtime seems to have two linkers; one for modules and one for components. Is this expected to be the case in the future? Or will we end up in a situation where there is one conceptual "linker" that links both components and modules?

Joel Dice (Aug 11 2023 at 16:07):

@Alex Crichton is probably the one to answer that. I suspect we'll have both kinds of linker for a while, at least, since they each have unique capabilities and abstractions.

Merlijn Sebrechts (Aug 11 2023 at 16:17):

Ok, another question about the component model itself:

I haven't found much documentation about "worlds". I understand the use-case for the command/CLI world, for example. App and compiler devs can target this world, and know their app will work in any runtime that supports it.

But does a guest component need to explicitly declare what world they target? Or can they just declare to use a set of interfaces without defining a world?

Joel Dice (Aug 11 2023 at 16:31):

Generally speaking, we can refer to the union of interfaces which a component imports and exports as the "world" it targets. That world could be defined in a single .wit file like the CLI world or the wasi-http one, for example -- or it could be a combination of several worlds (e.g. both CLI and wasi-http, in which case the either the host would need to support both or parts would need to be virtualized in terms of the world(s) the host _does_ support).

Joel Dice (Aug 11 2023 at 16:32):

But since wit-bindgen and related tools require a world as input, you generally start with at least one world defined in a .wit file (which you might have written yourself or else downloaded from somewhere else).

Merlijn Sebrechts (Aug 11 2023 at 16:39):

How are guest worlds matched to host worlds? If the interfaces in a guest world are a subset of the interfaces in the host world, but they have different (world) names. Would that give any issues?

Joel Dice (Aug 11 2023 at 16:41):

No, that wouldn't be a problem. The name(s) of the world(s) are not included in the component type -- just the name(s) of the interfaces.

Joel Dice (Aug 11 2023 at 16:41):

Oh, I think I misread your comment.

Joel Dice (Aug 11 2023 at 16:41):

Yes, the names of the interfaces _do_ matter.

Merlijn Sebrechts (Aug 11 2023 at 16:41):

I think your first interpretation of my comment was correct

Merlijn Sebrechts (Aug 11 2023 at 16:41):

I meant different world names, not interface names

Joel Dice (Aug 11 2023 at 16:42):

Okay, yeah, world names don't matter (to my knowledge).

Merlijn Sebrechts (Aug 11 2023 at 16:43):

Ok, that makes sense, thanks!

Merlijn Sebrechts (Aug 11 2023 at 16:46):

Joel Dice said:

Generally speaking, we can refer to the union of interfaces which a component imports and exports as the "world" it targets. That world could be defined in a single .wit file like the CLI world or the wasi-http one, for example -- or it could be a combination of several worlds (e.g. both CLI and wasi-http, in which case the either the host would need to support both or parts would need to be virtualized in terms of the world(s) the host _does_ support).

How would such virtualization happen? Could the runtime add a component that bridges the gap between the guest world and runtime world?

Merlijn Sebrechts (Aug 11 2023 at 16:48):

(like a polyfill for wasm)

Joel Dice (Aug 11 2023 at 16:49):

Yes, the runtime _could_ do it, but usually the app developer would do it ahead of time using e.g. wasm-tools compose to combine the original component with a virtualizing component. See https://github.com/bytecodealliance/WASI-Virt for example.

Joel Dice (Aug 11 2023 at 16:50):

Another example where this is useful: the host _does_ support the required interfaces, but you're using a third-party subcomponent which you want to quarantine off and not give access to system resources.

Merlijn Sebrechts (Aug 11 2023 at 16:54):

Ah, that's interesting!

Merlijn Sebrechts (Aug 11 2023 at 16:55):

I'm not clear on how access rights are supposed to work in the component model in general. Can a runtime run a component that declares to use interfaces which the runtime disallows?

Joel Dice (Aug 11 2023 at 16:57):

A host runtime can do whatever it wants :smile:, which could include inserting a virtualizing shim, providing no-op (or trapping) stubs as native host functions, or refusing to instantiate the component.

Merlijn Sebrechts (Aug 11 2023 at 17:03):

Ok, cool! Do I understand it correctly then that the permissions system is currently ad-hoc and runtime-specific? Like, one runtime might support giving an app granular access to files, while another only supports binary all-or-nothing full filesystem access?

Joel Dice (Aug 11 2023 at 17:06):

Yes, that sounds correct.

Merlijn Sebrechts (Aug 11 2023 at 17:10):

Awesome, thanks a lot for helping me through this! :)

Last updated: Nov 22 2024 at 16:03 UTC