alexcrichton opened PR #14275 from alexcrichton:wasi-named-imports to bytecodealliance:main:
This commit is at least an initial stab at making the
wasmtime-wasiandwasmtime-wasi-httpcrates compatible with "named imports" or theimplementsfield in the component model. This field enables importing an interface under a kebab-name while annotating that it's additionally to be considered an import of another interface's name. One example use case for this feature is [dependency isolation][diso] when composing two components together -- if they both import the filesystem the final component will import the filesystem twice under two different kebab names which means both components can have a different view of the filesystem.Wasmtime previously gained support for named imports and
implementsinbindgen!as part of #13513 where thenamed_importsoption can be specified atbindgen!-time which generates traits that take an extra id-style parameter. This runtime parameter indicates which kebab-name is being invoked through which the runtime can then dispatch on.The goal here is to actually wire all this up in a way that's usable for embedders. Specifically
named_importsbindings generation is now available for all WASIp2 and WASIp3 interfaces. Additionally all implementations of theseid-carrying traits are routed through the previous implementations after locating the correct context to operate over.All implementations of WASI functionality are already modeled more-or-less as methods on
Wasi*CtxView-style types which internally have a borrow to the actual state and the resource table to operate on. This fits quite cleanly with named imports where conceptually what we want is the ability to configure the context-per-kebab-name. This in theory will keep the maintenance burden managable as there's still largely one source of truth for the implementation. This neatly works for allHost-style traits which are literally methods onWasi*CtxViewtypes, meaning theid-carrying versions actually do just acquire aWasi*CtxViewand then delegate the method. This requires more finesse for*WithStoretraits which work withAccessandAccessor, however. Theidparameter cannot be threaded into thefn(..)within theAccessor, so refactoring is performed where appropriate to make the implementation of each interface a one-liner to reduce duplication.The end result of all of this is that this is a very large commit but it's written in such a way that the Rust compiler in theory should catch all mistakes. In other words we're heavily relying on the type system and type checking here and don't ever rely on duplication of methods that hopefully-won't-change. There's a lot of traits and a lot of interfaces, hence the size of the commit, but conceptually everything is intended to be pretty simple.
Some design decisions as part of this commit, in no particular order:
IDs are represented by
wasmtime_wasi::NamedIdwhich is a newtype-wrapper aroundusize. The goal here is to enable an efficient implementation of dealing with ids. This notably forces the embedder to derive some sort of string-to-id (and perhaps back) map when adding items to a linker.All of this is opt-in and nothing is changed by default. For example the
wasmtimeCLI does not support any of this yet -- in theory that would require the ability to configure-Sflags per-named-import as opposed to all-at-once.Mapping a
NamedIdto a context is abstracted behind a trait rather than dictating that aVecorHashMapor similar is required. This increases the cognitive load when reading code (more generics), but avoids making this design decision within these crates and leaves exact representations up to embedders.The
HasDataimplementation can't reuse the preexistingWasiCli, and this uses a newWasiCliNamed<T>instead. This enables threading this trait-to-find-a-context to the right location for*WithStoretrait impls.Some miscellaneous
bindgen!issues have been fixed during this commit to ensure that this compiles and works correctly.An attempt has been made at documenting all the new primitives/structs/etc here. These are sort of difficult to align correctly unless you know what you're doing, so the documentation and examples are intended to serve as a way of spreading this knowledge.
One possible alternative I ended up deciding not to do was to put some sort of map-to-context storage within each preexisting context type. For example commit would be simpler for the
*WithStoreand infrastructure if it reused the exact sameSelftype as all other impls do. My thinking though is that this requires dictating the use of aHashMapor something else which I was hoping to avoid. Additionally the preexisting context structures are already minimal enough that they're basically what you already want as the source for each implementation, so I wanted to lean on them as much as possible.The main wrinkle in the new implementation is that
Accessorcarriesfn(..)to project out it'sD::Data<'_>which means that it can't close over any information. This feature needs to in theory close over anid: NamedId, however, and there's no easy way to put this square peg into a round hole. To work around this internal implementations withinwasmtime-wasi{,-http}now have a genericFparameter which is a closure which projects data, but this closure is typically only ever on the stack and doesn't make its way to the heap. This was one of the more awkward things to work around in this commit.The design here is intentionally done to help ensure that this commit is correct with minimal testing. It's not really feasible to duplicate the entire test suite just for named imports but these are duplicate trait impls which otherwise shouldn't be wrong. By ensuring that there's either strict delegation or each-function-is-at-least-one-line that the light amount of testing here is sufficient for keeping this working over time.
[diso]: https://github.com/spinframework/spin/pull/3708
<!--
Please make sure you include the following information:
If this work has been discussed elsewhere, please include a link to that
conversation. If it was discussed in an issue, just mention "issue #...".Explain why this change is needed. If the details are in an issue already,
this can be brief.Our development process is documented in the Wasmtime book:
https://docs.wasmtime.dev/contributing-development-process.htmlPlease review the Bytecode Alliance's AI tool usage policy at
https://github.com/bytecodealliance/governance/blob/main/AI_TOOL_POLICY.mdPlease ensure all communication follows the code of conduct:
https://github.com/bytecodealliance/wasmtime/blob/main/CODE_OF_CONDUCT.md
-->
alexcrichton requested rvolosatovs for a review on PR #14275.
alexcrichton requested wasmtime-wasi-reviewers for a review on PR #14275.
alexcrichton requested pchickey for a review on PR #14275.
alexcrichton requested wasmtime-core-reviewers for a review on PR #14275.
alexcrichton updated PR #14275.
github-actions[bot] added the label wasi on PR #14275.
:memo: rvolosatovs submitted PR review:
Sorry for the delay, missed this one in my inbox. Currently still working on a full review, but I noticed that the PR introduces a regression against latest
main(b7764a705c245aa6fda46f9050e5cf9be62162b7), so sharing it early.For a case like this:
pub struct ErrorA; pub struct ErrorB; wasmtime::component::bindgen!({ inline: " package test:collision; interface a { enum error { failed } } interface b { enum error { failed } } interface combined { use a.{error as error-a}; use b.{error as error-b}; first: func() -> result<_, error-a>; second: func() -> result<_, error-b>; } world test { import combined; } ", imports: { default: trappable }, named_imports: { "test:collision/combined": usize }, trappable_error_type: { "test:collision/a.error" => ErrorA, "test:collision/b.error" => ErrorB, }, });I'm getting:
error[E0308]: mismatched types --> src/lib.rs:3:1 | 3 | / wasmtime::component::bindgen!({ 4 | | inline: " 5 | | package test:collision; 6 | | interface a { enum error { failed } } ... | 21 | | }, 22 | | }); | | ^ | | | | |__expected `ErrorA`, found `ErrorB` | arguments to this function are incorrect | note: method defined here --> src/lib.rs:3:1 | 3 | / wasmtime::component::bindgen!({ 4 | | inline: " 5 | | package test:collision; 6 | | interface a { enum error { failed } } ... | 21 | | }, 22 | | }); | |__^ = note: this error originates in the macro `wasmtime::component::bindgen` (in Nightly builds, run with -Z macro-backtrace for more info)While it works on current main
:memo: rvolosatovs submitted PR review.
:speech_balloon: rvolosatovs created PR review comment:
nit; since we're dropping these, should probably drop this one as well?
?;
:speech_balloon: rvolosatovs created PR review comment:
Is the plan to indefinitely duplicate implementations of all WASI interfaces using this scheme or could we consider always generating bindings with a
id: Option<NamedId>orid: Option<T>? (perhaps once the feature is stable)On a similar note, since additional functionality is likely to be added to WIT and be relevant (e.g. annotations, external-id etc.), would it make sense to introduce something like a unified context type? Something like a
struct InterfaceContext<Name, Id, Annotations> { name: Option<Name>, external_id: Option<Id>, annotations: Option<Annotations>, } struct FuncContext<Name, Id, Annotations> { interface: InterfaceContext<Name, Id>, external_id: Option<Id>, annotations: Option<Annotations>, // not sure what this is yet, but we probably don't want to use wasmtime::component::Val, so likely this is a WIT type passed to bindgen given latest discussions? } impl FuncContext { fn is_empty(&self) -> bool { .. } } impl InterfaceContext { fn is_empty(&self) -> bool { .. } }We could then generate something like:
trait HostFields { fn from_list(&mut self, cx: wasmtime::component::FuncContext<NamedId, ExternalId, Annotations>, entries: Vec<(String, Vec<u8>)>) -> HeaderResult<Resource<FieldMap>>I guess what I'm trying to point out is that it seems like this goes beyond just the interface names, since the
external-idand annotations might also be of importance for the embedder.More concretely:
- Should we always generate bindings, which take an additional parameter instead of the split into named and unnamed?
- Should we generalize this to account for the upcoming additions and instead allow embedders to derive a custom per-function context at link time? (this would replace a generic
{Func,Interface}ContextI suggested above by either a wasmtime-wasi provided abstraction or, (IMO) ideally, a genericTthat embedder could supply and derive at link time)WDYT?
alexcrichton edited PR #14275:
This commit is at least an initial stab at making the
wasmtime-wasiandwasmtime-wasi-httpcrates compatible with "named imports" or theimplementsfield in the component model. This field enables importing an interface under a kebab-name while annotating that it's additionally to be considered an import of another interface's name. One example use case for this feature is [dependency isolation][diso] when composing two components together -- if they both import the filesystem the final component will import the filesystem twice under two different kebab names which means both components can have a different view of the filesystem.Wasmtime previously gained support for named imports and
implementsinbindgen!as part of #13513 where thenamed_importsoption can be specified atbindgen!-time which generates traits that take an extra id-style parameter. This runtime parameter indicates which kebab-name is being invoked through which the runtime can then dispatch on.The goal here is to actually wire all this up in a way that's usable for embedders. Specifically
named_importsbindings generation is now available for all WASIp2 and WASIp3 interfaces. Additionally all implementations of theseid-carrying traits are routed through the previous implementations after locating the correct context to operate over.All implementations of WASI functionality are already modeled more-or-less as methods on
Wasi*CtxView-style types which internally have a borrow to the actual state and the resource table to operate on. This fits quite cleanly with named imports where conceptually what we want is the ability to configure the context-per-kebab-name. This in theory will keep the maintenance burden managable as there's still largely one source of truth for the implementation. This neatly works for allHost-style traits which are literally methods onWasi*CtxViewtypes, meaning theid-carrying versions actually do just acquire aWasi*CtxViewand then delegate the method. This requires more finesse for*WithStoretraits which work withAccessandAccessor, however. Theidparameter cannot be threaded into thefn(..)within theAccessor, so refactoring is performed where appropriate to make the implementation of each interface a one-liner to reduce duplication.The end result of all of this is that this is a very large commit but it's written in such a way that the Rust compiler in theory should catch all mistakes. In other words we're heavily relying on the type system and type checking here and don't ever rely on duplication of methods that hopefully-won't-change. There's a lot of traits and a lot of interfaces, hence the size of the commit, but conceptually everything is intended to be pretty simple.
Some design decisions as part of this commit, in no particular order:
IDs are represented by
wasmtime_wasi::NamedIdwhich is a newtype-wrapper aroundusize. The goal here is to enable an efficient implementation of dealing with ids. This notably forces the embedder to derive some sort of string-to-id (and perhaps back) map when adding items to a linker.All of this is opt-in and nothing is changed by default. For example the
wasmtimeCLI does not support any of this yet -- in theory that would require the ability to configure-Sflags per-named-import as opposed to all-at-once.Mapping a
NamedIdto a context is abstracted behind a trait rather than dictating that aVecorHashMapor similar is required. This increases the cognitive load when reading code (more generics), but avoids making this design decision within these crates and leaves exact representations up to embedders.The
HasDataimplementation can't reuse the preexistingWasiCli, and this uses a newWasiCliNamed<T>instead. This enables threading this trait-to-find-a-context to the right location for*WithStoretrait impls.Some miscellaneous
bindgen!issues have been fixed during this commit to ensure that this compiles and works correctly.An attempt has been made at documenting all the new primitives/structs/etc here. These are sort of difficult to align correctly unless you know what you're doing, so the documentation and examples are intended to serve as a way of spreading this knowledge.
One possible alternative I ended up deciding not to do was to put some sort of map-to-context storage within each preexisting context type. For example commit would be simpler for the
*WithStoreand infrastructure if it reused the exact sameSelftype as all other impls do. My thinking though is that this requires dictating the use of aHashMapor something else which I was hoping to avoid. Additionally the preexisting context structures are already minimal enough that they're basically what you already want as the source for each implementation, so I wanted to lean on them as much as possible.The main wrinkle in the new implementation is that
Accessorcarriesfn(..)to project out it'sD::Data<'_>which means that it can't close over any information. This feature needs to in theory close over anid: NamedId, however, and there's no easy way to put this square peg into a round hole. To work around this internal implementations withinwasmtime-wasi{,-http}now have a genericFparameter which is a closure which projects data, but this closure is typically only ever on the stack and doesn't make its way to the heap. This was one of the more awkward things to work around in this commit.The design here is intentionally done to help ensure that this commit is correct with minimal testing. It's not really feasible to duplicate the entire test suite just for named imports but these are duplicate trait impls which otherwise shouldn't be wrong. By ensuring that there's either strict delegation or each-function-is-at-least-one-line that the light amount of testing here is sufficient for keeping this working over time.
[diso]: https://github.com/spinframework/spin/pull/3708
Closes #14185
<!--
Please make sure you include the following information:
If this work has been discussed elsewhere, please include a link to that
conversation. If it was discussed in an issue, just mention "issue #...".Explain why this change is needed. If the details are in an issue already,
this can be brief.Our development process is documented in the Wasmtime book:
https://docs.wasmtime.dev/contributing-development-process.htmlPlease review the Bytecode Alliance's AI tool usage policy at
https://github.com/bytecodealliance/governance/blob/main/AI_TOOL_POLICY.mdPlease ensure all communication follows the code of conduct:
https://github.com/bytecodealliance/wasmtime/blob/main/CODE_OF_CONDUCT.md
-->
:memo: alexcrichton submitted PR review.
:speech_balloon: alexcrichton created PR review comment:
I'm not really sure myself what the end-state of this is all going to be, but my thinking for now is, yeah, we'd duplicate bindings for now with named imports.
I'm hesitant to always generate traits with
Option<Id>given the various use cases in play here. Thewasmtime-wasicrate in theory needs to be as-general-as-possible and as-flexible-as-possible to serve a relatively wide variety of use cases which is the main constraint. WithOption<Id>the implementation has to always be ready for either case ofNoneorSome, even in the situation when an embedder doesn't want to deal with named imports at all. Given that I'm hesitant to impose the runtime requirement that all embedders provide some piece of functionality to handle theSomeeven when that's never used. By having separate impls/functions it keeps the functionality needed to be provided by the runtime separate at least.Not that I think this is a great state of affairs though, it's not exactly pleasant having to duplicate things. While the risk of divergence is somewhat low due to all the generated traits and such it's still quite annoying to have to deal with. My rough thinking was to walk down the incremental "just add one more feature" approach for now and then see how far that takes us in maybe 6mo-1y time and look back to see if we can design things better.
For threading more information about the imports through, that's a good question and one I'd not considered! I'm always wary of runtime parameters/state though in the sense that it's on the hot path for all host functions as the state needs to be assembled. I'm also a bit wary of adding more generics into the system where
wasmtime-wasiis sort of already at the breaking point...One perhaps saving grace though is the
lookupis already passed the kebab-name of the import and that'd be a relatively natural place to put everything else. Basically we could hand everything over to the embedder and say "go ham with this but give back an id" and the embedder could record in its own tables the mapping from id to various import attributes.
alexcrichton updated PR #14275.
:thumbs_up: rvolosatovs submitted PR review:
LGTM apart from the
trappable_error_typeissue
alexcrichton updated PR #14275.
alexcrichton commented on PR #14275:
I tried a few different tacts for how to solve that and I ended up with the latest commit. @rvolosatovs mind double-checking that to make sure it's ok?
I'm realizing that the handling of trappable error types in named imports is kind of unfortunate where it's basically a lot more duplication than it otherwise is for non-named imports. I don't know of an easy way to solve that, but I do know of a hard way to solve that, and I basically decided to not do that just yet (documented in the code comments in the lates commit)
:thumbs_up: rvolosatovs submitted PR review:
Looks like using named imports with interfaces containing
unstabledoes not work:wasmtime::component::bindgen!({ inline: " package wasmtime:test; world test { import a: wasi:sockets/network@0.2.12; } ", path: "src/p2/wit", world: "wasmtime:test/test", with: { "wasi:sockets/network": wasmtime_wasi::p2::bindings::sockets::network, }, named_imports: { "wasi:sockets/network": wasmtime_wasi::NamedId, }, });fails with:
error[E0616]: field `network_error_code` of struct `wasmtime_wasi::p2::bindings::sockets::network::LinkOptions` is private --> crates/wasi/tests/all/p2/async_.rs:580:1 | 580 | / wasmtime::component::bindgen!({ 581 | | inline: " 582 | | package wasmtime:test; ... | 595 | | }, 596 | | }); | |__^ private field | = note: this error originates in the macro `wasmtime::component::bindgen` (in Nightly builds, run with -Z macro-backtrace for more info) help: a method `network_error_code` also exists, call it with parentheses | 596 | })(_);That said, it does not work on
maineither, where it fails with:error[E0425]: cannot find type `LinkOptions` in this scope --> crates/wasi/tests/all/p2/async_.rs:513:1 | 513 | / wasmtime::component::bindgen!({ 514 | | inline: " 515 | | package wasmtime:test; ... | 528 | | }, 529 | | }); | |__^ not found in this scope | = help: consider importing one of these structs: crate::p2::api::LinkOptions crate::p2::async_::named_wasi_import::LinkOptions = note: this error originates in the macro `wasmtime::component::bindgen` (in Nightly builds, run with -Z macro-backtrace for more info)so this does not appear to be a regression and probably can be fixed in a follow-up, therefore LGTM
:speech_balloon: rvolosatovs created PR review comment:
looks like this assertion fires with an invocation like this:
wasmtime::component::bindgen!({ inline: r#" package test:pkg; interface types { variant error { test, } } interface test { use types.{error}; test: func() -> result<_, error>; } world repro { import test; } "#, world: "repro", trappable_error_type: { "test:pkg/test.error" => String, }, imports: { default: trappable, } });Current
mainsilently ignores the config.Not sure there's anything worth fixing here, just calling it out. A nicer error message would probably help, but the panic is arguably an improvement compared to current
main
alexcrichton updated PR #14275.
:memo: alexcrichton submitted PR review.
:speech_balloon: alexcrichton created PR review comment:
I've gone ahead and removed the assertion for now and included this test case to at least ensure it's not a proc macro panic.
alexcrichton updated PR #14275.
alexcrichton commented on PR #14275:
I've spun out the unstable options issue to https://github.com/bytecodealliance/wasmtime/issues/14332 for now
alexcrichton has enabled auto merge for PR #14275.
alexcrichton added PR #14275 Add support for named imports to WASI implementations to the merge queue.
:check: alexcrichton merged PR #14275.
alexcrichton removed PR #14275 Add support for named imports to WASI implementations from the merge queue.
Last updated: Sep 20 2026 at 18:08 UTC