pchickey opened PR #12496 from pchickey:component_type_to_val to bytecodealliance:main:
<!--
Please make sure you include the following information:
If this work has been discussed elsewhere, please include a link to that
conversation. If it was discussed in an issue, just mention "issue #...".Explain why this change is needed. If the details are in an issue already,
this can be brief.Our development process is documented in the Wasmtime book:
https://docs.wasmtime.dev/contributing-development-process.htmlPlease ensure all communication follows the code of conduct:
https://github.com/bytecodealliance/wasmtime/blob/main/CODE_OF_CONDUCT.md
-->
pchickey requested alexcrichton for a review on PR #12496.
pchickey requested wasmtime-core-reviewers for a review on PR #12496.
pchickey edited PR #12496.
pchickey edited PR #12496:
This PR adds the method
fn to_val<S>(&self, store: StoreContextMut<S>) -> Result<Val>to wasmtime'sComponentTypetrait. This is a first step in a bigger project to provide better debugging facilities for wasmtime component model execution.Context
Wasmtime separates component model values into two separate worlds:
- a typed representation, where Rust datatypes corresponding to component model types derive or impl the
ComponentTypetrait, plus optionally theLiftandLowertraits (required to be used in arguments and return values of import functions, respectively) which provide wasmtime the means to convert to/from each Rust type and its canonical ABI representations. To use this representation for an import function wasmtime users usewasmtime::component::Linker::func_wrapand friends, and on an export function usewasmtime::component::Instance::typed_funcand friends.- a tagged representation, where all component model types are represented using the different variants of the
wasmtime::component::Valenum. To use this representation for an import function, wasmtime users usewasmtime::component::Linker::func_newand on an exportwasmtime::component::Instance::func.Prior to this PR, users have to choose whether to opt into the typed representation or the tagged representation based on what their application needs, and we (the wasmtime developers) focused most of our efforts on the typed representation because that ends up being the right choice for most embedders.
Users of the typed representation do so almost exclusively through the use of
wasmtime::component::bindgen!, a proc macro which generates Rust structs and enums corresponding to component model types it parses from wit, and delegates to the#[derive(ComponentType, Lift, Lower)]derive proc-macros in those generated types (which themselves also are provided by the wasmtime crate). This bindings generation typically takes place in library code, e.g.wasmtime-wasi,wasmtime-wasi-httpand related crates, andbindgentakes a variety of options that influence exactly what sort of Rust representation to use for wit types - theComponentTypederive macros permit flexibility in various ways.On the other hand, the tagged representation is useful for library code that exists independently from a given wit description. The library for a human-friendly serialization format for component model values
https://crates.io/crates/wasm-waveexports some traits which wasmtime provides impls for in terms ofwasmtime::component::Val. Wave ends up being useful for features inwasmtime-clilikewasmtime run --invoke ..., which I introduced in https://github.com/bytecodealliance/wasmtime/pull/10054. The C API, which is used for bindings to other languages, also uses the Val representationIn most Rust wasmtime embeddings the tagged representation isn't actually used much, because "serious" embeddings are all written in terms of the typed representation.
My goal is to create better debugging tools for wasmtime component model programs, providing wasmtime embedders and cli users with the capability to inspect, and eventually record/replay or modify component model values. Like the wave crate and invoke integration, debugging tools benefit from being independent of a given wit description. Debugging tools must integrate with wasmtime embeddings regardless of whether they use the typed or tagged representation to implement import funcs and call export funcs, and since most embeddings are making substantial use of
bindgen!in public and private crates against public and private wits, my approach is to provide a bridge where a debugging library can operate on just the tagged representation of values, using a new mechanism built into all typed representations to convert to tagged representation. Further mechanisms in wasmtime to hook into library code using the typed representation will come in future PRs, see https://github.com/bytecodealliance/wasmtime/pull/12532 for work in progress.Converting typed to tagged
For every value in a typed component model value representation (i.e. any value of a type which has an
impl ComponentType), there also exists an equivalent component model value inwasmtime::component::Val(tagged) representation. Prior to this PR, there was no mechanism to convert a typed value to a tagged one.This PR adds a method
to_valtoComponentTypeto convert to a Val:fn to_val<S>(&self, store: StoreContextMut<S>) -> Result<Val>The design choices in this signature are:
* The method takes&self, notself, because the reverse conversion does not exist, and in the context of adding debugging hooks to existing code using the typed representation, we don't want to destroy the typed representation in order to observe it.
* The method gives the owned representationVal, which means making a structural clone of the value. For a witlist<u8>this ends up having some fairly ugly overhead: the typicalVec<u8>, which implsComponentType, Lift, Lower, converts to theVal::List(Vec<Val>)variant, where each vec element is then aVal::U8(u8)variant. Other variants, likeVal::Record,Val::Variant,Val::Enum, andVal::Flagsend up usingStringrepresentations of the various wit identifiers used for record rows, variant and enum tags, and each flag value. This requires substantially more memory to represent, and is substantially less efficient to traverse!
* The owned representation ofValhas further caveats where it does not provide the same safety guarantees as the typed representation for resource, future, and stream representations, see https://docs.rs/wasmtime/latest/wasmtime/component/struct.ResourceAny.html for information on the requirements for dropping a resource, which also apply when in a leaf node of aVal. WhenResource<T>::to_val(&self)is invoked, FIXME FIGURE OUT WHAT HAPPENS HERE AND WHAT THE USER NEEDS TO DO TO HANDLE IT. DOES VAL NEED Adrop{,_async}(self,StoreContextMut)METHOD? and put whatever i learn in the proper docs
* The conversion requires a store. I originally usedimpl AsContextMutfor the store, but Alex suggested I switch to useStoreContextMut, which then requires the function to be generic on the type in the store. For this type variable I useSinstead of the usualTbecauseTis often already in use in the impl.
* The conversion is fallible. The conversion can fail if you use the wrong store for a resource type. Soon, when Nick gets around to changing the Val representation to the new fallible allocation types, it will also be fallible due to OOM, so its nice that the general shape of things is there.The
ComponentTypederive macro fills out a definition of to_val mechanically - this was the only aspect of the PR that was subtle to figure out. All of theimpl ComponentTypefor Rust types provided by wasmtime get a straightforward mechanical implementation.TODO before this PR can land:
- [ ] literally any testing at all. im confident that at the time of writing not a single invocation of to_val has ever been executed. whatever it typechecked
- [ ] rust docs need better
- [ ] Alex gets another pass at the design, which his feedback already substantially improved.
pchickey edited PR #12496:
This PR adds the method
fn to_val<S>(&self, store: StoreContextMut<S>) -> Result<Val>to wasmtime'sComponentTypetrait. This is a first step in a bigger project to provide better debugging facilities for wasmtime component model execution.Context
Wasmtime separates component model values into two separate worlds:
- a typed representation, where Rust datatypes corresponding to component model types derive or impl the
ComponentTypetrait, plus optionally theLiftandLowertraits (required to be used in arguments and return values of import functions, respectively) which provide wasmtime the means to convert to/from each Rust type and its canonical ABI representations. To use this representation for an import function wasmtime users usewasmtime::component::Linker::func_wrapand friends, and on an export function usewasmtime::component::Instance::typed_funcand friends.- a tagged representation, where all component model types are represented using the different variants of the
wasmtime::component::Valenum. To use this representation for an import function, wasmtime users usewasmtime::component::Linker::func_newand on an exportwasmtime::component::Instance::func.Prior to this PR, users have to choose whether to opt into the typed representation or the tagged representation based on what their application needs, and we (the wasmtime developers) focused most of our efforts on the typed representation because that ends up being the right choice for most embedders.
Users of the typed representation do so almost exclusively through the use of
wasmtime::component::bindgen!, a proc macro which generates Rust structs and enums corresponding to component model types it parses from wit, and delegates to the#[derive(ComponentType, Lift, Lower)]derive proc-macros in those generated types (which themselves also are provided by the wasmtime crate). This bindings generation typically takes place in library code, e.g.wasmtime-wasi,wasmtime-wasi-httpand related crates, andbindgentakes a variety of options that influence exactly what sort of Rust representation to use for wit types - theComponentTypederive macros permit flexibility in various ways. But, at any rate, to my knowledge every "serious" embedding of wasmtime (rough definition: engineers ship it to prod at dayjob) is written in Rust and usesbindgenexclusively or very close to it.On the other hand, the tagged representation is useful for library code that exists independently from a given wit description. The library for a human-friendly serialization format for component model values
https://crates.io/crates/wasm-waveexports some traits which wasmtime provides impls for in terms ofwasmtime::component::Val. Wave ends up being useful for features inwasmtime-clilikewasmtime run --invoke ..., which I introduced in https://github.com/bytecodealliance/wasmtime/pull/10054. The C API, which is used for bindings to other languages, also uses the Val representation, because C APIs and the bindgen proc macro are oil and water.My goal is to create better debugging tools for wasmtime component model programs, providing wasmtime embedders and cli users with the capability to inspect, and eventually record/replay or modify component model values. Like the wave crate and invoke integration, debugging tools benefit from being independent of a given wit description. Debugging tools must integrate with wasmtime embeddings regardless of whether they use the typed or tagged representation to implement import funcs and call export funcs, and since most embeddings are making substantial use of
bindgen!in public and private crates against public and private wits, my approach is to provide a bridge where a debugging library can operate on just the tagged representation of values, using a new mechanism built into all typed representations to convert to tagged representation. Further mechanisms in wasmtime to hook into library code using the typed representation will come in future PRs, see https://github.com/bytecodealliance/wasmtime/pull/12532 for work in progress.Converting typed to tagged
For every value in a typed component model value representation (i.e. any value of a type which has an
impl ComponentType), there also exists an equivalent component model value inwasmtime::component::Val(tagged) representation. Prior to this PR, there was no mechanism to convert a typed value to a tagged one.This PR adds a method
to_valtoComponentTypeto convert to a Val:fn to_val<S>(&self, store: StoreContextMut<S>) -> Result<Val>The design choices in this signature are:
* The method takes&self, notself, because the reverse conversion does not exist, and in the context of adding debugging hooks to existing code using the typed representation, we don't want to destroy the typed representation in order to observe it.
* The method gives the owned representationVal, which means making a structural clone of the value. For a witlist<u8>this ends up having some fairly ugly overhead: the typicalVec<u8>, which implsComponentType, Lift, Lower, converts to theVal::List(Vec<Val>)variant, where each vec element is then aVal::U8(u8)variant. Other variants, likeVal::Record,Val::Variant,Val::Enum, andVal::Flagsend up usingStringrepresentations of the various wit identifiers used for record rows, variant and enum tags, and each flag value. This requires substantially more memory to represent, and is substantially less efficient to traverse!
* The owned representation ofValhas further caveats where it does not provide the same safety guarantees as the typed representation for resource, future, and stream representations, see https://docs.rs/wasmtime/latest/wasmtime/component/struct.ResourceAny.html for information on the requirements for dropping a resource, which also apply when in a leaf node of aVal. WhenResource<T>::to_val(&self)is invoked, FIXME FIGURE OUT WHAT HAPPENS HERE AND WHAT THE USER NEEDS TO DO TO HANDLE IT. DOES VAL NEED Adrop{,_async}(self,StoreContextMut)METHOD? and put whatever i learn in the proper docs
* The conversion requires a store. I originally usedimpl AsContextMutfor the store, but Alex suggested I switch to useStoreContextMut, which then requires the function to be generic on the type in the store. For this type variable I useSinstead of the usualTbecauseTis often already in use in the impl.
* The conversion is fallible. The conversion can fail if you use the wrong store for a resource type. Soon, when Nick gets around to changing the Val representation to the new fallible allocation types, it will also be fallible due to OOM, so its nice that the general shape of things is there.The
ComponentTypederive macro fills out a definition of to_val mechanically - this was the only aspect of the PR that was subtle to figure out. All of theimpl ComponentTypefor Rust types provided by wasmtime get a straightforward mechanical implementation.TODO before this PR can land:
- [ ] literally any testing at all. im confident that at the time of writing not a single invocation of to_val has ever been executed. whatever it typechecked
- [ ] rust docs need better
- [ ] Alex gets another pass at the design, which his feedback already substantially improved.
pchickey updated PR #12496.
pchickey updated PR #12496.
pchickey closed without merge PR #12496.
pchickey commented on PR #12496:
@alexcrichton and I just discussed this in depth and we concluded there is no correct way to deal with the fact that Val contains ResourceAny / FutureAny / StreamAny in this context - constructing any of those must be done as a move of the typed version. So, unfortunately, this work is scrapped for now, and I have another completely different approach I will be starting on instead.
Last updated: Feb 24 2026 at 04:36 UTC