Bailey Hayes (Nov 22 2022 at 16:13): Bailey Hayes (Nov 22 2022 at 16:13):How do you see the WasiCtx and WasiConfig idioms changing with the component model? If I build for components, the current tuple doesn't reflect whether I built for wasi. The component storedata doesn't appear to integrate with a WasiCtx.
Lann Martin (Nov 22 2022 at 16:17):For wasi integration with the e.g. Rust stdlib it would have to be part of the target tuple
Lann Martin (Nov 22 2022 at 16:18):https://github.com/bytecodealliance/cargo-component#wasi-support
Bailey Hayes (Nov 22 2022 at 16:18):Most of the examples and tests I've seen so far don't use WASI if built with components. WASI isn't mentioned in the tracking issue: https://github.com/bytecodealliance/wasmtime/issues/4185. I'm looking at lower layers than cargo-component and I'm wondering if there's a gap we missed here?
Lann Martin (Nov 22 2022 at 16:22):I think the gap is just "WASI for components", which is in progress
Bailey Hayes (Nov 22 2022 at 16:24):I'm looking for the design or tracking doc for how WASI is supposed to work with components.
Lann Martin (Nov 22 2022 at 16:32):I see. I think WASI component support doesn't really fall "under" the component model work, it just depends on it. I haven't seen any formal tracking of work on it.
Bailey Hayes (Nov 22 2022 at 16:36):@Luke Wagner or @Alex Crichton I'm writing out an elixir OTP host that embeds wasmtime. The last part that's fuzzy for me is around how the component model will work with WASI. Is WASI optional? The way I understand things, it should be optional. Earlier versions of this host created a module and inspected for WASI imports and then built up a wasi store if the module was wasi enabled. This is a little awkward with the component-model feature enabled. I want this host to work with wasm32-unknown-unknown ABI, wasm32-wasi-unknown, and wasm32-<to be decided component triple>. Right now I'm building out the API to compile and instantiate a module or component (even if the API is just stubs). I have a component OR module working, just not WASI enabled. I'm not sure how and where I should pass around the WasiCtx in the component case. If that's going to be part of the StoreData for the component, easy-peasy.
Alex Crichton (Nov 22 2022 at 16:39):The current thinking, which isn't well-written down and is still being figured out, is:
wasm32-wasi*.witx to new-style component model*.wit filesDoes that make sense?
Luke Wagner (Nov 22 2022 at 16:45):I agree that conceptually "WASI" should be optional to use when building a component (just like when using a core module). I would even hope that, in the final goal state, there is need for a "WASI store", which was in earlier times necessitated by the magical nature of WASI which, with the component model, should cease to be magical at all (as a consequence of being virtualizable). But I can't speak to whether there are any (perhaps temporary) limitations that tie the two together atm.
Bailey Hayes (Nov 22 2022 at 16:46):wasi-preview2 has worlds and wit which is good and I'm following. But even if it defines wit, I haven't seen a sketch for how to pass around environment variables, stdin, etc specific to non-core components. I don't think we want the same wasictx for a component and all of its dependencies. I'm really asking for what the union between how a component imports wasi ctx's.
A possible answer is: from the wasmtime API, we expect every ComponentStore to optionally have a WasiCtx. Some external definition like from a world file would make it so that you can recurse through the components and populate these. But the reason I ask, is that I'm wondering if instead we could do away with the WasiCtx. This seems like info that we could embed within each component dynamically as part of the composition against a world file.
Bailey Hayes (Nov 22 2022 at 16:49):Heh and @Luke Wagner already thinking around what I was asking. Yes, it seems like the idea of a WasiCtx is a thing that's grafted on and not integrated with components (and should be done away with in API's like wasmtime).
Alex Crichton (Nov 22 2022 at 16:51):For things like env vars and such it's a work-in-progress still, dan/pat are working actively on this and should have more to show soon
Alex Crichton (Nov 22 2022 at 16:51):They're out this week but I'd recommend reaching out to them for more information
Bailey Hayes (Nov 22 2022 at 16:52):and to be clear, I don't expect working code or even code to exist. I'm really just thinking around my high-level stubs so that I can publish an alpha and iterate as the implementations roll out
Lann Martin (Nov 22 2022 at 16:53):There is nothing in the current wasmtime component implementation that would get rid of WasiCtx (or something just like it)
Bailey Hayes (Nov 22 2022 at 16:54):it also doesn't add it
Lann Martin (Nov 22 2022 at 16:55):Well, if you have a host implementation of wasi (like wasmtime-wasi today), that implementation is going to have state associated with it, and that state has to be part of the Store data. I haven't seen anything about changing that general model, and wasmtime's component Linker still works that way
Lann Martin (Nov 22 2022 at 16:57):That isn't intrinsic to WASI, but to any host implementation of any interface
Bailey Hayes (Nov 22 2022 at 16:58):If I have a composed component, then in the imports it should have all of the information needed about what would have previously been in a WasiCtx. As a host implementer, those details are likely no longer needed in the API, hence the original question, how are we expecting those idioms to change.
The best I can find for what is planned here is: https://github.com/WebAssembly/component-model/blob/main/design/mvp/examples/SharedEverythingDynamicLinking.md#runtime-dynamic-linking
Alex Crichton (Nov 22 2022 at 16:59):Hm sorry I'm getting a bit lost in what you're asking about, it seems you're asking about both how embeddings will work and how guests are generated and how guests will work?
Alex Crichton (Nov 22 2022 at 16:59):The WasiCtx state isn't going away, the idea it is "just picks up new trait impls" for wit-bindgen-generated traits for the wasi-preview2 interfaces
Alex Crichton (Nov 22 2022 at 17:00):The wasm32-wasi target is not expected to go away and it's not planned at this time to add a new component target, instead the existing target will be used with an adapter, under development, to transform compiler output to a component
Alex Crichton (Nov 22 2022 at 17:00):The difference between core-wasm-wasi and component-wasi is the output artifact, whether it's a module or a component
Alex Crichton (Nov 22 2022 at 17:00):In that the embedder APIs for instantiating components are also completely different than those of modules -- similar in nature but different entrypoints.
Bailey Hayes (Nov 22 2022 at 17:05):Say I compose a component with a world for wasi-cloud. Will the imports in the component-wasi artifact include additional information like "envVars" for a sub-component?
Alex Crichton (Nov 22 2022 at 17:05):I don't have a great way to answer that I think, I can talk about various layers but I don't know if that's what you're looking for
Alex Crichton (Nov 22 2022 at 17:06):the current rough plan is that a wasi command takes env vars as a parameter to the "start" function more or less
Lann Martin (Nov 22 2022 at 17:07):That would be for preview2, right?
Lann Martin (Nov 22 2022 at 17:08):@Bailey Hayes I have to do this same work for Spin. I've just been putting it off :smile:
Bailey Hayes (Nov 22 2022 at 17:09):I'm looking to get fully plugged in so that I can contribute better to upstream
Alex Crichton (Nov 22 2022 at 17:09):Yeah passing env vars as an argument would be a preview2 thing
Alex Crichton (Nov 22 2022 at 17:09):(if that's what's settled on)
Bailey Hayes (Nov 22 2022 at 17:13):preview2 or hypothetical world of ideal component support, I have still only seen how we think reactors will work, e.g. like stdin/stdout being passed around starting with the core module acting as a CLI. In that case you'd have a full WasiCtx to instantiate the component. Instead, in this future world, I want to see that the composed component has all of the right information, e.g. the blobstore component only has the ctx needed for connecting to the blobstore. This can be specified as part of the world file so that it's clear what WASI ctx goes where. Then the question for a host implementer using Wasmtime is why would I instantiate the WasiCtx and store with that information when this is something that should come from parsing a component?
Lann Martin (Nov 22 2022 at 17:17):You still need state for wasi to track e.g. open files. That state has to be reachable from the Store data. You could potentially replace some of what WasiCtxBuilder does with magical import-based config but you can't remove WasiCtx
Bailey Hayes (Nov 22 2022 at 17:22):Yeah I think a magic WasiCtxBuilder is what I'm looking for. Other constraints is that we may want to be able to change connection strings or other data at runtime. I don't want it locked into something after calling the start function (fine for instantiation, but needs to be mutable).
Bailey Hayes (Nov 22 2022 at 17:40):I think I landed on what I needed. I'm not going to expose a WasiCtx to mix-in for components since that appears to be a [future] anti-pattern. A world file is a safer and idiomatic way to initialize a component's WasiCtx. WasiCtx's can be attached to an individual component's store using a utility like WasiCtxBuilder. The populated Store will have a list (tree?) of components, each with a WasiCtx that the caller can add in overrides. hand waves
module API: WasmRuntime.Wasmtime.start_link(%{module: module, wasi: wasi})
component API:
WasmRuntime.Wasmtime.build_store(component) # calls WasiCtxBuilder
# mixin wasi opts, then compile and instantiate
WasmRuntime.Wasmtime.start_link(%{component: component})
Bailey Hayes has marked this topic as resolved.
Alex Crichton (Nov 22 2022 at 17:58):The Store-specific state is disconnected sort of from WASI and how a component looks internally. The Store state is just used to satisfy the how imports of the final component, and whatever is used internally within the component is unknown to the store/embedder
Last updated: Nov 22 2024 at 17:03 UTC