Joel Dice (Nov 06 2023 at 18:01): Joel Dice (Nov 06 2023 at 18:01):Context: I'm working on adding resource support to wasm-compose and wasi-virt. In the process, I was going to update the middleware example in the wasm-tools repo to use wasi-http-style resources to represent requests and responses. However I immediately ran into a problem which leads me to wonder if implementing general-purpose middleware using component composition for wasi:http/incoming-handler might not be possible.
The example in the wasm-tools repo involves composing a service component with a middleware component such that the latter compresses the body of the response returned by the former. That works just fine when the response is represented as a record with a list<u8> body. With wasi-http, though, both requests and responses are represented as resources, and the signature of the wasi:http/incoming-handler#handle function takes IncomingRequest and ResponseOutparam resource handles. Ideally, the middleware component would export the handle function from the host and import it from the service component. Moreover, the middleware component should import the relevant resource types from the host and export its own "virtualized" versions of those resources to the service component. However, a component cannot import a function which refers to types that it is exporting, which means the handle function it imports from the service component can only refer to the host's versions of those resource types.
Is there a way around this? The best I can come up with so far (and this just occurred to me, so it's a half-baked idea) is to add a third component to the above scenario which exports virtualized versions of the relevant resources plus some kind of "hook" API the middleware component can use to intercept the resource method calls it cares about. Then both the middleware and the service component would import the resource types from the third component, which in turn would import the "real" versions from the host.
Joel Dice (Nov 07 2023 at 20:39):Just had a good conversation with @Luke Wagner about this, which I'll try to summarize here (any errors are my own, which perhaps he'll be kind enough to correct).
First, you can create a component which both imports from and exports to another component. This can't be expressed in WIT and can't be done using wasm-compose (to my knowledge), but you could theoretically build a wrapper component around the original one such that the wrapper instantiates the original and handles the cyclical relationship using lifted functions which use call_indirect at the core Wasm level. In fact, you can create arbitrary graphs of components containing cycles as long as the cycles are "broken" using call_indirect. Presumably wasi-virt does something like this.
Second, you can always do service chaining the old-fashioned way by having the middleware component do an outbound HTTP request to the service component (which the host might optimize if it notices the service can be invoked without using the network at all). I.e. you don't need component composition to do service chaining, and you probably won't want to use it for that until Preview 3.
Once Preview 3 arrives, we'll be able to do efficient, local service chaining using component composition, and this will not require virtualizing resource types. In my middleware/service example, the middleware component can call the service component's handle function, which returns a Response resource, from which middleware can extract the body as a stream (and anything else it wants), create a new Response with new stream, and pipe from the original stream to the new one while applying any desired transformations (e.g. compression). None of that requires virtualizing any resource types, breaking cycles, etc. (i.e. it can all be done with standard, wasm-compose-style composition).
kellytk (Nov 14 2023 at 15:13):Re: last paragraph, is the juggling kinda similar to how it goes with Rust's std::process::Command? (stdin et al)
Joel Dice (Nov 14 2023 at 17:06):Yeah, that seems like a good analogy. You can create multiple Commands and pipe the stdout of one to the stdin of another, using the equivalent plumbing to what a POSIX shell would do when you use the | (pipe) operator.
Kate Goldenring (Nov 14 2023 at 22:25):@Joel Dice thank you for the summary. Just a couple clarifying questions.
You mention, "I.e. you don't need component composition to do service chaining, and you probably won't want to use it for that until Preview 3." Service chaining seems to be a really big use case that WASI HTTP + preview 2 has unlocked. I am not sure I understand why we don't want to use it yet, even without WASI virt.
I just want to clarify why we think preview 3 will solve this. It sounds like with preview 3 we are avoiding needing to import/export Types? Is this a goal or side effect of preview 3. Just catching up.
Joel Dice (Nov 14 2023 at 22:26):Guy and I just realized wasi-virt --allow-http is broken for exactly the reason I described above. It currently relies on wasm-compose, which can't handle cycles (as noted earlier). If you're only doing outbound requests, you're fine (or at least you will be once I PR a fix for that), but if you're handling inbound requests you hit the cycle issue.
In my summary above, I basically concluded that we don't need or want virtualization to implement middleware anyway, but Guy reminded me that if you're not virtualizing wasi-http, you can't use wasi-virt to virtualize anything involving streams. That means no virtual filesystem, etc. At most, you can virtualize the bits that don't involve streams (e.g. env, clocks, and random), but that's it. Kind of a sad state of affairs.
So now Guy and I are thinking we do want to support virtualizing all of wasi-http, including inbound, which I think means we need to do one of the following:
wasm-composewasm-compose in wasi-virt and implement a custom component wrapping solutionwasi-virt can use Joel Dice (Nov 14 2023 at 22:30):@Kate Goldenring: Till and I believe the best way to do general purpose service chaining with Preview 2 is by locally connecting outbound requests to incoming handlers without going over the network. That allows us to do concurrent I/O with minimal buffering, which is not possible otherwise via composition given the current state of the component model. For middleware applications that don't need to stream request or response bodies across component boundaries, composition can still be a good option, though.
Joel Dice (Nov 14 2023 at 22:33):@Kate Goldenring: The plan for Preview 3 is to include native async and stream support in the component model, which will allow concurrent I/O across composed component boundaries.
Pat Hickey (Nov 14 2023 at 22:35):one way to explain this that may be helpful: in preview 2, scheduling of concurrent IO is managed with wasi:io/poll and wasi:io/stream, which are component model interfaces that expose resources (pollable, input-stream, output-stream)
Pat Hickey (Nov 14 2023 at 22:36):in order to import a component model interface that exposes any sort of io (meaning, anything in the interface returns a pollable or stream, like the wasi-http interface does), you also need to import the wasi:io package's interfaces from that same component
Pat Hickey (Nov 14 2023 at 22:37):and you can only have a single import of the wasi:io package's imports per component
Pat Hickey (Nov 14 2023 at 22:37):this effectively means, every time you import an interface such as http, you have to import it from a component that provides the entire world of io and every other interface that uses io
Joel Dice (Nov 14 2023 at 22:38):@Peter Huene do you have thoughts on the wasi-virt situation I described above?
Pat Hickey (Nov 14 2023 at 22:39):wasi-virt is the only component that tries to provide this complete wrapper of your world, but wasi-virt itself is incomplete so it doesnt actually do it yet. once joel (or someone else) fixes the problems with wasi-virt, it will provide the entire set of interfaces you import, even if all you need to do is override one part of the filesystem, it needs to virtualize your http as well
Pat Hickey (Nov 14 2023 at 22:39):ive called this style of component composition "onion composition" previous, because each layer you add on has to wrap the entire other layer, like an onion
Pat Hickey (Nov 14 2023 at 22:40):in short, it is not ideal. it is just the way these things have to work, at the moment, because wasi:io is an ordinary wit package.
Pat Hickey (Nov 14 2023 at 22:41):the goal for preview 3 (though, preview 3 is just whatever ships at the end of 2024, and who knows if this will be complete, so id actually prefer we start calling it something else like "first-class component model async" or something) is that instead of being a wit package, the primitives for scheduling and streams become component model primitives
Pat Hickey (Nov 14 2023 at 22:41):at which point we have a lot more power with how they can be implemented, they dont have to follow many of the rules of the canonical ABI as it exists today
Pat Hickey (Nov 14 2023 at 22:41):and then we can have composition of components that use the stream and scheduling primitives without having to do that onion composition
Pat Hickey (Nov 14 2023 at 22:43):among other things, that will make service chaining between http much simpler, because one element in the chain wont have to onion-wrap the other - if the inner element needs filesystem, the outer element doesnt have to virtualize that filesystem
Pat Hickey (Nov 14 2023 at 22:43):hopefully all of that makes sense, please let me help clarify if it doesnt!
Joel Dice (Nov 16 2023 at 21:10):Here's an update on the state of wasi-virt support for wasi:http/incoming-handler. First, a summary of the problem:
wasi-virt model of using wasm-compose to compose a virtualization component (virt.wasm) with an application component (app.wasm) won't work if app.wasm targets wasi:http/incoming-handler. Specifically, virt.wasm cannot import wasi:http/incoming-handler#handle from app.wasm because the parameter types (incoming-request and response-outparam) are the types virt.wasm is exporting to app.wasm.wasi:http/types, plus some helper functions from virt.wasm to convert from the former to the latter. wrapper.wasm exports a version of wasi:http/incoming-handler#handle that accepts the host's versions of incoming-request and response-outparam, uses the helper functions from virt.wasm to convert them to the virtualized versions of those types, and calls app.wasm's version of wasi:http/incoming-handler#handle. This can all be expressed in the component model, but not in WIT or any tool that deals with WIT.After discussing this with @Alex Crichton, we've concluded that although it's technically possible to implement the three-component solution described above with some low-level WASM file manipulation, it would be better to add syntax to WIT which allows it to be expressed at that level. That's not something we'll want to tackle until after WASI Preview 2 ships, though. So we feel that this is not the right time to try to support wasi:http/incoming-handler in wasi-virt. Instead, we plan to make the WIT syntax additions a priority post WASI Preview 2, at which point we'll have what we need to work on wasi-virt.
Joel Dice (Nov 16 2023 at 21:17):Here's a diagram, in case it helps.
cycle.drawio1.png
Luke Wagner (Nov 20 2023 at 20:26):Great to hear you're digging into virtualization, and these challenges make sense. Two ideas:
wasm-compose today using wasm-compose's existing input syntax, @Peter Huene and others have been working hard on a new input syntax for wasm-compose that captures pretty much the full expressivity of component linking (and with a new readable curly-brace syntax in the style of, and literally super-setting WIT). I'm pretty sure your use case should be quite neatly expressible in this syntax (tentatively called "WAC" for "WebAssembly Composition"), so maybe ping Peter to use your use case as a motivating example?instantiate). The challenge here is that it's less clear what the developer tooling should look like b/c we're authoring core functions that are lifted into the child component (passed in via with, not exported) and we're lowering exports of the child component (accessed via alias export, not imported). I've been imagining what this might look like using a proc-macro on a static variable in Rust setting and using ESM future features (based on source-phase imports) in a JS setting, so I'm happy to chat about that in more detail some time. Ryan Levick (rylev) (Dec 01 2023 at 17:34):I've created a POC for doing middleware style chaining using wasi-http: https://github.com/rylev/wasi-http-middleware-prototype
The middleware calls down the middleware chain by calling into its outgoing-handler import which is intercepted by the runtime and forwarded to the downstream component which receives the request (without knowing that it actually came from a middleware component).
Currently the demo code always forwards the outgoing-handler's request to the downstream components, but we'll want to come up with some protocol for distinguishing between passing a request down a middleware chain and making a legitimate HTTP call. @Lann Martin had the idea of using the special TLD .alt for this.
Last updated: Nov 22 2024 at 17:03 UTC