Catherine (whitequark) (Jan 22 2024 at 23:47): Catherine (whitequark) (Jan 22 2024 at 23:47):I've noticed that when composing components, jco (or wasm-tools compose... I'm not sure what's the best place to post this thread but I assume all the relevant parties do read the jco stream) leaves all of the constituent core wasm modules as-is and leaves it to the host to connect imports with exports
I think it would be valuable to have a capability for flattening components, i.e. combining all of the core wasm modules into one by (a) using multi-memory and multi-table to provide the same strong isolation (b) rewriting all of the memory and table instruction indices (c) generating Wasm trampolines for export-to-import connections that are internal to the flattened core wasm modules
the output would still be a component and jco should be able to bind it to JS just as you can now, but there are several benefits to this approach which currently cannot be reaped:
I'm interested to work on this, but it's a fairly big task, so I'd like to know first where should I work on it (wasm-tools I assume), whether such a contribution is desired, and what are the acceptance criteria
I'm personally most worried about the trampolines. I am sure I can get 80% of the way there, but nailing down every corner case of the canonical ABI is going to be extremely difficult; it is a big and complex ABI that is also new, and I am sure to run into difficulties as one of the first (the first, possibly?) non-BA engineers to attempt implementing it
Yoshua Wuyts (Jan 23 2024 at 15:54):Am I understanding it right that the proposed flattening approach would likely be less cache-friendly? If I'm following, then every update would create a new flattened big blob, rather than having multiple smaller blobs which can be individually cached and invalidated? So it's trading off properties for different ones, and so we'd likely end up wanting to support both approaches?
Yoshua Wuyts (Jan 23 2024 at 15:54):Catherine (whitequark) said:
I'm interested to work on this, but it's a fairly big task, so I'd like to know first where should I work on it (wasm-tools I assume), whether such a contribution is desired, and what are the acceptance criteria
These sound like @Guy Bedford questions to answer.
Catherine (whitequark) (Jan 23 2024 at 17:00):If I'm following, then every update would create a new flattened big blob, rather than having multiple smaller blobs which can be individually cached and invalidated?
Correct.
So it's trading off properties for different ones, and so we'd likely end up wanting to support both approaches?
Correct.
My main use case for Wasm, YoWASP, ships all of the pieces together, and the main core Wasm file is something like 15 MB in size (I think this will grow up to 150 MB after I start using WASI-Virt and bundle all of the files I'm currently unconditionally downloading anyway), I'm not particularly concerned about the inability to cache the adapter and other accessory modules.
Yoshua Wuyts (Jan 23 2024 at 17:10):okay, that makes sense to me - thanks for clarifying!
Guy Bedford (Jan 23 2024 at 18:55):The reason the trampolines are needed in the first place is due to V8 not having multi-memory support. That story is changing fast though since it is now getting supported in the latest JS platforms.
For those that are happy to only support the latest platforms, we could therefore add a --multi-memory option to jco transpile that could skip unnecessary adapter construction when using JCO. This work would be amazing to explore if someone is interested (looks to @Alex Crichton :P)
@Catherine (whitequark) so I think splitting up such a problem into (a) static component linking optimizations and (b) jco multi-memory support.
For (a) the question then is if this is really needed and what components you are practically looking at here. For WASI-Virt in particular, if you've got an application with multiple components all using WASI-Virt we probably don't want to have them all optimized in this way --- instead, we probably want a tool that does the opposite! That can extract out the shared WASI-Virts and treat them as a shared dependency that gets instantiated for each of them somehow.
All in all, theres a huge space of very very interesting problems to solve here, so hope that helps find focus as some initial feedback.
Catherine (whitequark) (Jan 23 2024 at 20:57):For those that are happy to only support the latest platforms, we could therefore add a --multi-memory option to jco transpile that could skip unnecessary adapter construction when using JCO.
That would be nice.
The reason the trampolines are needed in the first place is due to V8 not having multi-memory support.
When I said "trampoline" I meant "function that transfers data from one linear memory to another". It seems I've misused the terminology. As long as you have multiple address spaces you will need such a function, I believe, and as far as I know this isn't currently something that jco can generate, instead relying on the host Wasm engine to transfer data between modules. Correct me if I'm wrong, please.
For (a) the question then is if this is really needed and what components you are practically looking at here.
What I would like to have is a command component that runs another command component via something like popen(). The idea here is that toolchains often come in multiple pieces that are all UNIX commands (like gcc, having cc1 and as and ld as the subcommands) and you can't really make the whole thing work unless a driver command can call subcommands. (I work with FPGA toolchains, which are slightly different but the general idea is the same: you have a popen() or posix_spawn() call to a statically known target or at least one of a known list, and the call not reentrant, but can be made several times.) So I'd like to be able to package the entire thing as one single WASI command, which would work on any WASI+CM host without the need to embed each piece individually.
I wanted this for a long time. Actually, first I wanted to embed the auxiliary filesystem data often used by toolchains (think /usr/include) in the command, which WASI-Virt solves now. So now I'm thinking about the next stage of the process.
Catherine (whitequark) (Jan 23 2024 at 20:58):The reason I bring up flattening is that when working on a single core module it's much easier to polyfill e.g. a "restart" entry point (which lets you run a single instantiation of a command more than once) than if you're working with several interconnected core modules.
Guy Bedford (Jan 23 2024 at 21:06):When I said "trampoline" I meant "function that transfers data from one linear memory to another". It seems I've misused the terminology. As long as you have multiple address spaces you will need such a function, I believe, and as far as I know this isn't currently something that jco can generate, instead relying on the host Wasm engine to transfer data between modules. Correct me if I'm wrong, please.
Yes this isn't quite correct. JCO generates the adaptors to get around having to support multi memory. That is done in https://github.com/bytecodealliance/jco/blob/main/crates/js-component-bindgen/src/core.rs which is called from https://github.com/bytecodealliance/jco/blob/main/crates/js-component-bindgen/src/lib.rs#L123. Having a flag to switch it off might even be straightforward.
Since running a command is just calling a run() function, creating a run() function which wraps multiple run functions in a single component, where the individual components are imported sounds like a VERY SIMPLE tool to build to me. Perhaps that's just the OSS project you need here. A component-based tool to take multiple CLIs, and define the rules for creating a "super CLI" that calls the appropriate one. This is component composition 101, I don't think there's anything too tricky there?
Catherine (whitequark) (Jan 23 2024 at 22:12):creating a run() function which wraps multiple run functions in a single component, where the individual components are imported sounds like a VERY SIMPLE tool to build to me
No that's not what I'm talking about. I'm talking about one command being able to run another (from deep within its code), while being able to set their arguments and environment, provide data on stdin and capturing data from stdout/stderr streams. That's considerably more complex and even wrapping a command so that instead of an argument-less run() function you have a run(argv: list<string>, ...) function isn't something that today's wasm-tools compose can easily do. There's also no way to run the same instantiation of a command multiple times since the start function poisons itself when it's first called and can't be invoked again. This means that something like the gcc compiler driver calling cc1 several times isn't possible to do currently by composing components.
Guy Bedford (Jan 23 2024 at 22:29):Right, so on the simple CLI interception side, there are a few features to think about:
Remember that the CLI is just a run() function. If (1) is solved, it's a function that can be called multiple times, so then it's not some posix primitive just another exported function that can be called anytime. This makes things much much simpler is what I'm getting at.
For (2) the question is if you want that routing to be "dynamic" or "static". If its static and you always control the stream, WASI-Virt customization for this makes sense. But if it's dynamic, it might have to rather be part of the solution to (1) instead, properly parameterizing the execution.
(3) feels like it comes fairly naturally part with (1) and (2) solved?
Catherine (whitequark) (Jan 23 2024 at 23:43):Remember that the CLI is just a
run()function
I do know that. You've also just explained what I said above back to me, which I don't really appreciate. This simplicity is why I am interested in a CM or WASI-Virt based solution in the first place, compared to the solutions that are accessible to me now.
(1) is straightforward to solve regardless of whether you have a single core module within a component or several of them, it's just more work in the latter case, and it seems like parsing a component is pretty involved and the libraries aren't anywhere close to being mature yet.
(2) can be more difficult to solve depending on the interface chosen. My first pick was to have a "wrapper" component which imports run() from an inner one, exports streams to an inner one, and exports run() to an outer one, but this isn't actually possible to do with wasm-tools compose at the moment. Given that I've switched the strategy to exporting a number of functions like set-arguments and set-envrionment, which works but is less elegant. I think there is value in having a redirection-capable command's entry point still be just run(...), which would take as arguments the environment, redirected streams, and so on, because I use Wasm to turn complex UNIX commands like compilers into pure functions, and so it's natural to have them be a function.
(3) I am not sure what you mean by "super-call".
Guy Bedford (Jan 24 2024 at 01:21):Only excited about your use case here and wanting to help, it's been really great to follow your explorations of the model. Will be more careful with tone.
I suppose the streams virtualization exactly needs WASI-Virt's full streams replacements as there's no easy way to create a rerouting stream otherwise.
For the use case of turning commands into pure functions, that would be a great tool to have, and it comes up a lot in the context of WASI Virt. Definitely it involves quite a bit of binary work. It would be a great addition to the WASI-Virt project or as another OSS component project though to have this kind of tooling working for arbitrary CLIs for the component model.
For (3) I was meaning pretty much as in your model of creating a new CLI which internally can call pure component functions that correspond to the other CLIs, as it sounds like you are doing.
Always happy to discuss framing and collaboration if/where it helps, because there is a real ability for solutions that can work widely here and be maintained to everyones benefit in the component tooling space.
Catherine (whitequark) (Jan 24 2024 at 09:38):I suppose the streams virtualization exactly needs WASI-Virt's full streams replacements as there's no easy way to create a rerouting stream otherwise.
Is there not? I feel like maybe there could be, but I haven't sat down and tried to write one, and you've certainly thought about this far more than I did. It seems like poll would be the biggest problem, but there are some cases where it's possible to wrap. For example, if you can only redirect stdin from a buffer and stdout to a buffer, you do not have any concurrency with the caller, so stdin is only readable until exhausted and stdout is always writable, and then you could just delegate to an outer poll for filesystem/network resources. And if you want parts of the caller to execute concurrently with the callee and stdin/stdout to be connected to code in the caller, maybe that could be integrated with the wrapper poll somehow, though I can't think of how to make the interface not be onerous.
It would be a great addition to the WASI-Virt project or as another OSS component project though to have this kind of tooling working for arbitrary CLIs for the component model.
This is something I am motivated to work on in the little free time I have, so I'll probably try to chip at the edges of this task. Do you suppose adding reset would be a good start?
Ralph (Jan 24 2024 at 11:04):as an aside, @Catherine (whitequark), your pushing here is extremely fascinating. You're pushing me to ponder more questions -- a real gift for me. Like Guy, I'm excited about these areas.
Catherine (whitequark) (Jan 24 2024 at 12:37):Thanks! I think the next thing I'll release is a very raw prototype of something that packages a command into something that can be launched by another command, and then look into implementing the reset transformation. That feels foundational to most other capabilities.
Ralph (Jan 24 2024 at 12:43):I'd love to try! Let us know if you do this.....
Guy Bedford (Jan 24 2024 at 18:24):Is there not? I feel like maybe there could be, but I haven't sat down and tried to write one, and you've certainly thought about this far more than I did.
The problem here is creating custom virtual streams. As long as the stream is a non-virtualized host stream, it should be possible to intercept the stdio streams with _any host stream_. But it would still have to be a host-defined stream. To fully virtualize the stream gets into the WASI-Virt problem space, which is that once you virtualize any stream you need to virtualize every single interface that uses streams.
This is something I am motivated to work on in the little free time I have, so I'll probably try to chip at the edges of this task. Do you suppose adding reset would be a good start?
The reset transformation involves injecting a new function into the core module, while the pure component wrapping step involves parameterizing how the CLI should be called and configuring how to expose that as a function. I think either can be done first.
With regards to the virtualization, if you do want to virtualize the stdin and stdout and hit the problem of needing virtual streams, such that WASI-Virt functionality is needed, I am more than happy to take this as a WASI-Virt PR. Because this is one of the big use cases folks have for virt.
Of course, no pressure at all, and only if you are interested in collaborating on it. Sometimes it's nice just to solve ones own use cases / have ones own projects too. So whatever suits you best, we are all just exploring here together.
Guy Bedford (Jan 24 2024 at 18:43):Maybe the virt step can even be split into two: (1) creating any virtual streams if necessary which are applied with WASI Virt, and (2) separately creating a "wrapper" that composes in front of the command (as opposed to WASI virt which composes as imports behind the command). (1) likely still needs some WASI Virt changes maybe? and the wrapper for (2) is something that has come up for WASI virt before, for example for virtualizing incoming HTTP.
There are definitely a bunch of complexity to the composition parts here, one thing stating it all at a high level, and another doing it. There's lots of help in these channels to be found as needed, I could get a WASI Virt stream going on Zulip if it helps as well? I know @Joel Dice and others have looked into some of these problems before and might have feedback to share too.
Ralph (Jan 24 2024 at 21:25):wasi-virt is definitely an up and coming target area, for sure.
Catherine (whitequark) (Jan 24 2024 at 21:46):WASI-Virt stream on Zulip would be great
in terms of collaboration, I always prefer working with an upstream project if possible; I'm very happy to conform to the standards and culture of an existing project, especially if it's a BA one, and I do like having all of one's virtualization needs be solved in one place
Guy Bedford (Jan 24 2024 at 21:49):Ok I've set it up at https://bytecodealliance.zulipchat.com/#narrow/stream/422794-WASI-Virt/topic/CLI.20Virtualization and created a thread to continue this discussion there.
Catherine (whitequark) (Jan 24 2024 at 21:49):I think I maybe don't fully understand how the output of componentize is structured yet. My first impression/understanding was that the four core .wasm files that are created by jco transpile when I build a YoWASP project correspond to the original wasi preview1 core file, the preview1 adapter, and then two other shims or something? I think I'll need to look closer at the output of WASI-Virt to understand exactly what's going on there
Guy Bedford (Jan 24 2024 at 21:53):The other two small core wasm files are the multi-memory workaround trampolines / adaptors I always forget what we call them too. Created https://github.com/bytecodealliance/jco/issues/356
Katie Bell (Jan 25 2024 at 02:39):Chiming in here, to add a +1 to supporting --multi-memory. A friend and I were exploring running WASI preview 2 apps in the browser, but because browsers don't support components yet, we were trying to use use jco transpile naively expecting it'd give us a single core module (we didn't want to rely on the generated wrapper/trampoline code), but of course it has to split up all the components into separate modules as described above.
--multi-memory sounds like exactly what we need, and yes, I realise a single file won't cache well, and it's a bit weird to code against the WASI preview 2 API in flattened non-component form, but we're looking at this as a temporary solution to start working on a preview 2 implementation in the hope that browsers support the component model later on.
Last updated: Dec 23 2024 at 13:07 UTC