Erik S. (Jul 15 2024 at 23:50): Erik S. (Jul 15 2024 at 23:50):I understand that the WASM specification doesn't allow for JIT. And that it might relate to requirements regarding correctness and integrity guarantees of the compiled native code. But couldn't those guarantees still be provided, with recompilation similar to Liftoff/TurboFan? Or would it put too much of a burden on WASM engines, if the specification required them to also to support JIT?
It seems to me that JIT is performance-critical for languages like Python or JavaScript.
And that, if the module cannot be allowed to rewrite its own code, there should be a provision to dynamically instantiate submodules, so that a JIT engine could generate components adhoc, i.e. for hot code.
But why isn't this reasonable?
Jacob Lifshay (Jul 16 2024 at 00:17):some wasm hosts are microcontrollers and the wasm code running on them was ahead-of-time compiled to native code, so they don't actually have any compiler or interpreter on them at all. imo having JIT as an optional extension is fine but it shouldn't be required
Ralph (Jul 16 2024 at 08:05):First, the determinism inside wasm is a feature for those who want to eliminate the ability to create arbitrary code inside the sandbox.
Second, javascript and python are not performance critical languages in any way -- even though _decades_ of amazing work has made them as fast as possible. They're really USEFUL languages! Making interpreted languages fast inside wasm is likely going to be better served by the hard work currently underway (I'm thinking of things like YouTube - Wasm Research Day 2024 – Chris Fallin, wevaling the wasms: AOT JS Compilation to WebAssembly) than by imagining wasm should be radically different than it is. However, that's just one position. Ymmv. None of those features preclude the possibility.
Erik S. (Jul 16 2024 at 10:26):Ralph said:
Second, javascript and python are not performance critical languages in any way
Sorry, I meant that dynamic languages, that must be interpreted if JIT is not available, will have terrible performance.
Yes, AOT would be preferable, but it's a heavy bias against dynamic programming.
The security concerns are valid. For example, a server program could be hacked somehow and made to modify its own memory, and then its behavior, to then exploit its imported functions.
But does this apply to submodules? I.e. the program could instantiate a new Wasm module with no imports, to significantly restrict dynamically compiled code?
Ralph (Jul 16 2024 at 11:02):terrible performance is not true, or rather, depends on your scenario and/or needs. If you need faster performance, you wouldn't be using wasm; if you really needed it, you wouldn't be using a dynamic language (though go runtime does very well). Here we're just about what "needs" and "terrible" and "performance" actually mean. But!
The point of import and export control across the native host is a real one, but of course we can also protect the host with other means. Hosts always must be secured -- obviously in any compute the initiating process must be trusted and protected. That goes without saying. The typical scenario for wasm is protection for the host and other modules and not from within the module -- the assumption is that this is something you produced, so you control it. And the imports and exports should be tightly created and strongly typed -- no "here's a byte[] for fun".
So to your question about submodules: in reality, you CAN do dynamically created code, if you're strongly motivated. :-) https://wingolog.org/archives/2022/08/18/just-in-time-code-generation-within-webassembly is one approach. :-)
Ralph (Jul 16 2024 at 11:04):in short, there's tons of work going into this, and in addition there ARE ways to handle it. The real question is whether it needs to be a feature of wasm itself. <shrug goes here />
Erik S. (Jul 16 2024 at 11:54):@Ralph Thanks Ralph for both links, very interesting.
"Most people I have talked to that work on WebAssembly implementations in JS engines believe that a JIT proposal will come some day" - Andy Wingo
Ralph (Jul 16 2024 at 12:30):Yup. Some day!
Chris Fallin (Jul 16 2024 at 15:39):Sorry, I meant that dynamic languages, that must be interpreted if JIT is not available, will have terrible performance.
Yes, AOT would be preferable, but it's a heavy bias against dynamic programming.
I'd encourage you to watch my talk that Ralph linked above! There are ways to do AOT compilation of dynamic languages.
One important key insight is that we can contain all the dynamism of types and resulting behavior dispatch in ICs / inline-caches, and we can compile those ahead-of-time, putting the runtime choice in just an indirect call, for example. This gets us to a "baseline compiler" (in SpiderMonkey terms) level.
In general I think there is a deeply embedded implicit assumption in a lot of folks' thinking here that we must have an online compiler (JIT) but IMHO we just need to think a bit harder about the phasing separation (compile time vs. run time) and rearchitect things a bit.
Chris Fallin (Jul 16 2024 at 15:48):@Erik S. One other thing I would mention: this phasing separation -- ahead-of-time compilation -- is useful and desirable in some scenarios. For example with "short-lived instances", where a Wasm module is instantiated to perform one function execution (HTTP request, custom database function, ...), we really don't have a chance to "warm up", even if we did have a JIT primitive. In such cases, we need the toolchain to decide ahead of time; and maybe we can have a slower feedback loop (PGO-style rebuild after observation/profiling). Note again in my talk above I discuss this and outline how once we do ICs with indirect calls for pure AOT, we can do PGO-guided inlining alone (and no language-specific code!) to get much of the way toward a type-optimizing compiler
Soni L. (Jul 17 2024 at 15:44):JIT support in wasm2c will come eventually
Erik S. (Jul 17 2024 at 19:45):Chris Fallin said:
Erik S. ahead-of-time compilation -- is useful and desirable in some scenarios. For example with "short-lived instances", where a Wasm module is instantiated to perform one function execution ..., we really don't have a chance to "warm up"
I agree that JIT is often associated with fast compilation and lazy full optimization of hot code (e.g. in browsers).
But couldn't the WASM primitive request a desired optimization level (e.g. equivalent to AOT/full optimization), and the host could implement bytecode/compilation caching (for short-lived repeated instances).
Obviously this could be used to DoS the compiler, but for FaaS you just charge for compile time then.
Chris Fallin (Jul 17 2024 at 19:58):Sure, you could cache the result for later -- but if you get the types during one run, and mostly benefit from them later, you're already halfway to a PGO-style AOT, no? In other words I think this "convergently evolves" toward more or less the same point I'm aiming at starting from AOT.
Ralph (Jul 17 2024 at 20:07):Also, at this point it's all "sure, we could". The larger point is that we do not need to; and while we might get indeterminism later, we now know it's a massive security weakness - despite it being darned useful.
Last updated: Nov 22 2024 at 16:03 UTC