Aaron Franke (May 17 2024 at 19:42): Aaron Franke (May 17 2024 at 19:42):Hello, I am curious if wasmtime would allow me to use WebAssembly instead of LLVM or another bytecode for a programming language I am working on. Is it possible to use wasmtime with 64-bit WebAssembly, so 64-bit memory addresses? https://github.com/WebAssembly/memory64
Alex Crichton (May 17 2024 at 20:28):Short answer: yes memory64 works, it's off-by-default but you can turn it on with this knob
Longer answer: you're probably going to have a tough time writing applications here. WASI for example does not work with memory64 at this time (with Wasmtime at least), so you'd have to "start from nothing" in terms of what Wasmtime provides out-of-the-box. If you were planning on implementing your own host runtime hooks and such then that won't be an issue, but if you were planning on using WASI that might be a bit of a speed bump.
It's also worth nothing that the C/Rust toolchains don't have precompiled versions of 64-bit wasm (at least not that I know of). Both should have basic support at least but it's not super widely used so you might run into speed bumps along the way
Aaron Franke (May 17 2024 at 21:44):Host runtime hooks as in providing an API for the language to call into the system, like interacting with files? I intend for this to be a language embedded into things, so it would have manual bindings.
Chris Fallin (May 19 2024 at 00:20):It's worth understanding the tradeoffs here too: 64-bit addressing implies manual bounds-checks as well (which are slow, 30-50% penalty), while the default configuration for wasm32 uses virtual-memory tricks (a reserved 4GiB section of address space in the host 64-bit process) to make Wasm memory accesses almost as fast as native. ("almost" because the address-mode selection is a little less rich: one has to add in the heap base as well.)
I've seen several times folks reach for wasm64 by analogy to native platforms, where the 32->64 bit upgrade was a clear win: 64-bit-wide registers and basic operations, and on some platforms more registers too (x86-64 upgraded 8->16 GPRs, ARM32 to aarch64 upgraded 16->32 GPRs). It's worth noting that wasm32 already has builtin 64-bit datatypes and 64-bit operators; and when compiled by a Wasm engine on a 64-bit host can make use of all the host registers; the only difference is the address width, hence the addressable range. In some cases wasm32 benchmarks can actually be slightly faster than native on 64-bit hosts because of less cache pressure / memory bandwidth: 32-bit addresses are more compact.
Finally it's worth noting that interactions with the host will require moving memory into/out of the Wasm heap anyway (i.e. passing host pointers directly is not really a thing) so the host address size shouldn't figure into the tradeoff.
So all that said: is the reason for wasm64 here that you expect the size of the Wasm guest heap to exceed 4GiB (or want to support that case)? Or some other reason?
Aaron Franke (May 19 2024 at 05:19):There is not a specific reason like a need for a specific application to use lots of memory. The problem is that I want this to be general-purpose and do not want to be artificially limited, to allow users to make any app with this language. Actually, I am attempting to rewrite an existing programming language which already supports 64-bit addresses, so it would be an unacceptable downgrade to use 32-bit addresses, I want this to serve as a drop-in replacement for the original language.
Jacob Lifshay (May 19 2024 at 08:38):I expect future processors will probably gain special instructions to support wasm bounds-checked load/store/call-indirect without the slow-down.
Aaron Franke (May 19 2024 at 09:55):Another idea, what about something like 48-bit? That might allow skipping the check while also being a higher limit than is required for all modern systems. It could be stored in a 64-bit integer for alignment and then bitwise-AND'd to 48 bits.
Jacob Lifshay (May 19 2024 at 10:18):well, x86_64 currently supports 57-bit addresses, and it seems very plausible for new supercomputers to need >=64-bit addresses in the next few decades. This is part of why RISC-V already defined a (not yet stable) 128-bit pointer size ISA.
Chris Fallin (May 19 2024 at 15:07):I see; I would only point out then that to avoid “artificial limits” it’s probably best to avoid wasm64, which will cause issues with runtimes (not yet supported everywhere), Wasm tooling (same), and interop with 99% of existing ecosystem. The pragmatic approach today if you’re looking for adoption is to use wasm32; the ecosystem hasn’t transitioned yet.
Anyway that’s all I can provide so all the best to you!
Ralph (May 20 2024 at 16:46):Chris, what if I really did want the >4GiB guest heap, and only really that?
Chris Fallin (May 20 2024 at 17:08):Then in that exact case, wasm64 is the answer!
David Lloyd (May 20 2024 at 17:32):I suspect that 64-bit addressing would be much more useful in conjunction with non-contiguous memory (i.e. free-form page mapping)
David Lloyd (May 20 2024 at 17:33):multi-memory with contiguous-only allocation is strongly reminiscent of 80286 protected mode
Last updated: Nov 22 2024 at 16:03 UTC