Stream: git-wasmtime

Topic: wasmtime / issue #10740 Regression for custom LinearMemor...

Wasmtime GitHub notifications bot (May 07 2025 at 10:14):

V0ldek opened issue #10740:

Hello.

I've been using wasmtime with a custom memory implementation that is backed by my custom managed virtual mmap. I don't think the implementation details of my custom struct matter. I've been on wasmtime 25.0 and everything worked fine. After an update to wasmtime 32.0 I had to rewrite the wasmtime::LinearMemory impl, but it just got simplified. However, running my code now gets me this panic:

<details>
<summary>Stacktrace</summary>

thread 'main' panicked at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/vm/memory.rs:536:29:
internal error: entered unreachable code: memory_image is Some only for mmap-based memories
stack backtrace:
   0: __rustc::rust_begin_unwind
             at /rustc/a15cce2690e8fab72422515c9dc02c6fbc506733/library/std/src/panicking.rs:697:5
   1: core::panicking::panic_fmt
             at /rustc/a15cce2690e8fab72422515c9dc02c6fbc506733/library/core/src/panicking.rs:75:14
   2: wasmtime::runtime::vm::memory::LocalMemory::new
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/vm/memory.rs:536:29
   3: wasmtime::runtime::vm::memory::Memory::new_dynamic
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/vm/memory.rs:240:22
   4: <wasmtime::runtime::vm::instance::allocator::on_demand::OnDemandInstanceAllocator as wasmtime::runtime::vm::instance::allocator::InstanceAllocatorImpl>::allocate_memory
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/vm/instance/allocator/on_demand.rs:119:22
   5: wasmtime::runtime::vm::instance::allocator::InstanceAllocator::allocate_memories
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/vm/instance/allocator.rs:463:27
   6: wasmtime::runtime::vm::instance::allocator::InstanceAllocator::allocate_module::{{closure}}
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/vm/instance/allocator.rs:406:13
   7: wasmtime::runtime::vm::instance::allocator::InstanceAllocator::allocate_module
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/vm/instance/allocator.rs:405:15
   8: wasmtime::runtime::instance::Instance::new_raw
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/instance.rs:285:13
   9: wasmtime::runtime::instance::Instance::new_started_impl
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/instance.rs:207:33
  10: wasmtime::runtime::instance::Instance::new_started
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/instance.rs:195:9
  11: wasmtime::runtime::instance::InstancePre<T>::instantiate
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/instance.rs:904:18
  12: wasmtime::runtime::linker::Linker<T>::instantiate
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/linker.rs:1097:9
  13: anyblox::programs::wasm::WasmProgram::prepare::{{closure}}
             at ./anyblox/src/programs/wasm.rs:125:60
...

</details>

where the prepare function at the bottom is the one that instantiates a module with:

linker.instantiate(&mut store, &self.module)

I see how this gets caused under the hood, namely the LinearMemoryProxy always creates a Raw memory from a pointer and the LocalMemory::new function then enters the unreachable branch, but I don't understand if this is intentional or an overlook during the refactoring between versions 25 and 32.

I'd be interested on why this behaviour changed and if it's possible to restore the ability to use mem images with custom memory implementations. My custom linear memory is backed by an mmap as well, so there should not be any fundamental issues preventing it.

Wasmtime GitHub notifications bot (May 07 2025 at 14:45):

alexcrichton edited issue #10740:

Hello.

I've been using wasmtime with a custom memory implementation that is backed by my custom managed virtual mmap. I don't think the implementation details of my custom struct matter. I've been on wasmtime 25.0 and everything worked fine. After an update to wasmtime 32.0 I had to rewrite the wasmtime::LinearMemory impl, but it just got simplified. However, running my code now gets me this panic:

<details>
<summary>Stacktrace</summary>

thread 'main' panicked at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/vm/memory.rs:536:29:
internal error: entered unreachable code: memory_image is Some only for mmap-based memories
stack backtrace:
   0: __rustc::rust_begin_unwind
             at /rustc/a15cce2690e8fab72422515c9dc02c6fbc506733/library/std/src/panicking.rs:697:5
   1: core::panicking::panic_fmt
             at /rustc/a15cce2690e8fab72422515c9dc02c6fbc506733/library/core/src/panicking.rs:75:14
   2: wasmtime::runtime::vm::memory::LocalMemory::new
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/vm/memory.rs:536:29
   3: wasmtime::runtime::vm::memory::Memory::new_dynamic
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/vm/memory.rs:240:22
   4: <wasmtime::runtime::vm::instance::allocator::on_demand::OnDemandInstanceAllocator as wasmtime::runtime::vm::instance::allocator::InstanceAllocatorImpl>::allocate_memory
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/vm/instance/allocator/on_demand.rs:119:22
   5: wasmtime::runtime::vm::instance::allocator::InstanceAllocator::allocate_memories
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/vm/instance/allocator.rs:463:27
   6: wasmtime::runtime::vm::instance::allocator::InstanceAllocator::allocate_module::{{closure}}
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/vm/instance/allocator.rs:406:13
   7: wasmtime::runtime::vm::instance::allocator::InstanceAllocator::allocate_module
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/vm/instance/allocator.rs:405:15
   8: wasmtime::runtime::instance::Instance::new_raw
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/instance.rs:285:13
   9: wasmtime::runtime::instance::Instance::new_started_impl
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/instance.rs:207:33
  10: wasmtime::runtime::instance::Instance::new_started
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/instance.rs:195:9
  11: wasmtime::runtime::instance::InstancePre<T>::instantiate
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/instance.rs:904:18
  12: wasmtime::runtime::linker::Linker<T>::instantiate
             at /home/mat/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wasmtime-32.0.0/src/runtime/linker.rs:1097:9
  13: anyblox::programs::wasm::WasmProgram::prepare::{{closure}}
             at ./anyblox/src/programs/wasm.rs:125:60
...

</details>

where the prepare function at the bottom is the one that instantiates a module with:

linker.instantiate(&mut store, &self.module)

I see how this gets caused under the hood, namely the LinearMemoryProxy always creates a Raw memory from a pointer and the LocalMemory::new function then enters the unreachable branch, but I don't understand if this is intentional or an overlook during the refactoring between versions 25 and 32.

I'd be interested on why this behaviour changed and if it's possible to restore the ability to use mem images with custom memory implementations. My custom linear memory is backed by an mmap as well, so there should not be any fundamental issues preventing it.

Wasmtime GitHub notifications bot (May 07 2025 at 14:49):

alexcrichton commented on issue #10740:

Thanks for the report, and seems reasonable to fix! I got about halfway through removing/resolving the differences between LinearMemory and RuntimeLinearMemory but never finished the work, and I think that would probably help here. Agreed we should fix this though, and if you're interested to work on it I'd be happy to help review a PR.

Wasmtime GitHub notifications bot (May 08 2025 at 09:59):

V0ldek commented on issue #10740:

What would you see as a good fix here?

I'm not sure what the invariants for the code in vm/memory.rs are, but if it needs to know if the memory is mmapped or not then I was thinking maybe instead of LinearMemory::as_ptr(&self) -> *mut u8 there should be LinearMemory::as_base(&self) -> MemoryBase that explicitly says if this is just any pointer or an mmap?

Wasmtime GitHub notifications bot (May 08 2025 at 15:47):

alexcrichton commented on issue #10740:

Unfortunately I don't think there's any fix for this right now without changing the APIs themselves. You're right though in that the fix will be somewhere around here. Personally I'd like to remove the need for LinearMemoryProxy and changing the raw runtime internals to "just" using LinearMemory, updating the signatures as necessary to morally match RuntimeLinearMemory. I'm not certain we'll want MemoryBase as an externally-facing abstraction, though.

Wasmtime GitHub notifications bot (May 22 2025 at 15:46):

V0ldek commented on issue #10740:

I currently don't have cycles to tackle this, but I wanted to note one more thing that would be good for ergonomics of custom memory:

I'd like to be able to access the specific Box<dyn LinearMemory> out of an Instance after it's instantiated. Currently it doesn't seem to be possible even in host functions that get a Caller.

Wasmtime GitHub notifications bot (Sep 24 2026 at 09:42):

V0ldek commented on issue #10740:

Wanted to ask if there was any further developments here. I'm still locked to 25, which is now 23 versions behind, and that seems very unwise for a security-critical crate.

Every time I look at this I get confused by what exactly is going on with the LinearMemory implementations. This issue is important to me, but I don't know _what_ the correct design here is.

Wasmtime GitHub notifications bot (Sep 24 2026 at 09:44):

V0ldek edited a comment on issue #10740:

Wanted to ask if there was any further developments here. I'm still locked to 25, which is now 23 versions behind, and that seems very unwise for a security-critical crate.

Every time I look at this I get confused by what exactly is going on with the LinearMemory implementations. This issue is important to me, but I don't know _what_ the correct design here is.

What things does the internal runtime need from the memory that are not directly provided by the public trait?

Wasmtime GitHub notifications bot (Sep 25 2026 at 15:47):

alexcrichton commented on issue #10740:

The change here, what I think is your confusion, and part of my confusion, is I believe downstream of #9687. In that commit image management changed to hold an Arc<Mmap> internally within image slots which public external users of Wasmtime are unable to provide since that's an internal type. The PR says:

One option would be to sort-of-revert that change where MemoryImageSlot holds a raw pointer to the mmap that it's manipulating, and at that point I think it should be possible to delete RuntimeLinearMemory with some light refactoring, making LinearMemory the One True Abstraction, meaning that this issue should get resolved. I don't fully understand the consequences of such a change, however, because I'm not sure what the future work was lined up after #9687.

@sunshowers if you don't mind me pinging you on this, would you perhaps be able to shed some light on this? Specifically for you, your historical PR to Wasmtime, https://github.com/bytecodealliance/wasmtime/pull/9687, hinted at future work which built on the refactorings you were working on, but I'm not sure what that future work was. I'm not sure if that would become reliant on memory-image-mapping holding strong references to Mmap internals of where the image is mapped into which would constraint the API and design of possible solution for this issue. If it was mostly just shuffling things around thats where, to solve this issue, it might make sense to undo some of the refactorings there but I don't want to be too hasty about that.

Wasmtime GitHub notifications bot (Sep 26 2026 at 18:15):

V0ldek commented on issue #10740:

Why do the internals need to know if the memory backing is an mmap or not?

Wasmtime GitHub notifications bot (Sep 28 2026 at 20:49):

alexcrichton commented on issue #10740:

I'm not sure myself, but memory mappings are notoriously different across platforms, even Windows and Linux for example. I think this depends on the direction of future changes after #9687 which I'm not sure of myself. I'm not an expert on memory mappings in general to know if there are any platforms which require specific knowledge of how the backing memory for a file mapping is memory-mapped.

Last updated: Oct 11 2026 at 04:10 UTC