Stream: wasmtime

Topic: ✔ State of memory management in Wasmtime

Tyler Rockwood (Oct 12 2023 at 16:07):

A hopefully quick question. My understanding is that Wasmtime uses mmap in 3 places:

1. During compilation for all of the generated code(which is marked executable) and other bits (like data segments to load into LinearMemory)
2. Linear Memory (memory provided to Wasm modules via memory.grow, load and store instructions and friends).
3. Async stacks (The seperate stack WebAssembly executes on) are also mmap.

There are currently configuration options to use custom allocators from embedders for 2 and 3.

Does that sound correct? Are there other locations where mmap is used? Thanks in advance!

fitzgen (he/him) (Oct 12 2023 at 16:14):

two other cases off the top of my head:

• wasmtime will call malloc for various things which will call mmap
• depending on configuration, mmap can be used for cow memory images

Tyler Rockwood (Oct 12 2023 at 16:22):

wasmtime will call malloc for various things which will call mmap

Is this different than usage of Rust allocator?

fitzgen (he/him) (Oct 12 2023 at 16:22):

nope

Tyler Rockwood (Oct 12 2023 at 16:22):

Cool - thanks! That was all I wanted to double check. Appreciate the rapid response :thumbs_up:

fitzgen (he/him) (Oct 12 2023 at 16:23):

https://docs.rs/wasmtime/latest/wasmtime/struct.Config.html#method.memory_init_cow

fitzgen (he/him) (Oct 12 2023 at 16:23):

^ info about the cow memory images

Tyler Rockwood (Oct 12 2023 at 16:24):

Thanks I was just going to look that up.

Tyler Rockwood (Oct 12 2023 at 16:25):

Do you have a code pointer for where this happens (no worries I can also go hunt it down)

Tyler Rockwood (Oct 12 2023 at 16:26):

https://github.com/bytecodealliance/wasmtime/blob/738d41eaf2e257f776dc5643b19e41ae0a3e2700/crates/runtime/src/cow.rs#L97

Tyler Rockwood (Oct 12 2023 at 16:26):

Is that it?

Tyler Rockwood (Oct 12 2023 at 16:27):

Does that still work if with_host_memory is being used?

Tyler Rockwood (Oct 12 2023 at 16:31):

Looks like not: https://github.com/bytecodealliance/wasmtime/blob/738d41eaf2e257f776dc5643b19e41ae0a3e2700/crates/wasmtime/src/trampoline/memory.rs#L127

fitzgen (he/him) (Oct 12 2023 at 16:33):

I haven't ever actually poked at the cow memory images implementation myself, so I'd just do exactly what you are doing to answer these questions, which is dig in and read the code

fitzgen (he/him) (Oct 12 2023 at 16:33):

so: good luck!

Tyler Rockwood (Oct 12 2023 at 16:34):

Seems like if a custom host memory is being used those cow images should be disabled. Anyways, thanks for the help! Looks like I should plumb another config option through the C-API

Notification Bot (Oct 12 2023 at 16:34):

Tyler Rockwood has marked this topic as resolved.

Joel Dice (Oct 12 2023 at 16:34):

I haven't dug into the low-level details, but my high level understanding is that when you instantiate a module or component from a .cwasm file, Wasmtime will try to COW the memory image in that file.

Alex Crichton (Oct 13 2023 at 05:35):

One thing perhaps worth noting is that whilen COW uses mmap it always does so with MMAP_FIXED. If your custom host memory is not page aligned then you should disable COW

Tyler Rockwood (Oct 16 2023 at 17:35):

Custom host memory doesn't support COW at all, so I wonder if we should have the config assert that cow is not on with custom host memory.

Ref: https://github.com/bytecodealliance/wasmtime/blob/738d41eaf2e257f776dc5643b19e41ae0a3e2700/crates/wasmtime/src/trampoline/memory.rs#L127

Alex Crichton (Oct 16 2023 at 17:37):

that's a good point yeah, that should probably assert that it's None and whenever a custom memory is set in Config it additionally validates that cow is disabled

Last updated: Nov 22 2024 at 17:03 UTC