Stream: git-wasmtime

Topic: wasmtime / issue #8511 > 1 GB Memory Allocation Issue

Wasmtime GitHub notifications bot (Apr 30 2024 at 22:36):

chanattan opened issue #8511:

Hello,

I have looked through the documentation and some previous issues of Wasmtime
so sorry in advance if this has already been answered. I can't get a Wasm module
to be able to allocate more memory than 1GB before reaching a memory error,
in this case thread '<unnamed>' panicked at library/alloc/src/raw_vec.rs:534:5: capacity overflow. I am trying to instantiate a Wasm module through a native program
with Wasmtime 18.0.0 with the following code:

`let engine = Engine::default();
let mut linker = Linker::new(&engine);

wasmtime_wasi::add_to_linker(&mut linker, |s| s)?;
let module = Module::from_file(&engine, wasm_prg)?;

let wasi = WasiCtxBuilder::new()
    .inherit_stdio()
    .inherit_args()?.preopened_dir(Dir::from_std_file(std::fs::File::open(path)?), path)?
    .build();

let mut store = Store::new(&engine, wasi);
let memory_type = MemoryType::new(1, Some(16384*4));
Memory::new(&mut store, memory_type).unwrap();`

let instance = linker
        .instantiate(&mut store, &module).unwrap();
 match instance.get_export(&mut store, "function")
        .and_then(|e| e.into_func())
        .ok_or(0) {
          Ok(func) => ...
 }`

Which instantiates a Wasm module and call a function that allocates a Vec of certain size > 1 GB this way:

let mut buf: Vec<u8> = Vec::with_capacity(2217296726 as usize);

Here is the full error upon execution:

thread '<unnamed>' panicked at library/alloc/src/raw_vec.rs:534:5:
capacity overflow
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Error: error while executing at wasm backtrace:
    0: 0x121cc - <unknown>!__rust_start_panic
    1: 0x11f54 - <unknown>!rust_panic
    2: 0x11e87 - <unknown>!std::panicking::rust_panic_with_hook::h01fa6d9286535f9c
    3: 0x1125c - <unknown>!std::panicking::begin_panic_handler::{{closure}}::h322b08995dd5f5cc
    4: 0x111bc - <unknown>!std::sys_common::backtrace::__rust_end_short_backtrace::hd7b53dba62f7f0aa
    5: 0x1183e - <unknown>!rust_begin_unwind
    6: 0x16cf6 - <unknown>!core::panicking::panic_fmt::h2a1cf3beec13dfc3
    7: 0x16697 - <unknown>!alloc::raw_vec::capacity_overflow::h71c1886b236d8c9d
    8: 0x1505 - <unknown>!benchmark_wasm_joins
note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information

Since Wasm should support 4GB of memory I thought it was only a question of option toggling but I don't see how to do it in the first code section. Hope it was clear enough.

Wasmtime GitHub notifications bot (Apr 30 2024 at 22:36):

chanattan edited issue #8511:

Hello,

I have looked through the documentation and some previous issues of Wasmtime
so sorry in advance if this has already been answered. I can't get a Wasm module
to be able to allocate more memory than 1GB before reaching a memory error,
in this case thread '<unnamed>' panicked at library/alloc/src/raw_vec.rs:534:5: capacity overflow. I am trying to instantiate a Wasm module through a native program
with Wasmtime 18.0.0 with the following code:

`let engine = Engine::default();
let mut linker = Linker::new(&engine);

wasmtime_wasi::add_to_linker(&mut linker, |s| s)?;
let module = Module::from_file(&engine, wasm_prg)?;

let wasi = WasiCtxBuilder::new()
    .inherit_stdio()
    .inherit_args()?.preopened_dir(Dir::from_std_file(std::fs::File::open(path)?), path)?
    .build();

let mut store = Store::new(&engine, wasi);
let memory_type = MemoryType::new(1, Some(16384*4));
Memory::new(&mut store, memory_type).unwrap();`

let instance = linker
        .instantiate(&mut store, &module).unwrap();
 match instance.get_export(&mut store, "function")
        .and_then(|e| e.into_func())
        .ok_or(0) {
          Ok(func) => ...
 }

Which instantiates a Wasm module and call a function that allocates a Vec of certain size > 1 GB this way:

let mut buf: Vec<u8> = Vec::with_capacity(2217296726 as usize);

Here is the full error upon execution:

thread '<unnamed>' panicked at library/alloc/src/raw_vec.rs:534:5:
capacity overflow
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Error: error while executing at wasm backtrace:
    0: 0x121cc - <unknown>!__rust_start_panic
    1: 0x11f54 - <unknown>!rust_panic
    2: 0x11e87 - <unknown>!std::panicking::rust_panic_with_hook::h01fa6d9286535f9c
    3: 0x1125c - <unknown>!std::panicking::begin_panic_handler::{{closure}}::h322b08995dd5f5cc
    4: 0x111bc - <unknown>!std::sys_common::backtrace::__rust_end_short_backtrace::hd7b53dba62f7f0aa
    5: 0x1183e - <unknown>!rust_begin_unwind
    6: 0x16cf6 - <unknown>!core::panicking::panic_fmt::h2a1cf3beec13dfc3
    7: 0x16697 - <unknown>!alloc::raw_vec::capacity_overflow::h71c1886b236d8c9d
    8: 0x1505 - <unknown>!benchmark_wasm_joins
note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information

Since Wasm should support 4GB of memory I thought it was only a question of option toggling but I don't see how to do it in the first code section. Hope it was clear enough.

Wasmtime GitHub notifications bot (Apr 30 2024 at 23:02):

cfallin closed issue #8511:

Hello,

I have looked through the documentation and some previous issues of Wasmtime
so sorry in advance if this has already been answered. I can't get a Wasm module
to be able to allocate more memory than 1GB before reaching a memory error,
in this case thread '<unnamed>' panicked at library/alloc/src/raw_vec.rs:534:5: capacity overflow. I am trying to instantiate a Wasm module through a native program
with Wasmtime 18.0.0 with the following code:

`let engine = Engine::default();
let mut linker = Linker::new(&engine);

wasmtime_wasi::add_to_linker(&mut linker, |s| s)?;
let module = Module::from_file(&engine, wasm_prg)?;

let wasi = WasiCtxBuilder::new()
    .inherit_stdio()
    .inherit_args()?.preopened_dir(Dir::from_std_file(std::fs::File::open(path)?), path)?
    .build();

let mut store = Store::new(&engine, wasi);
let memory_type = MemoryType::new(1, Some(16384*4));
Memory::new(&mut store, memory_type).unwrap();`

let instance = linker
        .instantiate(&mut store, &module).unwrap();
 match instance.get_export(&mut store, "function")
        .and_then(|e| e.into_func())
        .ok_or(0) {
          Ok(func) => ...
 }

Which instantiates a Wasm module and call a function that allocates a Vec of certain size > 1 GB this way:

let mut buf: Vec<u8> = Vec::with_capacity(2217296726 as usize);

Here is the full error upon execution:

thread '<unnamed>' panicked at library/alloc/src/raw_vec.rs:534:5:
capacity overflow
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Error: error while executing at wasm backtrace:
    0: 0x121cc - <unknown>!__rust_start_panic
    1: 0x11f54 - <unknown>!rust_panic
    2: 0x11e87 - <unknown>!std::panicking::rust_panic_with_hook::h01fa6d9286535f9c
    3: 0x1125c - <unknown>!std::panicking::begin_panic_handler::{{closure}}::h322b08995dd5f5cc
    4: 0x111bc - <unknown>!std::sys_common::backtrace::__rust_end_short_backtrace::hd7b53dba62f7f0aa
    5: 0x1183e - <unknown>!rust_begin_unwind
    6: 0x16cf6 - <unknown>!core::panicking::panic_fmt::h2a1cf3beec13dfc3
    7: 0x16697 - <unknown>!alloc::raw_vec::capacity_overflow::h71c1886b236d8c9d
    8: 0x1505 - <unknown>!benchmark_wasm_joins
note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information

Since Wasm should support 4GB of memory I thought it was only a question of option toggling but I don't see how to do it in the first code section. Hope it was clear enough.

Wasmtime GitHub notifications bot (Apr 30 2024 at 23:02):

cfallin commented on issue #8511:

It appears that this is a Rust standard library issue on the guest side (i.e., your Wasm module, not Wasmtime itself): this line of code in the Vec implementation checks that the allocation size is less than isize::MAX, and on a 32-bit platform such as Wasm, isize (signed pointer-sized integer) ranges up to +2GiB, not +4GiB. Your allocation request is just over 2GiB in length, hence the error.

I'm not sure why Vec is this way; you'd have to read the library source to find out more, probably; but as this is not a Wasmtime issue (we're executing the module correctly) I'll go ahead and close this issue.

Wasmtime GitHub notifications bot (May 01 2024 at 06:50):

bjorn3 commented on issue #8511:

LLVM considers it UB for allocations larger than isize::MAX to exist. Libstd panics here to avoid this UB.

Wasmtime GitHub notifications bot (May 01 2024 at 15:33):

chanattan commented on issue #8511:

Thank you for your answers.
I am still looking for an alternative to fix this issue.
The initial goal remains simple, allocate the maximum memory in a Wasm module in Rust, i.e., 4GB currently.
Do you know what is being used to achieve that?

Wasmtime GitHub notifications bot (May 01 2024 at 15:42):

bjorn3 commented on issue #8511:

You have to make multiple separate allocations to allocate more than 2GB total. There is no way to make a single contiguous allocation.

Wasmtime GitHub notifications bot (May 01 2024 at 15:45):

chanattan commented on issue #8511:

I was afraid this was the only solution.
Thank you nonetheless.

Wasmtime GitHub notifications bot (May 01 2024 at 15:52):

cfallin commented on issue #8511:

To add a bit of nuance: not possible with LLVM, apparently. However, @bjorn3, do you know if the UB applies to a slice constructed manually (std::slice::from_raw_parts)? If that's allowed to range up to usize::MAX, then potentially you could write your own manual allocator function that uses memory.grow to grab a large (4GiB - epsilon) chunk. dlmalloc on the guest side is specifically compatible with "discontiguous grow" (some other client also using memory.grow).

Wasmtime GitHub notifications bot (May 01 2024 at 15:53):

cfallin edited a comment on issue #8511:

To add a bit of nuance: not possible with LLVM wrt Rust's allocator, apparently. However, @bjorn3, do you know if the UB applies to a slice constructed manually (std::slice::from_raw_parts)? If that's allowed to range up to usize::MAX, then potentially you could write your own manual allocator function that uses memory.grow to grab a large (4GiB - epsilon) chunk. dlmalloc on the guest side is specifically compatible with "discontiguous grow" (some other client also using memory.grow).

Wasmtime GitHub notifications bot (May 01 2024 at 17:52):

bjorn3 commented on issue #8511:

However, @bjorn3, do you know if the UB applies to a slice constructed manually (std::slice::from_raw_parts)?

Yes that is still UB if you actually try to use it. LLVM assumes that any allocation is less than isize::MAX bytes. From the LLVM LangRef:

https://llvm.org/docs/LangRef.html#getelementptr-instruction

These rules are based on the assumption that no allocated object may cross the unsigned address space boundary, and no allocated object may be larger than half the pointer index type space.

and also the couple of rules above it for getelementptr inbounds (which is the instruction rustc uses for all slice indexing, pointer offsetting for field accesses and ptr::offset and so on (but not ptr::wrapping_offset)

Wasmtime GitHub notifications bot (May 01 2024 at 18:01):

cfallin commented on issue #8511:

That's too bad. @chanattan it appears this means that you'll be unable to use any LLVM-based toolchain for wasm32 (e.g. wasi-sdk, or emscripten, or Rust, or tinygo, or ...) to allocate a single object >2GiB. You could still write your own (i) allocator, as above, and (ii) load/store primitives, in raw WAT, and link that into your wasm module; raw Wasm should have no issues with a single e.g. 3.75GiB array in memory. But you'd need LTO-time inlining to make that fast (maybe wasm-opt with the right options, I'm not sure).

Wasmtime GitHub notifications bot (May 01 2024 at 19:47):

chanattan commented on issue #8511:

I will probably go for a custom allocation. I will eventually look at the rest later.
Thank you again for your precious pieces of information.

Last updated: Nov 22 2024 at 17:03 UTC