Stream: git-wasmtime

Topic: wasmtime / issue #7848 Memory grow fill error

Wasmtime GitHub notifications bot (Jan 30 2024 at 22:24):

ZentsuGo opened issue #7848:

Hello,

I'm trying to grow the memory of a Wasm module with memory.grow(...) method where the memory is from the exports of the instance, writing on it and reading at the end regions seems to be working on the host program. However, when I try to read an end region from the Wasm module I seem to be getting this error:

error while executing at wasm backtrace: 0: 0x41fc4 - <unknown>!std::io::stdio::print_to_buffer_if_capture_used::h349c4cb436a0d078 1: 0x421e9 - <unknown>!std::io::stdio::_print::hed99a81f9f8d046f

It should be noted that such behavior doesn't happen when I write a content (anywhere in the grown memory) that is less than the initial number of pages (which is 17 by default). So in order to trigger this kind of error I would fill a chunk of memory whether at the beginning or the end regions.
As such, I think that I may probably have problems with reading inside the Wasm module or simply there is an error in the memory linking.

I am accessing memory parts within the Wasm module this way:

let memory = unsafe { extern "C" { static memory: [u8; 1246298112]; } &memory as *const _ as *const u8 };

where I tried to put an large size for this memory.
Apologies if this is no standard way to interact with memory, that is the only working thing for me so far.

Wasmtime GitHub notifications bot (Jan 31 2024 at 02:23):

alexcrichton commented on issue #7848:

Thanks for the report! Are you able to reproduce this issue without using unsafe? That may be the cause of what's happening here.

Otherwise though do you have a way to share to reproduce this? In lieu of that it's not clear what the unsafe code doing and the unsafe code could be causing the issue here.

Wasmtime GitHub notifications bot (Jan 31 2024 at 13:35):

ZentsuGo edited issue #7848:

Hello,

I'm trying to grow the memory of a Wasm module with memory.grow(...) method where the memory is from the exports of the instance, writing on it and reading at the end regions seems to be working on the host program. However, when I try to read an end region from the Wasm module I seem to be getting this error:

error while executing at wasm backtrace: 0: 0x41fc4 - <unknown>!std::io::stdio::print_to_buffer_if_capture_used::h349c4cb436a0d078 1: 0x421e9 - <unknown>!std::io::stdio::_print::hed99a81f9f8d046f

It should be noted that such behavior doesn't happen when I write a content (anywhere in the grown memory) that is less than the initial number of pages (which is 17 by default). So in order to trigger this kind of error I would fill a chunk of memory whether at the beginning or the end regions.
As such, I think that I may probably have problems with reading inside the Wasm module or simply there is an error in the memory linking.

I am accessing memory parts within the Wasm module this way:

let memory = unsafe { extern "C" { static memory: [u8; 1246298112]; } &memory as *const _ as *const u8 };

where I tried to put a large size for this memory.
Apologies if this is no standard way to interact with memory, that is the only working thing for me so far.

Wasmtime GitHub notifications bot (Jan 31 2024 at 14:02):

ZentsuGo commented on issue #7848:

Thanks for the answer.

The unsafe block defines the memory access inside the Wasm module, the use of extern static requires an unsafe block unfortunately.

I am working in the context of WASI-Wasmtime, here is some code to reproduce this.
The main method code of the host program:

  let engine = Engine::default();
  let mut linker = Linker::new(&engine);
  wasmtime_wasi::add_to_linker(&mut linker, |s| s)?;

  let wasi = WasiCtxBuilder::new()
      .inherit_stdio()
      .inherit_args()?.preopened_dir(Dir::from_std_file(std::fs::File::open(path)?), path)?
      .build();

  let mut store = Store::new(&engine, wasi);

  // Instantiate our module with the imports we've created, and run it.
  let module = Module::from_file(&engine, "example.wasm")?;

  let memory_type = MemoryType::new(1, Some(20000));
  let memory = Memory::new(&mut store, memory_type).unwrap();

  // Define the memory in the linker
  linker.define(
      &mut store,
      "",
      "memory",
      memory,
  )?;

  let instance = linker
      .instantiate(&mut store, &module).unwrap();

  // Get the memory, to make sure it works as expected
  let memory = instance
      .get_memory(&mut store, "memory").ok_or(0).unwrap();

  // We grow the memory.
  let previous_nb_pages = memory.grow(&mut store, 19000).unwrap();
  println!("Previous number of pages: {}", previous_nb_pages);


  let mem_size = memory_data.len();
  // Ensure enough space in memory
  if mem_size < 14500000 {
      println!("Not enough space in memory to write string.");
  } else {
      // Fill a chunk of memory.
      for i in 0..14200000 {
          memory.write(&mut store, i, "a".as_bytes()).unwrap();
      }

      println!("String written to memory.", string_to_write);
  }

  match instance
  .get_export(&mut store, "print_string")
  .and_then(|e| e.into_func())
  .ok_or(0) {
      Ok(read_string) => {
          match read_string.call(&mut store, &[Val::I32(14100000 as i32), Val::I32(150 as i32)], &mut []) {
              Ok(()) => {
                  println!("String read.");
              },
              Err(e) => {
                  println!("Error while reading: {}", e);
              }
          }
      },
      Err(e) => {
          println!("Error while looking for the function: {}", e);
      }
  }

Here I simply filled with lots of a the memory, but I could have filled more than 17 pages of content in the Wasm module anywhere in the memory.
I am guessing the define in the linker with the memory doesn't work as expected, I tried to define the memory with another name and the get_memory didn't find the value for this key name.

Here is the code of the wasm module:

pub fn read_string_from_memory(offset: usize, len: usize) -> String {
    let memory = unsafe {
        // Access the memory directly using the global memory object
        extern "C" {
            static memory: [u8; 1246298112];
        }
        &memory as *const _ as *const u8
    };

    // Print memory ptr
    println!("Memory ptr: {:p}", memory);

    // Create a slice from the memory pointer
    let memory_slice = unsafe { std::slice::from_raw_parts(memory.offset(offset as isize), len+offset) };

    let buffer = memory_slice[..len].to_vec();

    String::from_utf8_lossy(&buffer).to_string()
}

#[no_mangle]
pub extern "C" fn print_string(offset: usize, len: usize) {
    let string = read_string_from_memory(offset, len);
    println!("String from memory: {}", string);
}

If there is another way to get the memory through the Wasm module without using unsafe, I'd be happy to have it though.

Wasmtime GitHub notifications bot (Jan 31 2024 at 14:36):

bjorn3 commented on issue #7848:

It looks like you are overwriting the entire linear memory from the host. This would overwrite both the stack of the wasm module (harmless while there is no wasm function active, but calling any wasm function will overwrite your stored data again) and any data stored by the wasm module (this is likely to lead to corruption for the wasm module, causing it to crash.) I did recommend having the wasm module itself allocate memory using it's memory allocator and then pass the address of this memory chunk to the host.

Wasmtime GitHub notifications bot (Jan 31 2024 at 15:49):

ZentsuGo commented on issue #7848:

Thanks @bjorn3 for your answer.
In this case it doesn't seem like memory.grow actually grows the memory's module or that it grows a separate memory, since I am writing only to a portion of the memory.

Indeed I did not seem to be able to find clear instructions on the memory allocation of the wasm module, would you mind to redirect me to the appropriate documentation please?

Wasmtime GitHub notifications bot (Jan 31 2024 at 16:24):

bjorn3 commented on issue #7848:

Indeed I did not seem to be able to find clear instructions on the memory allocation of the wasm module, would you mind to redirect me to the appropriate documentation please?

You can export a function from your wasm module which calls malloc or whichever allocation function is used by the language you use. And then from the host you can call this function whenever you need to get some memory chunk to write to.

In this case it doesn't seem like memory.grow actually grows the memory's module or that it grows a separate memory, since I am writing only to a portion of the memory.

If it didn't grow the memory, the crash would happen right when writing. From what I can tell based on the backtrace snippet you gave, the crash happens much latter somewhere inside of the printing machinery of libstd, but not at a place where the value you are trying to print would be accessed. This indicates to me that there is memory corruption of some sort within the wasm module.

Wasmtime GitHub notifications bot (Jan 31 2024 at 17:04):

ZentsuGo commented on issue #7848:

Now it seems to be working, considering the returned pointer refers to an offset to the memory pointer retrieved from the instance in the host program.
It seems like the allocating function is enough, since the memory of the module seems to adapt its size upon the call of the function. At least it seems to be working without the use of memory.grow(...).

However, when deallocating the memory chunk, the memory size seems to remain constant. However memory has indeed been freed since I can no longer write on it and read back causing a similar malloc error.

Thank you both!

Wasmtime GitHub notifications bot (Jan 31 2024 at 17:05):

ZentsuGo closed issue #7848:

Hello,

I'm trying to grow the memory of a Wasm module with memory.grow(...) method where the memory is from the exports of the instance, writing on it and reading at the end regions seems to be working on the host program. However, when I try to read an end region from the Wasm module I seem to be getting this error:

error while executing at wasm backtrace: 0: 0x41fc4 - <unknown>!std::io::stdio::print_to_buffer_if_capture_used::h349c4cb436a0d078 1: 0x421e9 - <unknown>!std::io::stdio::_print::hed99a81f9f8d046f

It should be noted that such behavior doesn't happen when I write a content (anywhere in the grown memory) that is less than the initial number of pages (which is 17 by default). So in order to trigger this kind of error I would fill a chunk of memory whether at the beginning or the end regions.
As such, I think that I may probably have problems with reading inside the Wasm module or simply there is an error in the memory linking.

I am accessing memory parts within the Wasm module this way:

let memory = unsafe { extern "C" { static memory: [u8; 1246298112]; } &memory as *const _ as *const u8 };

where I tried to put a large size for this memory.
Apologies if this is no standard way to interact with memory, that is the only working thing for me so far.

Wasmtime GitHub notifications bot (Jan 31 2024 at 18:15):

ZentsuGo edited a comment on issue #7848:

Now it seems to be working, considering the returned pointer refers to an offset to the memory pointer retrieved from the instance in the host program.
It seems like the allocating function is enough, since the memory of the module seems to adapt its size upon the call of the function. At least it seems to be working without the use of memory.grow(...).

However, when deallocating the memory chunk, the memory size seems to remain constant. But memory has indeed been freed since I can no longer write on it and read back causing a similar malloc error.

Thank you both!

Wasmtime GitHub notifications bot (Jan 31 2024 at 18:22):

bjorn3 commented on issue #7848:

However, when deallocating the memory chunk, the memory size seems to remain constant.

Correct. Wasm doesn't have a way to shrink the linear memory. Only to grow it. The memory allocator will still internally mark it as free and reuse it for a future memory allocation, but it doesn't have a way to tell the wasm runtime that it doesn't a chunk of memory anymore.

Last updated: Nov 22 2024 at 16:03 UTC