Stream: general

Topic: How to pass a pointer to array of structs from C -> WASM

Gavin Ray (Jul 26 2023 at 21:02):

I've been trying to google this for 2 days, and I am having a hard time figuring out a working example of how to do something like this:

// module.cpp
#include <cstdio>
extern "C" {
struct MyStruct {
    int a;
    int b;
};

void process_data(MyStruct* structs, int count) {
    for (int i = 0; i < count; ++i) {
        MyStruct current = structs[i];
        printf("a: %d, b: %d\n", current.a, current.b);
    }
}
}

// host.cpp
int main() {
  MyStruct data[] = { { 1, 2 }, { 3, 4 }, { 5, 6 } };
  // Initialize WASM/WASI environment, load "module" file
  auto module = wasmtime::Module(...);
  auto process_data_fn = std::get<wasmtime::Func>(*instance.get(store, "process_data"));
  process_data_fn.call(store, /* don't think I can just pass a pointer to host memory here */);
}

From what I've read, I _think_ I can maybe used "Shared Memory" to copy the structs and pass a pointer to the memory?
Or maybe I can export malloc from the WASM module and call that from host to get a pointer to a buffer, which I can then copy into?

I've also read something about using __heap_base somehow, but I didn't really follow this.

Any advice would be greatly appreciated, thank you :pray:

fitzgen (he/him) (Jul 26 2023 at 21:08):

Wasm cannot access arbitrary host memory by design. You will need to malloc space in the guest (which means the guest module needs to export a malloc function and your host code needs to know the name it is exported under) and then copy the data from the host to the guest.

I suggest avoiding int and other types that can have different sizes on different architectures for this sort of thing, since the host is 64-bit and the guest Wasm is 32-bit.

Even better, use the component model and wit-bindgen rather than rolling your own custom conventions here.

Gavin Ray (Jul 26 2023 at 21:09):

@fitzgen (he/him) Thank you for the reply!

Is there any way to share a memory space between host code and a WASM module?

Also, I had not heard of wit-bindgen or the component model before, I'll check these out :pray:

fitzgen (he/him) (Jul 26 2023 at 21:16):

Gavin Ray said:

Is there any way to share a memory space between host code and a WASM module?

The Wasm's linear memory is in the host process's address space, and you can do things like implement a custom memory creator that gives out regions of memory you want to share with the guest: https://docs.rs/wasmtime/latest/wasmtime/trait.MemoryCreator.html

But the guest and host ultimately don't really have knowledge of each other's ABI details (like where static globals are placed in memory, what calling conventions to use, how struct fields are arranged, what malloc/free to use, etc) and this is ultimately going to make trying to share memory together painful. Not impossible, but both sides will have to be written specifically to interact well with each other, and you'll essentially be inventing your own custom ABI details for all this stuff.

I recommend going with the component model, which standardizes these things for you. It is shared-nothing, in that different Wasm modules and the host are all sandboxed from each other and no one exports their memory or exposes internal details, so you can't share views of memory that update as either side makes changes. But you get nice security properties and a lot of tooling that makes passing complex data around easy.

Gavin Ray (Jul 26 2023 at 21:59):

@fitzgen (he/him) I managed to get it to work like this:
https://gist.github.com/GavinRay97/8e83758cf8c1c9d886df154ee2a6b7d7

One thing I don't understand though, is that if I remove the functions _malloc and _free from module.cpp, then it ceases to work because it can't find the function malloc.

Is this some sort of automagic thing with underscored names?

Gavin Ray (Jul 26 2023 at 22:07):

Ah I think it may actually be because malloc and free aren't exported unless used somewhere

Last updated: Nov 22 2024 at 16:03 UTC