Providing an externally-allocated memory buffer in the C API · general

The C API for providing a memory import utilises the wasm engine's own internal memory construction:

wasm_memory_t* wasm_memory_new(wasm_store_t*, const wasm_memorytype_t*);

wasm_memory_t is an anonymous struct, and while wasm_memory_data() can retrieve the raw pointer for the underlying buffer, there doesn't appear to be any way to provide this buffer directly.

wasm_memory_t* wasm_memory_new_with_buffer(wasm_store_t*, void* buffer, uint32_t size, void (*free_fn)(uint32_t));

My use case is to provide a virtual-mem-mapped buffer that allows transparent memory sharing between module instances. I have a working proof of concept already but needed to hack the wamr implementation to get at the linear memory allocation. (Note that wamr does provide some custom allocator functionality, but there's no facility for detecting which of the allocations is specifically for the module's linear memory buffer.)

bjorn3 (Oct 05 2021 at 09:44):

Alex Crichton (Oct 05 2021 at 14:15):

@Michael Martin this isn't supported in the C API right now but it should be supported in the Rust bindings through the LinearMemory trait and friends, if you're interested though I think it'd be pretty reasonable to expose the functionality through the C API

Michael Martin (Oct 07 2021 at 23:00):

Thanks Alex. Do you know what the process for requesting changes to the C API is?

Alex Crichton (Oct 07 2021 at 23:19):

Currently we don't have a formal process, but it's typically fine to do in a PR for an extension like this, review will help take care of the fiddly bits with the API and such

Michael Martin (Oct 11 2021 at 23:31):

Thanks again Alex. I'll put a PR together and send it out soon. @bjorn3 noted - I'll include a grow arg in the PR.

Michael Martin (Nov 02 2021 at 04:10):

I have a work-in-progress implementation for this but I'm not sure how best to update the API itself. In the implementation I've just added another function for creating instances:

wasm_module_t *module = wasm_module_new(...);
wasm_linear_buffer_alloc_t lba = { ... };   // pointers to malloc, realloc and free functions
wasm_instance_t *instance = wasm_instance_new_with_lb_alloc(store, module, ..., &lba);

However, this doesn't seem to fit with the general philosophy of the API - I don't imagine we'd want new creation functions for every new feature added. Another option could be to set the allocator functions on the wasm_module_t object:

wasm_module_t *module = wasm_module_new(...);
wasm_linear_buffer_alloc_t lba = { ... };
wasm_module_set_linear_buffer_alloc(&module, &lba);
wasm_instance_t *instance = wasm_instance_new(store, module, ...);

I don't think this is a good approach either. The module object is an otherwise constant runtime representation of the wasm binary, whereas the new allocator functions should be instance-specific.

I also can't have a function for updating the instance object (e.g. wasm_instance_set_linear_buffer_alloc(&instance, &lba)) since the buffer allocation happens during the call to wasm_instance_new. Thoughts?

(I haven't made this change in the wasm-c-api repo yet because that uses V8 for building the examples and I don't want to go to the trouble of implementing the proposal in V8 until it has some traction.)

Comparing bytecodealliance:main...mykmartin:linear-buffer-alloc · bytecodealliance/wasm-micro-runtime

WebAssembly Micro Runtime (WAMR). Contribute to bytecodealliance/wasm-micro-runtime development by creating an account on GitHub.