Stream: general

Topic: ✔ How do wasip1 adapters allocate memory?

Slava Kuzmich (Jul 03 2024 at 15:25):

Hi! How do wasip1 adapters allocate memory via cabi_realloc and memory.grow?

Context:
I'm trying to adapt wasip1 Kotlin libraries to wasip2, using the

wasm-tools component new --adapt wasi_snapshot_preview1.reactor.wasm

Kotlin uses WasmGC for its own memory. It allocates linear memory only for the scope of import/export calls, and immediately copies the data to/from GC memory and deallocates the linear memory when it goes out of scope. This stack-based allocator owns the whole single linear memory. It works fine for both wasip1 and C-M, but I'm having trouble making adapters between the two work.

I see that by default wasip1 adapters piggyback on cabi_realloc to allocate memory for itself, like this:

at kotlin.wasm.unsafe.cabi_realloc
at wit-component:adapter:wasi_snapshot_preview1.allocate_stack
at wit-component:adapter:wasi_snapshot_preview1.fd_write
at wit-component:shim.adapt-wasi_snapshot_preview1-fd_write
at kotlin.io.println

Current Kotlin's cabi_realloc expects to be called only from canonical ABI, and crashes.

Do adapters expect cabi_realloc to behave like a full libc realloc? Or is there a simpler contract?

I also see the --realloc-via-memory-grow, which seems to work. Does it use the grown page temporarily, or could the main core module corrupt the adapter state when allocating its own memory.

Joel Dice (Jul 03 2024 at 15:32):

Hi Slava. I helped implement this part of the adapter, so I'll share what I know:

• The adapter needs a full page (i.e. 64KB) of memory for its exclusive use for as long as the component is instantiated (i.e. it's not a temporary allocation)
• If the module being adapted exports cabi_realloc, it will use that by default (as you noted). Otherwise, it will use memory.grow to allocate a single page and assume the module will not try to use it. Modules built using a reasonably recent version of wasi-libc should work fine -- it's allocator knows to only use pages it has allocated itself.

Is it possible to either make Kotlin's cabi_realloc work when the adapter calls it or make it use only linear memory it allocated itself via memory.grow? The other option would be to use WASIp2 directly, in which case you won't need the adapter at all. That's the approach the TinyGo folks took.

Notification Bot (Jul 03 2024 at 16:01):

Slava Kuzmich has marked this topic as resolved.

Last updated: Nov 22 2024 at 17:03 UTC