Chris Fallin (Mar 20 2024 at 15:03): Chris Fallin (Mar 20 2024 at 15:03):A question for various folks familiar with wasi-libc and/or expected memory.grow semantics (@Dan Gohman ?): let's say I want to bundle some data with a Wasm snapshot and stick some pointers into the Wasm heap so that when re-enlivened, the module can access that data. (A second memory is out because it has to be accessible to C.) Are the semantics of memory.grow and the expectations of dlmalloc such that I can effectively do an offline memory.grow, sticking the data at the end of the snapshot's memory (at its minimum size) and bumping its minimum size?
I do see configuration options in dlmalloc about whether to expect non-contiguous sbrk or not, I'm not sure if this is just for optimization or required for correctness though...
(Let's consider this "opt-in" from the point of view of the module: it's asking for the changes via weval intrinsics, so visible semantics changes are fine)
fitzgen (he/him) (Mar 20 2024 at 15:04):if I understand what you are suggesting, whether this will work is really dependent on the particular malloc implementation and the linker
Dan Gohman (Mar 20 2024 at 15:05):Yes, the intention is, dlmalloc and memory.grow or "logical aot memory.grow" can interleave arbitrarily
Chris Fallin (Mar 20 2024 at 15:05):@Dan Gohman that's perfect, thanks
Chris Fallin (Mar 20 2024 at 15:06):(@fitzgen (he/him) I think I'm fine assuming wasi-libc here, as we integrate with e.g. its startup symbol names with wizer anyway)
fitzgen (he/him) (Mar 20 2024 at 15:07):(I don't know the particulars here, just that some allocators will take advantage of lld's exported __heap_base and assume that they own everything after that up to memory.size and so any appended pages would fall into that region that they assume that they own)
Chris Fallin (Mar 20 2024 at 15:11):fair enough; the alternative is requiring the guest to pre-allocate a buffer large enough, but we don't know how big necessarily ahead-of-time (this is for a "pre-wevaled functions" lookup table to be inserted after wevaling); or I guess interpreting malloc offline (!); I think I'm comfortable leaning on Dan's statement about dlmalloc above and requiring wasi-libc for now though (as this is opt-in)
Alex Crichton (Mar 20 2024 at 15:13):@fitzgen (he/him) fwiw there's now __heap_end in LLD to help with this
Alex Crichton (Mar 20 2024 at 15:13):(and targeted at enabling exactly this use case)
fitzgen (he/him) (Mar 20 2024 at 15:13):TIL!
Dan Gohman (Mar 20 2024 at 15:17):Yes, and dlmalloc uses it
Last updated: Nov 22 2024 at 17:03 UTC