Stream: git-wasmtime

Topic: wasmtime / PR #14419 pooling allocator: let an embedder r...

Wasmtime GitHub notifications bot (Sep 25 2026 at 08:51):

HasanH47 opened PR #14419 from HasanH47:release-idle-pool-memory to bytecodealliance:main:

Closes #14413 .

linear_memory_keep_resident trades memory for page faults: after a slot is freed, up to that much of it is reset in place and left resident so the next instantiation does not fault it back in. That is the right trade while slots are reused every few milliseconds, and the wrong one for a slot nothing has touched in an hour - resident memory then follows the peak concurrency a process has ever seen rather than its current load. Running one instance per application with a fresh instance per request, we measure ~0.5 GiB of resident slots at c=64 that does not come back down after the burst, so a host sized for a daily peak carries that peak all night.

Because the setting is fixed when the Engine is built, an embedder could only choose "always keep" or "never keep" - and "never keep" costs about 2× the CPU per request on one vCPU. Engine::release_idle_pool_memory() is the third choice, driven by the embedder rather than by a timer inside Wasmtime: an embedder that already knows when it has gone idle calls it from that path. It returns the bytes released, and 0 when the engine is not using the pooling allocator.

Nothing unique is released. The resident region holds exactly what the mapping restores on its own - the image for a slot that has one, written back by the reset, and zeros for a slot that does not - so where decommit_behavior is RestoreOriginalMapping the decommit puts those same contents back at the cost of a fault. That is the trade the rest of the slot already makes.

Three things a reviewer may want to poke at:

• Order. Slots are taken out of the free lists before their memory is released and returned afterwards, so nothing can allocate one while it is being decommitted. That is the order DecommitQueue::flush already uses.
• Locking. Each shard is locked in turn rather than all at once, and the madvise happens outside the lock.
• Failure. A failed decommit leaves the pages resident with the contents they had, which is a correct state: those slots go back reporting the same bytes as before and nothing is lost but the reclaim.

Scope: linear memories only. Tables and stacks have their own *_keep_resident and are untouched - memories dominate the number (8 MiB against 64 KiB for tables by default) and a smaller change is easier to review. Happy to extend it if you would rather have all three at once.

Tests cover both halves - that the bytes are released and the slot stays allocatable, and that the next instantiation still reads what its module's data segment says it should. A separate unit test covers slot affinity surviving the round trip, which an earlier draft of this got wrong.

Wasmtime GitHub notifications bot (Sep 25 2026 at 08:51):

HasanH47 requested cfallin for a review on PR #14419.

Wasmtime GitHub notifications bot (Sep 25 2026 at 08:51):

HasanH47 requested wasmtime-core-reviewers for a review on PR #14419.

Wasmtime GitHub notifications bot (Sep 25 2026 at 08:59):

HasanH47 commented on PR #14419:

Here are the before/after numbers I promised in the issue, from a standalone program rather than our runtime so you can re-run it: 64 pooling slots, linear_memory_keep_resident 8 MiB, each slot warmed by instantiating a module and dirtying 4 MiB of its memory, then all of them dropped. Linux 6.18, release build.

after 5 s idle	no call	release_idle_pool_memory()
unused_memory_bytes_resident	256.0 MiB	0.0 MiB
process RSS	266.9 MiB	10.9 MiB
next instantiate + 4 MiB touch	0.18 ms	2.23 ms
the one after that	0.02 ms	0.02 ms

So it gives back what it says it gives back - the RSS drop (256.1 MiB) matches the bytes the call reports (256.0 MiB) - and the control shows the memory does not come back on its own.

The last two rows are the other half of the trade, since it seemed worth measuring rather than asserting: the first instantiation after the quiet period pays ~2 ms re-faulting its pages, and the slot is warm again immediately after. That is the cost an embedder is accepting when it decides it is idle.

The program is ~80 lines and I'm happy to post it, or to re-run any of this on a different shape if you would rather have decommit_idle_slots_after(Duration) instead.

Wasmtime GitHub notifications bot (Sep 25 2026 at 09:03):

HasanH47 updated PR #14419.

Wasmtime GitHub notifications bot (Sep 25 2026 at 10:46):

github-actions[bot] added the label wasmtime:api on PR #14419.

Wasmtime GitHub notifications bot (Sep 26 2026 at 00:17):

:thumbs_up: cfallin submitted PR review:

Thanks! This looks reasonable to me. My only review comments are on the comments actually -- trying to hone things to be a little clearer and less verbose. Happy to merge once that's adjusted.

Wasmtime GitHub notifications bot (Sep 26 2026 at 00:17):

:speech_balloon: cfallin created PR review comment:

I think this doc-comment stole the doc-comment of the method below (unused_bytes_resident); perhaps a malformed diff? Can you insert the new method and its doc-comment before it (after line 581)?

Wasmtime GitHub notifications bot (Sep 26 2026 at 00:17):

:speech_balloon: cfallin created PR review comment:

Let's avoid the em-dashes in the doc comment here (soft preference for plain ASCII for compat and to avoid obvious LLM taint).

Also this is just a little too verbose and flowery for my taste at least. Maybe rewrite it to something like:

Releases memory the pooling allocator is keeping resident for slots that are not currently in use, returning how many bytes were released.

`PoolingAllocationConfig::linear_memory_keep_resident` keeps memory resident after an instance slot is freed, in order to enable faster instantiation on the next use of the same module. This increases resident memory size. If an embedder knows that load has decreased and wishes to free up memory, it might wish to purge these "warm slots".

This method releases all resident memory held by warm but unused slots. The tradeoff is that the next instantiation of any given module may be slower, because no cached memory mappings are present anymore.

[example]

Wasmtime GitHub notifications bot (Sep 26 2026 at 00:17):

:speech_balloon: cfallin created PR review comment:

As above, no need for the overly verbose LLM comments here. "trade is fixed when the Engine is built ..." is something that we can write once in a config option or method, not something we attach to every single piece.

"The second half of this test is the one that matters" is a fairly annoying LLM-ism too. Just say "This test verifies both that memory is freed, and that instantiation still works correctly afterward."

Wasmtime GitHub notifications bot (Sep 26 2026 at 00:17):

:speech_balloon: cfallin created PR review comment:

This doc-comment is a little verbose; no need to justify the whole feature, and the "Nothing unique is lost" comment is kind of inscrutable. I would rewrite it more like:

Releases the memory this pool is keeping resident for unused-but-warm slots.

Useful when the embedder knows that load has decreased and wishes to reduce resident memory usage at the cost of some instantiation time at the next instantiation.

Returns the number of bytes freed.

Wasmtime GitHub notifications bot (Sep 26 2026 at 00:45):

HasanH47 updated PR #14419.

Wasmtime GitHub notifications bot (Sep 26 2026 at 00:50):

:memo: HasanH47 submitted PR review.

Wasmtime GitHub notifications bot (Sep 26 2026 at 00:50):

:speech_balloon: HasanH47 created PR review comment:

You're right, and it was a bad insertion point on my side rather than a diff artifact. unused_bytes_resident has its doc-comment back, and the new method now sits above it with its own.

Wasmtime GitHub notifications bot (Sep 26 2026 at 00:54):

:memo: HasanH47 submitted PR review.

Wasmtime GitHub notifications bot (Sep 26 2026 at 00:54):

:speech_balloon: HasanH47 created PR review comment:

Taken, thanks I used your wording. The one thing I kept is a short paragraph at the end about slots being taken out of the free lists before the decommit and returned afterwards, since that is the invariant the unsafe block below relies on. Happy to drop it if you would rather that lived only at the call site.

Wasmtime GitHub notifications bot (Sep 26 2026 at 00:55):

:memo: HasanH47 submitted PR review.

Wasmtime GitHub notifications bot (Sep 26 2026 at 00:55):

:speech_balloon: HasanH47 created PR review comment:

Done, the test doc is your sentence now, and I trimmed the comments in the body to what they need to say. I also renamed the test data from our project's name to init/dirt that had no business being in an upstream test.

Wasmtime GitHub notifications bot (Sep 26 2026 at 00:55):

:thumbs_up: cfallin submitted PR review.

Wasmtime GitHub notifications bot (Sep 26 2026 at 00:55):

cfallin has enabled auto merge for PR #14419.

Wasmtime GitHub notifications bot (Sep 26 2026 at 00:56):

:speech_balloon: HasanH47 created PR review comment:

Rewritten to your version, with an intra doc link to PoolingAllocationConfig::linear_memory_keep_resident and the example kept. The whole diff is plain ASCII now I cleaned up the em-dashes in the inline comments too, not just this one.

Wasmtime GitHub notifications bot (Sep 26 2026 at 00:56):

:memo: HasanH47 submitted PR review.

Wasmtime GitHub notifications bot (Sep 26 2026 at 00:57):

cfallin added PR #14419 pooling allocator: let an embedder release memory kept resident for idle slots to the merge queue.

Wasmtime GitHub notifications bot (Sep 26 2026 at 01:24):

:check: cfallin merged PR #14419.

Wasmtime GitHub notifications bot (Sep 26 2026 at 01:24):

cfallin removed PR #14419 pooling allocator: let an embedder release memory kept resident for idle slots from the merge queue.

Last updated: Oct 11 2026 at 04:10 UTC