Stream: git-wasmtime

Topic: wasmtime / PR #14974 runtime: lazily initialize imported-...

Wasmtime GitHub notifications bot (Oct 23 2026 at 22:54):

leventov opened PR #14974 from convex-in-prod:upstream-lazy-imported-table-funcrefs to bytecodealliance:main:

Motivation

Active element segments targeting imported tables currently materialize every native function-reference record at instantiation. Large linked modules can initialize many such slots that the embedding never reads or calls.

Change

Retain eligible active-segment function indices in immutable module metadata and record bounded source/range bindings on the table. Existing lazy reads materialize references from the contributing instance when needed. Tagged explicit writes, including nulls, take precedence; later segments override earlier ranges. Unsupported segment forms and deferred-history overflow retain eager initialization.

Bounds checks, segment order and fuel charges stay at the original startup boundary. The Store retains contributing instances even when a later segment or start function traps. Pooled table replacement discards the deferred bindings. Metadata allocation failure declines deferral before changing table contents.

The new fallible internal initializer opts out of host-call hooks, matching the previous infallible ref_func sequence. Other builtin, host-function and component hooks retain their existing behavior. This requires a small internal flag through the trap wrapper; no public configuration option is added.

Serialization retains the segment metadata under the ordinary artifact compatibility contract. This does not add a downstream version-number scheme or a protected-memory API.

Validation

Built the extracted branch with Rust 1.97.1. All 16 tests from tests/all/table.rs passed, including the three new regressions, and the updated call-hook instantiation test passed. These exact test functions were compiled in a small external harness against this branch with Cranelift, async, the copying collector, pooling and call hooks enabled, avoiding unrelated CLI/WASI dependencies. Changed-file formatting and diff checks passed.

Regression cases cover overlapping segments and table copies, explicit null writes, source lifetime after failed startup, partial initialization before a bounds trap, serialized reload, equal initialization fuel, bounded-history fallback, pooled table reuse, and the absence of new host-hook transitions.

Savings depend on the number of function references that remain unused; this moves materialization to access rather than removing its cost for functions actually used.

Wasmtime GitHub notifications bot (Oct 23 2026 at 22:54):

leventov removed the review request for wasmtime-compiler-reviewers from PR #14974.

Wasmtime GitHub notifications bot (Oct 23 2026 at 22:54):

leventov requested cfallin for a review on PR #14974.

Wasmtime GitHub notifications bot (Oct 23 2026 at 22:54):

leventov requested wasmtime-compiler-reviewers for a review on PR #14974.

Wasmtime GitHub notifications bot (Oct 23 2026 at 22:54):

leventov requested wasmtime-core-reviewers for a review on PR #14974.

Wasmtime GitHub notifications bot (Oct 23 2026 at 23:06):

leventov edited PR #14974:

Motivation

Active element segments targeting imported tables currently materialize every native function-reference record at instantiation. Large linked modules can initialize many such slots that the embedding never reads or calls.

Change

Retain eligible active-segment function indices in immutable module metadata and record bounded source/range bindings on the table. Existing lazy reads materialize references from the contributing instance when needed. Tagged explicit writes, including nulls, take precedence; later segments override earlier ranges. Unsupported segment forms and deferred-history overflow retain eager initialization.

Bounds checks, segment order and fuel charges stay at the original startup boundary. The Store retains contributing instances even when a later segment or start function traps. Pooled table replacement discards the deferred bindings. Metadata allocation failure declines deferral before changing table contents.

The new fallible internal initializer opts out of host-call hooks, matching the previous infallible ref_func sequence. Other builtin, host-function and component hooks retain their existing behavior. This requires a small internal flag through the trap wrapper; no public configuration option is added.

Serialization retains the segment metadata under the ordinary artifact compatibility contract. This does not add a downstream version-number scheme or a protected-memory API.

Illustrative downstream memory measurement

A matched local population comparison using convex-wasm-compiler measured 0.962 MiB less marginal host resident memory per application instance. The linked Wasm application contained approximately 47,000 side-module function references. Both variants used the same Core Wasm, inputs and collection policy; each kept four mature instances alive while adding four more. Each variant completed 13,440 checked application calls and 264 held invocations successfully.

At the mature completion boundary:

Memory component, MiB per instance	Eager references	Deferred references
Marginal host anonymous resident backing	2.862	1.900
Guest anonymous backing	7.109	7.109

The host component fell by approximately 34%. Allocated host capacity was essentially unchanged: the saving came from avoiding resident backing for unused native function-reference records, rather than requesting smaller allocations. Guest memory matched at every compared boundary.

The host estimate is the increase in non-guest anonymous resident memory when growing from four to eight instances, divided by four. It can include continued changes in the original instances and shared allocator state; it is a marginal population measurement, not exact allocation ownership. Kernel deltas were noisy and are not claimed as a saving. These are historical downstream measurements of the implementation before extraction onto current upstream, not a fresh benchmark of this PR branch. They establish a useful memory reduction for this workload, not an isolated runtime speedup or the same saving for every module.

Validation

Built the extracted branch with Rust 1.97.1. All 16 tests from tests/all/table.rs passed, including the three new regressions, and the updated call-hook instantiation test passed. These exact test functions were compiled in a small external harness against this branch with Cranelift, async, the copying collector, pooling and call hooks enabled, avoiding unrelated CLI/WASI dependencies. Changed-file formatting and diff checks passed.

Regression cases cover overlapping segments and table copies, explicit null writes, source lifetime after failed startup, partial initialization before a bounds trap, serialized reload, equal initialization fuel, bounded-history fallback, pooled table reuse, and the absence of new host-hook transitions.

Savings depend on the number of function references that remain unused; this moves materialization to access rather than removing its cost for functions actually used.

Wasmtime GitHub notifications bot (Oct 23 2026 at 23:13):

leventov edited PR #14974:

Active element segments targeting imported tables eagerly materialize native function-reference records, even when those entries are never used. This extends Wasmtime's existing lazy function-table initialization to eligible imported-table segments.

Store immutable function indices in shared module metadata and source-instance/range bindings on the table. The existing lazy-read path materializes a reference from its contributing instance on first access. Later segments override earlier ranges, and explicit table writes, including nulls, override deferred entries.

Per-table bindings are capped at 128 to bound metadata and first-access lookup costs. Unsupported segments, a full binding list or metadata allocation failure use eager initialization. Bounds checks, segment ordering and fuel charges remain at their existing initialization boundaries. The Store retains source instances after failed startup, and pooled table reuse clears deferred bindings.

The new fallible initializer suppresses host-call hooks to preserve the previous ref_func behavior; other calls keep their existing hooks.

Memory benefit

A local population comparison in the downstream convex-wasm-compiler integration, with approximately 47,000 side-module function references, saved 0.962 MiB of host resident memory per application instance:

MiB per instance	Eager	Deferred
Marginal host anonymous resident memory	2.862	1.900
Guest anonymous memory	7.109	7.109

Both variants used identical Core Wasm, inputs and collection policy. Host memory was estimated from the increase when adding four instances to four already warmed instances. This is a population estimate, including allocator effects. Allocated capacity was essentially unchanged; the saving comes from leaving unused function-reference pages untouched. Savings depend on how many references the workload uses.

Validation

All 16 tests in tests/all/table.rs and the call-hook instantiation regression passed in a focused harness against this branch with copying GC and pooling enabled. Coverage includes overlapping segments and table copies, explicit null writes, source lifetime after failed startup, partial initialization before a bounds trap, serialized reload, unchanged fuel charges, bounded-history fallback and pooled table reuse.

Wasmtime GitHub notifications bot (Oct 24 2026 at 01:53):

github-actions[bot] added the label wasmtime:api on PR #14974.

Wasmtime GitHub notifications bot (Oct 26 2026 at 14:50):

alexcrichton commented on PR #14974:

This is a pretty intense PR that's quite invasive to runtime internals in pretty subtle ways. In the end this is still bottoming out in an O(N) initialization per-table-segment, it's just with memset instead of libcalls. In that sense this is changing the constant factor overhead (in albeit a large way) but not changing the algorithmic complexity at all. I suspect the memory savings per-instance are probably the VMContext regions to store initialized VMFuncRefs, but I'm not certain.

Is the goal here speed or memory savings? If speed I'd feel better leaning on https://github.com/bytecodealliance/wasmtime/pull/14683 which should make ref.func significantly faster than it is today. If memory savings, however, that PR won't help and this'd still want to be considered.

Wasmtime GitHub notifications bot (Oct 26 2026 at 18:44):

leventov commented on PR #14974:

The goal was memory savings.

Please forgive me an LLM reply (edited):

The memset and the saved memory concern different storage. The PR still clears every pointer-sized table slot. It avoids constructing the separate 32-byte VMFuncRef records inside VMContext until needed. Those record pages are not already initialized elsewhere. Wasmtime reserves their storage but deliberately does not zero or initialize the function-reference region during instance creation. Consequently, skipping record construction can leave whole pages without resident physical backing. If another initialization pass already wrote every page, this benefit would disappear.

My workload has 47k records. Empirical measurement shows that this change prevents > 900 KB of record pages are from being allocated in physical memory. This saving is contingent on the specific access pattern, but if only a small fraction of all records are expected to be initialized, the saving should broadly generalise, save for specifically constructed artificial access patterns that touch every page.

If memory savings, however, that PR won't help and this'd still want to be considered.

Why wouldn't it help? From what I can gather, it helps me in practice.

Wasmtime GitHub notifications bot (Oct 26 2026 at 18:53):

leventov edited a comment on PR #14974:

The goal was memory savings.

Please forgive me an LLM reply (edited):

The memset and the saved memory concern different storage. The PR still clears every pointer-sized table slot. It avoids constructing the separate 32-byte VMFuncRef records inside VMContext until needed. Those record pages are not already initialized elsewhere. Wasmtime reserves their storage but deliberately does not zero or initialize the function-reference region during instance creation. Consequently, skipping record construction can leave whole pages without resident physical backing. If another initialization pass already wrote every page, this benefit would disappear.

My workload has 47k records. Empirical measurement shows that this change prevents > 900 KB of record pages from being allocated in physical memory. This saving is contingent on the specific access pattern, but if only a small fraction of all records are expected to be initialized, the saving should broadly generalise, save for specifically constructed artificial access patterns that touch every page.

If memory savings, however, that PR won't help and this'd still want to be considered.

Why wouldn't it help? From what I can gather, it helps me in practice.

Wasmtime GitHub notifications bot (Oct 28 2026 at 14:46):

alexcrichton commented on PR #14974:

Overall I know LLMs are good at whipping up an implementation, but at least personally I'm not comfortable landing this change. A feature like this I think needs to be extremely carefully designed because any possible bug can pretty easily lead to a security issue for example. I also feel that this is sort of a half-measures implementation where the table is still memset to 0 and it's just the VMFuncRef storage is lazily initialized. If we're going the route of full lazy initialization of all tables in all places it seems like we should shoot for a full-and-complete solution of lazily initializing everything, not just the VMFuncRef slots.

In general though I think this is going to require some careful design and review. For example this probably shouldn't end up sprinkling a flag to disable call-hooks which seems pretty unrelated. Similarly this ideally would centralize "apply lazy init" at compile-time in one location as it is today, as opposed to spreading it across a few locations. The testing here is also quite light comparatively with the weight of this change, and that would probably want to be improved, ideally with *.wast, and possibly with better fuzzer integration.

Last updated: Oct 31 2026 at 23:03 UTC