Stream: git-wasmtime

Topic: wasmtime / issue #12311 Optimize guest-to-guest sync-to-s...

Wasmtime GitHub notifications bot (Jan 09 2026 at 21:52):

alexcrichton opened issue #12311:

This is a meta/tracking issue about remaining work necessary to optimize the guest-to-guest sync-to-sync adapter generated by Wasmtime when component-model-async is enabled. Some more historical discussion of this happened at #wasmtime > Wasmtime sync<->sync adapter optimizability @ 💬 as well, and I'll try to keep this up-to-date.

What is the problem

Wasmtime will compile an "adapter" with the FACT compiler when one guest component calls another. With the advent of component-model-async this adapter has a large number of permutations, for example the caller could be sync/async lowered, the callee could be sync/async lifted, and the function type itself could be sync or async. This specific issue is about the single case of a sync lowered caller, sync lifted callee, and sync function type. This doesn't mean the other permutations should be ignored, but that's the most interesting case for now.

Additionally with the advent of component-model-async it's required, spec-wise, to manage async-task-related-infrastructure when crossing component boundaries. Task infrastructure comes into play in a number of scenarios, such as:

• When a task calls an imported function, that creates a new task. This new task has the current task as a parent task.
• Intrinsics such as backpressure.{inc,dec} modify the backpressure counter in the current task.
• When a task exits/returns all of its pending subtasks are "reparented" to the task's own parent.

Effectively, there's substantial infrastructure pieces that may be used across component boundaries, and thus Wasmtime needs to handle this. This leads us to the problem: with component-model-async disabled this task management is all ignored as it's not applicable, but with component-model-async enabled this task management is enabled. This means that the sync<->sync adapter will call a host function to manage task infrastructure pieces.

This cost of this hostcall is relative to the situation of the adaptation being performed, but the goal of sync<->sync adapter is to, ideally, compile to a grand total of 0 instructions. Given that it's impossible to optimize away a call into the host, this issue is thus about the problem of solving the task infrastructure management problem without actually making a host call. This should restore the prior-to-component-model-async behavior of a sync<->sync adapter compiling to pure optimizable CLIF which mostly boils away.

History and Current Status

As of the time of this writing Wasmtime doesn't actually do any manipulation of task infrastructure on sync<->sync adapters. This is a bug and results in issues such as https://github.com/bytecodealliance/wasmtime/issues/12128 (plus many undocumented others we have since realized). @dicej will soon have a PR to fix this situation where task infrastructure will be maintained across these boundaries.

The plan is to have a PR which will enhance the sync<->sync adapter with task infrastructure management, conditionally. The condition will be based on whether the component-model-async wasm feature is enabled in the Config. This is intended to be a stopgap because embedders should not need to disable features for performance. For the time being though it'll retain the pre-p3 performance profile of sync adapters while retaining p3-relatevant spec compliance.

Future plans for optimization

Enabling Cranelift to compile these adapters to zero instructions is going to require special care and a number of refactorings of Wasmtime's task infrastructure in addition to new Cranelift optimizations. The general rough idea for the implementation is:

• A new VMAsyncTask type will be added. Fields this will contain are:
  • A "kind", more relevant in a moment
  • Fields for context.set {0,1}
  • A parent pointer for the parent task. Option<NonNull<VMAsyncTask>>
  • Backpressure fields (if necessary still, we've talked about removing backpressure)
  • A flag of whether this task can block or not.
• The Rust-based "full" async task will contain this field as well as any other tables and such necessary. This will be similar to VMContext vs vm::Instance, for example.
• The current task will be stored in VMContext or VMComponentContext (maybe both? unsure?)
• Sync<->sync adapters will allocate, on the stack, a VMAsyncTask with just these fields. This will be initialized with the current task and then the current task will be set to this.
• Manipulations of the current task will go directly through VMAsyncTask if applicable, e.g. context.{g,s}et {0,1}
• Manipulations of the current task that require Rust data structures, for example adding a subtask, will "promote" the task from the stack to the Rust heap. This will go back through the entire chain of tasks and promote them all to the heap most likely too.
• Returning from a sync<->sync adapter will restore the current task to its previous value.

Effectively, at a high level, sync<->sync adapters will allocate a task on the stack that, if necessary, will get promoted to the Rust heap to perform more expensive maniuplations on. In essence Rust-level tasks are lazily created only as necessary for "more complicated" things, like spawning subtasks, while low-level actions like context.get will remain efficient.

The resulting CLIF for a sync<->sync adapter will pseudo-code look like:

void adapter(vmctx *vmctx) {
    vmtask *prev_head = vmctx->current_task;
    vmtask stack_node;
    stack_node->kind = VMTASK_STACK;
    // ...
    stack_node->prev = prev_head;
    vmctx->current_task = &stack_node;

    the_callee_component(vmctx);

    vmctx->current_task = prev_head;
}

If the_callee_component(vmctx) is small enough the theory here is:

• Cranelift will see that vmctx->current_task is loaded, stored to, then stored to with the previous value. If the_callee_component(vmctx) has no obviously aliasing regions, then it can eliminate both stores as dead.
• If the_callee_component doesn't actually do anything like call the host then Cranelift will see that all the stores to stack_node are unused, so they're all eliminated.
• If all the previous loads/stores were eliminated, then the load from vmctx->current_task is also dead, so that's also eliminated.

I don't believe that Cranelift will perform all of these optimizations, but my understanding so far is that this is well within Cranelift's complexity budget and wheelhouse to implement optimizations like these.

Expected Timeline

The current plan is to ship the hostcall-to-manipulate-task-infrastructure with WASIp3 originally. Embeddings that need the highest performance on sync<->sync adapters will disable the component-model-async runtime feature (and maybe compile time feature). After WASIp3 ships and we have enough time to come back to this and design this all "for real" we'll implement this. At that point it won't matter if engines turn the component-model-async feature on-or-off, it'll be the same.

Another point to note here is that it's expected that in WASIp3 Wasmtime will need to pretty heavily optimize calls to context.{get,set}. This work, while not the same as optimizing get/set, is highly related and will likely be a prerequisite for this work. That's to say that this work isn't solely motivated by sync<->sync adapters, but instead it's motivated by other routes too.

Wasmtime GitHub notifications bot (Jan 09 2026 at 21:52):

alexcrichton added the wasm-proposal:component-model-async label to Issue #12311.

Wasmtime GitHub notifications bot (Jan 09 2026 at 22:14):

cfallin commented on issue #12311:

If the_callee_component doesn't actually do anything like call the host then Cranelift will see that all the stores to stack_node are unused, so they're all eliminated.

Unless I'm misunderstanding the problem statement, I think this is outside the scope of ordinary dead-store elimination or the sort of thing Cranelift would tackle: it implies interprocedural program analysis, which is fundamentally hard.

Said another way: you're pushing a local alloc onto a linked list, then calling some arbitrary code; absent some global analysis, we can't know that that code won't eventually reach some behavior that will require observing that list, right? And that global analysis would need to reason about the callgraph, which depends on a value-range analysis and points-to analysis, both of which are extremely expensive, imprecise (overly conservative / brittle, easy to collapse with the wrong operator), or both.

Separately, we'd also need an escape analysis to not do that local alloc at all, right? That's a whole separate can of worms. Possible, but complex.

Overall: I'd be somewhat concerned waiting for a "sufficiently smart compiler" to get good component-to-component call performance; while it is definitely within scope to build new optimizations, trying to derive-from-first-principles why the code we emitted is unnecessary is always less preferable than modifying the runtime (or spec?) so we don't need that code.

Wasmtime GitHub notifications bot (Jan 09 2026 at 22:23):

alexcrichton commented on issue #12311:

No no, I understand that interprocedural analysis is off the table. I can try to expand more on this in a Cranelift meeting if desired to double-check the optimizations are in-scope.

What I want Cranelift to be able to optimize is something like:

v0 = stack_addr ;; some stack-based node
v1 = load vmctx+0x100 ; load prev
store vmctx+0x100, v0
;; some inlined version of `the_callee_component` that clearly doesn't store to vmctx based on alias analysis
store vmctx+0x100, v1

Here the first store is dead since it's never read, so it's eliminated. It's also into the vmctx so it's trusted/notrap/etc. The second store is then the same as what was loaded, so there's no need to load-then-store, so it's eliminated. Then the load is eliminated because it's dead.

I'd be somewhat concerned waiting for a "sufficiently smart compiler" to get good component-to-component call performance

Oh don't worry, I've worked long enough with Rust and optimizations that a sufficiently-smart-compiler is "this either works with simple-ish heuristics or not at all".

Basically I didn't explicitly say that the_callee_component(vmctx) was inlined, but for all optimizations above I meant "this optimization is only applicable when the entire body is fully inlined". The puropse is to ensure component functions using unsafe intrinsics, which are expected to be fully inlined, to boil away the surrounding infrastructure

Wasmtime GitHub notifications bot (Jan 09 2026 at 22:26):

cfallin commented on issue #12311:

Ah, I see -- yeah, if we're also assuming cross-component inlining then this is again intraprocedural, and at least tractable. Thanks for the clarification!

Wasmtime GitHub notifications bot (Jun 10 2026 at 19:27):

fitzgen commented on issue #12311:

FYI, I'm circling back to the compile-time builtins and unsafe intrinsics and this issue is the main blocker for completing that work stream, so I'd like to start on this. Trying to make sure I understand everything involved and what I can do to split it up into incremental chunks, and which chunks are actually blockers for compile-time builtins performance vs just improving CM-async performance in general.

@alexcrichton

• Manipulations of the current task that require Rust data structures, for example adding a subtask, will "promote" the task from the stack to the Rust heap. This will go back through the entire chain of tasks and promote them all to the heap most likely too.

Can you clarify what you mean by promoting the task from the stack to the heap? Like literally moving the VMAsyncTask from a stack slot in a trampoline out to some Box<VMAsyncTask> or whatever on the native heap? If so, then I have two follow up questions:

1. IIUC, this promotion can be triggered deep down the stack for VMAsyncTasks in a stack slot of a frame up the stack, so how do frames up the stack know that their VMAsyncTask moved? Are you imagining we reuse the stack maps machinery for this? That the ABI is extended to return the updated pointer somehow? Something else?

2. We're talking about guest-to-guest calls, which use adapters that are generated Wasm, other than some specialized extra component builtins that are generated directly from CLIF. Since the VMAsyncTask needs to be inside a stack slot, and Wasm can't have explicit stack slots without a whole memory and shadow stack, that means that we need to use new CLIF-generated component builtins for this stuff, right? But unlike existing CLIF-generated component builtins, these would need to be whole trampolines because the frame (and its VMAsyncTask stack slot) needs to stay live across the whole rest of the adapter/cross-component call, right? That seems like a pretty big change to how guest-to-guest calls work, and also not necessarily something we would want to do when we can't rely on inlining to make the extra call frames go away (nor do we have gc sections to make the binary bloat go away). I guess all this boils down to: am I misunderstanding something? And if not, is this really the design we want?

Wasmtime GitHub notifications bot (Jun 10 2026 at 20:19):

alexcrichton commented on issue #12311:

Definitely no use of stack maps or ABI changes were envisioned, and as you've pointed out here this wasn't fleshed out the point of being an as-is spec that could be followed. Definitely not envisioning transitioning adapters to pure CLIF away from the wasm they have today as well.

My rough thinking on this was that the on-stack VMAsyncTask would be promoted to the current GuestTask-in-table that it is today. More-or-less the ConcurrentState::current_thread field would get removed in favor of "something stored in VMStoreContext". Accessors would go through some new method that would internally dispatch on whatever VMStoreContext said, and if it pointed to something on the stack then the current GuestTask would be lazily allocated, the id placed in the VMStoreContext, and then that'd be returned like usual today. The goal is that Rust has enough data to lazily execute that is enter_sync_call today if necessary. What I'd want to avoid is storing a stack pointer anywhere beyond the VMStoreContext because we otherwise will probably just lose track of it.

For things up-the-stack, I think this would mean that the trampoline would load-and-compare against what the currently running thread is. If it's the same upon entry, then it's restored, in compiled code, and otherwise it delegates to the host saying "pop your stuff".

For stack slots and such, this'll probably require some suite of intrinsics and such. Wasm already can't directly interact with the VMContext or VMStoreContext, but we could always invent one-off unsafe intrinsics for doing so and hook those in here. The stack slot itself could be handled in a few ways in theory:

• One is that because unsafe intrinsics are inlined directly we could rely on that. We could assert that a "standalone" version of "define a stack slot" is never generated and then call an intrinsic for "define a stack slot in my function" of sorts (which is inlined directly at translation time)
• Another is that we could flag wasm-to-wasm trampolines as "special" which would define a stack slot that intrinsics would then access.
• Another is that we could have a thin CLIF trampoline around the actual wasm-to-wasm trampoline which serves the purpose of doing raw vmctx things which would move logic out of the wasm.
• We could also go the whole 9 yards and emit custom wasm opcodes that are re-parsed/translated. Normal wasm functions wouldn't allow these opcodes but we could handle them specially (in theory).

I'll agree that none of these are really all that great options. We could perhaps start a bit more conservatively and say that there's only ever at most one "stack frame" which lives in the VMStoreContext itself. The entry point of sync-to-sync would then promote that in-use entry, if any, via a hostcall. If it's not in use then it would be flagged as in-use. That way we would only need to invent intrinsics to frob/modify VMContext which wouldn't be too bad (just some more UnsafeIntrinsic I believe).

is this really the design we want?

I'd consider this space open for interpretation to tweak/adjust the design as necessary. No need to take anything here as a hard-and-fast constraint. The main constraint which can't really be adjusted is the need handle the task stuff in the first place, that's sort of just a given now.

Wasmtime GitHub notifications bot (Jun 11 2026 at 21:03):

fitzgen commented on issue #12311:

Alex and I just chatted some more about this, and came up with this initial sketch of a plan for specifically unblocking the fast-path that unsafe intrinsics and compile-time builtins will be on:

• Add something like the JIT-accessible equivalent of the following types to VMStoreContext:
  ```rust
  struct VMStoreContext {
  // ...existing fields...

  current_thread: Option<VMLazyThreadPtr>,
  

  }

  enum VMLazyThreadPtr {
  Deferred(VmPtr<VMDeferredThread>),
  Forced(CurrentThreadId),
  }

  struct VMDeferredThread {
  // Parent thread.
  parent: Option<VMLazyThreadPtr>,

  // Deferred arguments to `vm::component::libcalls::enter_sync_call`.
  caller_instance: RuntimeComponentInstanceIndex,
  callee_async: bool, // TODO: might always be false for the lazy/deferred case?
  callee_instance: RuntimeComponentInstanceIndex,
  

  }
  ```

• when translating fused adapters to CLIF (recognized via inspecting the FuncKey), wrap the translation of the generated Wasm to include a little extra stuff before and after the main translation so we emit code something like this:

  ``rust function %adapter(...) -> ... { ss0 = explicit stack slot sized/aligned for a VMDeferredThread`

  block0(...):
  // Push this component call's deferred thread.
  let store_ctx = vmctx->store_ctx;
  *ss0 = VMDeferredThread {
  parent: store_ctx->current_thread,
  // ... arguments that would have been eagerly passed
  // to a libcalls::enter_sync_call call today...
  };
  vmctx->store_ctx.current_thread = Some(VMLazyThreadPtr::Lazy(&ss0));

  // ... regular translation of the fused adapter here ...
  
  if store_ctx->current_thread == Some(VMLazyThreadPtr::Lazy(&ss0)) {
      // If we didn't do any async operation or whatever that forced the
      // allocation of an actual task/thread, then we can just pop our
      // lazy thread off the stack.
      store_ctx->current_thread = ss0->parent;
  } else {
      // Otherwise, when the callee did some kind of async or concurrent
      // stuff, then take a slow, out-of-line libcal path that is equivalent
      // to what we do today.
      libcalls::exit_sync_call_slow();
  }
  
  return ...
  

  }
  ```

• In wasmtime::runtime::component::concurrent, wherever we access .current_thread, that needs to become a call to store.force_current_thread():
  rust impl StoreOpaque { fn force_current_thread(&mut self) -> CurrentThread { ... } }

  • Which probably means that a bunch of methods need to move to StoreOpaque from ConcurrentState or wherever. Possibly borrowing hiccups to resolve.
  • This mechanical method movement should be done as a mechanical commit first, before making the rest of the changes described to make things easier to review.

The idea is that compile-time builtins and unsafe intrinsics won't do anything that calls store.force_current_thread() which is the thing that promotes a deferred lazy thread into a runtime-managed CurrentThread structure. With inlining enabled, Cranelift should be able to see all that, and:

* do store-to-load forwarding to turn store_ctx->current_thread == ss0 into ss0 == ss0
* turn if ss0 == ss0 { then... } else { otherwise... } into unconditional then... via GVN and branch simplification
* at this point, we are just doing a couple of inline flag sets and unsets, similar to pre CM-async
* although we will still be doing the may-leave flag manipulation that pre CM-async does, so this is still additional overhead compared to pre CM-async. see additional notes below.
* eventually, once we implement dead-store elimination in Cranelift, it can additionally:
* recognize that store_ctx->current_thread = ss0 in the prologue is dead, since it is post-dominated by store_ctx->current_thread = ss0->parent in the epilogue and never read in between, and remove it
* recognize that ss0->parent is now a dead load as well and remove it
* recognize that the stores that write the deferred thread to ss0 are dead now as well after the previous load was removed and can themselves be removed
* recognize that the stack slot's address is never taken and it is never stored to or loaded from, and the stack slot can be removed as well
* at this point, we should now have identical codegen as pre CM-async

Additional notes:

• removing the may-leave flag manipulation requires either spec changes or, when lazy lowering becomes a thing, for the caller component to only use lazy lowering for all component functions it lowers (since lazy lowering never unsets the may-leave flag).
• If we could LICM trap[n]z instructions, then the may-leave flag manipulation maybe doesn't matter quite as much, since doing a tight loop over unsafe intrinsic calls to directly manipulate a host buffer or whatever could in theory hoist the may-leave flag manipulation our of the loop. But this could also be easy to accidentally break if there is any other side effect between the trap[n]z and the loop header which would prevent this code motion.

Wasmtime GitHub notifications bot (Jun 11 2026 at 21:49):

alexcrichton commented on issue #12311:

lgtm, and one final caveat will be to handle updating context.{get,set} values which currently live in the VMStoreContext. This is akin to this block in set_thread which is delegated to within enter_sync_call currently. Effectively the trampoline will need to load the previous values in the store, insert 0s, and then in the epilogue of the function restore the original values. That should be sufficient for handling this though.

Wasmtime GitHub notifications bot (Jul 02 2026 at 14:16):

fitzgen commented on issue #12311:

After #13695, we got the following speed up for our compile-time builtins benchmarks (which make many tiny cross-component calls) when cm-async is enabled:

increment-each-byte-in-buf/concurrency_support=true/compile-time-builtins-host-buf-api
                        time:   [749.55 µs 750.89 µs 752.28 µs]
                        change: [−98.997% −98.965% −98.931%] (p = 0.00 < 0.05)
                        Performance has improved.

increment-random-byte-in-buf/concurrency_support=true/compile-time-builtins-host-buf-api
                        time:   [396.64 ns 397.85 ns 399.26 ns]
                        change: [−64.968% −64.349% −63.716%] (p = 0.00 < 0.05)
                        Performance has improved.

We still have ways to go to get back to performance before cm-async, however:

increment-each-byte-in-buf/concurrency_support=false/compile-time-builtins-host-buf-api
                        time:   [318.46 µs 323.09 µs 328.66 µs]

increment-random-byte-in-buf/concurrency_support=false/compile-time-builtins-host-buf-api
                        time:   [99.524 ns 99.769 ns 100.01 ns]

So cm-async is still

• 2.32x slower on increment-each-byte-in-buf, and
• 3.99x slower on increment-random-byte-in-buf

Wasmtime GitHub notifications bot (Jul 02 2026 at 14:31):

fitzgen commented on issue #12311:

And my dead-store elimination branch (will make a PR soon) yields another small speed up:

increment-each-byte-in-buf/concurrency_support=true/compile-time-builtins-host-buf-api
                        time:   [760.93 µs 763.02 µs 765.31 µs]
                        change: [+0.7952% +1.0940% +1.4500%] (p = 0.00 < 0.05)
                        Change within noise threshold.

increment-random-byte-in-buf/concurrency_support=true/compile-time-builtins-host-buf-api
                        time:   [396.16 ns 397.44 ns 398.85 ns]
                        change: [−12.682% −10.658% −8.7674%] (p = 0.00 < 0.05)
                        Performance has improved.

increment-each-byte-in-buf/concurrency_support=false/compile-time-builtins-host-buf-api
                        time:   [296.47 µs 297.83 µs 299.17 µs]
                        change: [−26.293% −24.048% −21.729%] (p = 0.00 < 0.05)
                        Performance has improved.

increment-random-byte-in-buf/concurrency_support=false/compile-time-builtins-host-buf-api
                        time:   [99.568 ns 99.805 ns 100.05 ns]
                        change: [−25.627% −23.111% −20.554%] (p = 0.00 < 0.05)
                        Performance has improved.

Wasmtime GitHub notifications bot (Jul 02 2026 at 14:55):

alexcrichton commented on issue #12311:

The numbers here seem off from what I expect, so digging in a bit I think that the benchmark in question isn't measuring the desired overhead which is skewing things a bit. The benchmark here is measuring both the overhead of entering wasm and the component-to-component overhead since each iteration of the benchmark is invoking wasm. Entering wasm was not optimized in https://github.com/bytecodealliance/wasmtime/pull/13695, and that's also not the subject of this issue, although still worthwhile to track.

Using this diff and manually changing concurrency_support to true/false I get:

increment-random-byte-in-buf/compile-time-builtins-host-buf-api
                        time:   [4.6786 ns 4.7029 ns 4.7324 ns]
                        change: [+125.61% +141.99% +159.60%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 13 outliers among 100 measurements (13.00%)
  3 (3.00%) high mild
  10 (10.00%) high severe

which that number makes a lot more sense to me. With concurrency disabled the per-element increment is 1.8ns and with concurrency enabled it's 4.7ns. That's obviously still a difference, but this is more in-line with what I'd expect a handful of loads/stores to add as overhead (e.g. ~3ns)

Wasmtime GitHub notifications bot (Jul 02 2026 at 15:05):

fitzgen commented on issue #12311:

@alexcrichton yes, it is measuring the overhead of calling into Wasm, but that is the same constant overhead for all cases, so shouldn't really matter.

Using this diff

This is incrementing the same element every time in a way that Cranelift can see it is always the same element, so this isn't really measuring the intended workload anymore either. Better would be to inline the RNG into the guest or something.

Wasmtime GitHub notifications bot (Jul 02 2026 at 19:23):

alexcrichton commented on issue #12311:

The overhead isn't the same though because concurrency-support affects both the caller and callee side, so in this case I think the benchmark needs an update one way or another if the intention is to primarily measure the component-to-component boundary. Additionally the component-to-component boundary should be dwarfed by the host-to-wasm boundary, so I think it'd be good to get that out of the picture for measuring anyway.

Good point though about Cranelift optimizing this, but I still think that the benchmark here should be updated if it wants to be a meaningful data point for the issue here of guest-to-guest communication

Last updated: Jul 29 2026 at 05:03 UTC