Stream: git-wasmtime

Topic: wasmtime / PR #15034 Do not force the `VMDeferredThread` ...

Wasmtime GitHub notifications bot (Oct 27 2026 at 20:59):

fitzgen opened PR #15034 from fitzgen:lazy-lenders to bytecodealliance:main:

This commit introduces a lazy_lenders buffer on VMStoreContext. When we need to push a lender for a resource borrow, but haven't forced the lazy thread context yet, we push to lazy_landers. This avoids forcing the lazy thread context, like we previously did. Each VMDeferredThread keeps a watermark of the lazy_lenders length upon construction so that, when we do something that does force the lazy thread context, we can partition lazy_lenders by each thread context that is responsible for those borrows.

This does not change what is inlined in Wasm code or what is done in libcalls. In the inline exit-sync-call fast path, we bail out to the host when there are lazy lends to be undone. The host libcall still takes care to avoid forcing the VMDeferredThread right before we exit the call in this case.

Follow up commits will move a bunch of this work inline into the adapters, but I wanted to keep this introduction of lazy_lenders as small as I could.

Even so, this is a significant speedup for cm-online-stats, our adapter-call-heavy Sightglass benchmark:

execution :: instructions-retired :: cm-online-stats

    lazy-lenders/libengine.dylib is 8.4852x to 8.4854x faster than main/libengine.dylib!
    (confidence = 99%, p < 0.0001)

    ┌────────────┬────────────┬───────────────┬───────────────┬────────┬──────────────────────────────┐
    │ Min        │ Max        │ Mean          │ Median        │ Noise  │ Engine                       │
    ├────────────┼────────────┼───────────────┼───────────────┼────────┼──────────────────────────────┤
    │ 819957895  │ 827344185  │ 820584935.77  │ 819983269.00  │ 0.014% │ lazy-lenders/libengine.dylib │
    ├────────────┼────────────┼───────────────┼───────────────┼────────┼──────────────────────────────┤
    │ 6957508999 │ 6980867062 │ 6962057536.66 │ 6957759525.50 │ 0.192% │ main/libengine.dylib         │
    └────────────┴────────────┴───────────────┴───────────────┴────────┴──────────────────────────────┘

execution :: cycles :: cm-online-stats

    lazy-lenders/libengine.dylib is 7.41x to 7.48x faster than main/libengine.dylib!
    (confidence = 99%, p < 0.0001)

    ┌────────────┬────────────┬───────────────┬───────────────┬────────┬──────────────────────────────┐
    │ Min        │ Max        │ Mean          │ Median        │ Noise  │ Engine                       │
    ├────────────┼────────────┼───────────────┼───────────────┼────────┼──────────────────────────────┤
    │ 156815386  │ 183006652  │ 162831928.91  │ 161767696.00  │ 1.037% │ lazy-lenders/libengine.dylib │
    ├────────────┼────────────┼───────────────┼───────────────┼────────┼──────────────────────────────┤
    │ 1170592107 │ 1302149964 │ 1210932378.38 │ 1206847175.00 │ 2.839% │ main/libengine.dylib         │
    └────────────┴────────────┴───────────────┴───────────────┴────────┴──────────────────────────────┘

compilation :: instructions-retired :: cm-online-stats

    main/libengine.dylib is 1.00x to 1.03x faster than lazy-lenders/libengine.dylib!
    (confidence = 99%, p = 0.0017)

    ┌──────────┬──────────┬─────────────┬─────────────┬────────┬──────────────────────────────┐
    │ Min      │ Max      │ Mean        │ Median      │ Noise  │ Engine                       │
    ├──────────┼──────────┼─────────────┼─────────────┼────────┼──────────────────────────────┤
    │ 67817425 │ 85606560 │ 77573786.81 │ 77931355.50 │ 5.618% │ lazy-lenders/libengine.dylib │
    ├──────────┼──────────┼─────────────┼─────────────┼────────┼──────────────────────────────┤
    │ 66424126 │ 85683019 │ 76488563.76 │ 76928252.50 │ 5.252% │ main/libengine.dylib         │
    └──────────┴──────────┴─────────────┴─────────────┴────────┴──────────────────────────────┘

compilation :: cycles :: cm-online-stats

    No difference in performance.

    ┌──────────┬───────────┬─────────────┬─────────────┬─────────┬──────────────────────────────┐
    │ Min      │ Max       │ Mean        │ Median      │ Noise   │ Engine                       │
    ├──────────┼───────────┼─────────────┼─────────────┼─────────┼──────────────────────────────┤
    │ 23475857 │ 93001875  │ 51231120.40 │ 48774961.50 │ 32.407% │ lazy-lenders/libengine.dylib │
    ├──────────┼───────────┼─────────────┼─────────────┼─────────┼──────────────────────────────┤
    │ 22429448 │ 123149979 │ 53576849.80 │ 48918162.00 │ 43.167% │ main/libengine.dylib         │
    └──────────┴───────────┴─────────────┴─────────────┴─────────┴──────────────────────────────┘

<!--
Please make sure you include the following information:

• If this work has been discussed elsewhere, please include a link to that
  conversation. If it was discussed in an issue, just mention "issue #...".

• Explain why this change is needed. If the details are in an issue already,
  this can be brief.

Our development process is documented in the Wasmtime book:
https://docs.wasmtime.dev/contributing-development-process.html

Please review the Bytecode Alliance's AI tool usage policy at
https://github.com/bytecodealliance/governance/blob/main/AI_TOOL_POLICY.md

Please ensure all communication follows the code of conduct:
https://github.com/bytecodealliance/wasmtime/blob/main/CODE_OF_CONDUCT.md
-->

Wasmtime GitHub notifications bot (Oct 27 2026 at 20:59):

fitzgen requested cfallin for a review on PR #15034.

Wasmtime GitHub notifications bot (Oct 27 2026 at 20:59):

fitzgen removed the review request for wasmtime-compiler-reviewers from PR #15034.

Wasmtime GitHub notifications bot (Oct 27 2026 at 20:59):

fitzgen requested wasmtime-compiler-reviewers for a review on PR #15034.

Wasmtime GitHub notifications bot (Oct 27 2026 at 20:59):

fitzgen requested wasmtime-core-reviewers for a review on PR #15034.

Wasmtime GitHub notifications bot (Oct 27 2026 at 21:42):

fitzgen updated PR #15034.

Wasmtime GitHub notifications bot (Oct 27 2026 at 23:44):

github-actions[bot] added the label wasmtime:api on PR #15034.

Wasmtime GitHub notifications bot (Oct 29 2026 at 04:01):

:memo: cfallin submitted PR review:

Thanks! This rates as "plausible" to me only because I'm not deeply familiar with the borrow machinery here; @alexcrichton should probably look it over too. Just a few comments I noted below as I read through.

Wasmtime GitHub notifications bot (Oct 29 2026 at 04:01):

:speech_balloon: cfallin created PR review comment:

"Watermark" to me implies some sort of lazy "max seen so far" behavior; this is really just "start of our segment in lazy_lenders", right? So lazy_lenders_index or lazy_lenders_start might be more clear?

Wasmtime GitHub notifications bot (Oct 29 2026 at 04:01):

:speech_balloon: cfallin created PR review comment:

We could avoid the smallvec clone here with a hypothetical components_data.lazy_lenders_and_task_state_mut() returning a split borrow, perhaps (similar to the trick we play in a bunch of instances on StoreOpaque)?

Wasmtime GitHub notifications bot (Oct 29 2026 at 04:02):

cfallin requested alexcrichton for a review on PR #15034.

Wasmtime GitHub notifications bot (Oct 29 2026 at 15:13):

fitzgen updated PR #15034.

Wasmtime GitHub notifications bot (Oct 29 2026 at 15:34):

fitzgen updated PR #15034.

Wasmtime GitHub notifications bot (Oct 29 2026 at 20:56):

:memo: alexcrichton submitted PR review.

Wasmtime GitHub notifications bot (Oct 29 2026 at 20:56):

:speech_balloon: alexcrichton created PR review comment:

A few thoughts on this:

• One possibly more-optimizable shape here would be to store the number of lazy lens per-frame rather than storing the current length before the frame in the frame. That way if nothing actually does a borrow then this never even looks at lazy_lenders_start.
• Is band the right way to generate good clif here? I'd expect we'd have better codegen from two icmp + brif instructions as opposed to a band which'd require a bunch of seto and test and such. Perhaps arguably something to happen in ISLE too, but my guess is that our current codegen for brif(band(icmp(...), icmp(...))) is pretty bad

Wasmtime GitHub notifications bot (Oct 29 2026 at 20:56):

:speech_balloon: alexcrichton created PR review comment:

Subjectively I'd say we could probably get away with a much smaller value here, even something like 4 or maybe 8. This seems relatively difficult to fill in conventional components where there's not all that much cross-component-talk and fast paths generally need just one or two slots.

Wasmtime GitHub notifications bot (Oct 29 2026 at 20:56):

:speech_balloon: alexcrichton created PR review comment:

We talked about this briefly elsewhere, but personally I'd say just go ahead and remove this and inline the constant LAZY_LENDERS_CAPACITY everywhere since it seems pretty unlikely we'd want to double this at runtime.

Wasmtime GitHub notifications bot (Oct 29 2026 at 20:56):

:speech_balloon: alexcrichton created PR review comment:

Instead of a packed u64 would it make sense for this to be just two u32 fields, perhaps with the raw index type itself?

Wasmtime GitHub notifications bot (Oct 29 2026 at 20:56):

:speech_balloon: alexcrichton created PR review comment:

Would it make sense to back this up with a debug_assert!?

Wasmtime GitHub notifications bot (Oct 29 2026 at 20:56):

:speech_balloon: alexcrichton created PR review comment:

Instead of a LazyLendersFull error, could this perhaps return Option<..> where None means "no you can't push the capacity is full"

Wasmtime GitHub notifications bot (Oct 29 2026 at 20:56):

:speech_balloon: alexcrichton created PR review comment:

How come this is all #[cfg]'d to component-model-async? I'd naively expect the optimization here to equally apply with/without component-model-async

Wasmtime GitHub notifications bot (Oct 29 2026 at 20:56):

:speech_balloon: alexcrichton created PR review comment:

Would it be possible to just delete this code? This seems like it'll all quickly become dead code as this would otherwise happen in cranelift-compiled code otherwise. I think it's reasonable to add the lazy infrastructure here, but I also think it'd be best to avoid adding pieces that won't survive to the final implementation (where possible)

Wasmtime GitHub notifications bot (Oct 29 2026 at 20:56):

:speech_balloon: alexcrichton created PR review comment:

This method is mostly a duplicate of what's below, in particular the let guest block and assembling all the borrows and such. Could these methods be combined to share more of the implementation?

Wasmtime GitHub notifications bot (Oct 29 2026 at 20:56):

:speech_balloon: alexcrichton created PR review comment:

Any particular reason to split out reserve/push here? OOM-wise I figure it doesn't really matter if we drop a borrow or two while we're trapping due to OOM

Wasmtime GitHub notifications bot (Oct 29 2026 at 20:56):

:speech_balloon: alexcrichton created PR review comment:

We talked a bit about this elsewhere, but I wanted to write this down here too. I think it'd be best to ideally remove the need for this method altogether where the lazy lend is a record of "here's the borrows we haven't yet accounted for in the official tables" and then undoing that is a simple "move the stack pointer" style operation which needs no sort of destructor or such. That would make actually recording a lazy lend that much faster (just put the resource in a table) and would keep everything consistent I believe as well.

I'm hopeful that something like this would help simplify both this PR as-is and subsequent PRs to add more infrastructure, but I could also be off-the-mark here

Wasmtime GitHub notifications bot (Oct 29 2026 at 20:56):

:speech_balloon: alexcrichton created PR review comment:

To keep raw values, maybe flip this and have the host at u32::MAX and other types as-is?

Last updated: Oct 31 2026 at 23:03 UTC