Stream: git-wasmtime

Topic: wasmtime / PR #12990 MMU-based epoch interruption

Wasmtime GitHub notifications bot (Apr 08 2026 at 16:33):

erikrose opened PR #12990 from erikrose:epoch-mmu to bytecodealliance:main:

This is an implementation of #1749, specifically @cfallin's roadmap, with the goal of reducing the overhead of checking for the end of epochs.

Paul ran some benchmarks on this (broadly agreeing with our real-world experiments) which tell us:

• Current compare-against-a-deadline -Wepoch-interruption=y is a 14.4% hit versus doing nothing.
• Doing only dead loads in function prologues and loop headers (which were all that was implemented in this patch at the time of the bench) brings that down to a 2.8% hit. There will be some additional hit from the signal handler that actually effects the task switch, but that's on the cold path.

The above numbers are from SpiderMonkey, which I deem the most representative benchmark.

Status:

• [X] Add unmapped-on-interrupt page and pointer to it in vmctx.
• [X] Add method for embedder to call to bring an epoch to a close.
• [X] Add DeadLoadWithContext Cranelift instruction, and use it.
• [ ] Have DeadLoadWithContext emit metadata into compiled-artifact tables to tell the signal handler this is an interruption-point load.
• [ ] Add logic to signal handler that, when seeing such a PC, updates state to redirect to the stub, saving the original PC (probably in the scratch register).
• [ ] Add that stub, which saves all register state and invokes a hostcall with the recovered vmctx.

If the TLB shootdown arising from the frobbing of privs on the "interrupt page" proves too expensive, we can try a more indirect load instead, where, instead of messing with page privs, we mess with the address we're dead-loading from so it points to either a (statically) allowed or forbidden page. (Chris floated this idea at the 2026-04-08 Cranelift meeting.) Not many of the other mechanics need change.

Wasmtime GitHub notifications bot (Apr 08 2026 at 20:00):

github-actions[bot] added the label cranelift on PR #12990.

Wasmtime GitHub notifications bot (Apr 08 2026 at 20:00):

github-actions[bot] added the label cranelift:area:x64 on PR #12990.

Wasmtime GitHub notifications bot (Apr 08 2026 at 20:00):

github-actions[bot] added the label cranelift:area:machinst on PR #12990.

Wasmtime GitHub notifications bot (Apr 08 2026 at 20:00):

github-actions[bot] added the label cranelift:docs on PR #12990.

Wasmtime GitHub notifications bot (Apr 08 2026 at 20:00):

github-actions[bot] added the label wasmtime:config on PR #12990.

Wasmtime GitHub notifications bot (Apr 08 2026 at 20:00):

github-actions[bot] added the label wasmtime:api on PR #12990.

Wasmtime GitHub notifications bot (Apr 08 2026 at 20:00):

github-actions[bot] added the label cranelift:meta on PR #12990.

Wasmtime GitHub notifications bot (Apr 08 2026 at 20:02):

github-actions[bot] commented on PR #12990:

Label Messager: wasmtime:config

It looks like you are changing Wasmtime's configuration options. Make sure to
complete this check list:

• [ ] If you added a new Config method, you wrote extensive documentation for
  it.

  <details>

  Our documentation should be of the following form:

  ```text
  Short, simple summary sentence.

  More details. These details can be multiple paragraphs. There should be
  information about not just the method, but its parameters and results as
  well.

  Is this method fallible? If so, when can it return an error?

  Can this method panic? If so, when does it panic?

  Example

  Optional example here.
  ```

  </details>

• [ ] If you added a new Config method, or modified an existing one, you
  ensured that this configuration is exercised by the fuzz targets.

  <details>

  For example, if you expose a new strategy for allocating the next instance
  slot inside the pooling allocator, you should ensure that at least one of our
  fuzz targets exercises that new strategy.

  Often, all that is required of you is to ensure that there is a knob for this
  configuration option in [wasmtime_fuzzing::Config][fuzzing-config] (or one
  of its nested structs).

  Rarely, this may require authoring a new fuzz target to specifically test this
  configuration. See [our docs on fuzzing][fuzzing-docs] for more details.

  </details>

• [ ] If you are enabling a configuration option by default, make sure that it
  has been fuzzed for at least two weeks before turning it on by default.

[fuzzing-config]: https://github.com/bytecodealliance/wasmtime/blob/ca0e8d0a1d8cefc0496dba2f77a670571d8fdcab/crates/fuzzing/src/generators.rs#L182-L194
[fuzzing-docs]: https://docs.wasmtime.dev/contributing-fuzzing.html

---

<details>

To modify this label's message, edit the <code>.github/label-messager/wasmtime-config.md</code> file.

To add new label messages or remove existing label messages, edit the
<code>.github/label-messager.json</code> configuration file.

Learn more.

</details>

Wasmtime GitHub notifications bot (Apr 10 2026 at 21:50):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Apr 10 2026 at 21:50):

erikrose edited PR #12990:

This is an implementation of #1749, specifically @cfallin's roadmap, with the goal of reducing the overhead of checking for the end of epochs.

Paul ran some benchmarks on this (broadly agreeing with our real-world experiments) which tell us:

• Current compare-against-a-deadline -Wepoch-interruption=y is a 14.4% hit versus doing nothing.
• Doing only dead loads in function prologues and loop headers (which were all that was implemented in this patch at the time of the bench) brings that down to a 2.8% hit. There will be some additional hit from the signal handler that actually effects the task switch, but that's on the cold path.

The above numbers are from SpiderMonkey, which I deem the most representative benchmark.

Status:

• [X] Add unmapped-on-interrupt page and pointer to it in vmctx.
• [X] Add method for embedder to call to bring an epoch to a close.
• [X] Add DeadLoadWithContext Cranelift instruction, and use it.
• [x] Have DeadLoadWithContext emit metadata into compiled-artifact tables to tell the signal handler this is an interruption-point load.
• [ ] Add logic to signal handler that, when seeing such a PC, updates state to redirect to the stub, saving the original PC (probably in the scratch register).
• [ ] Add that stub, which saves all register state and invokes a hostcall with the recovered vmctx.

If the TLB shootdown arising from the frobbing of privs on the "interrupt page" proves too expensive, we can try a more indirect load instead, where, instead of messing with page privs, we mess with the address we're dead-loading from so it points to either a (statically) allowed or forbidden page. (Chris floated this idea at the 2026-04-08 Cranelift meeting.) Not many of the other mechanics need change.

Wasmtime GitHub notifications bot (Apr 27 2026 at 21:46):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (May 06 2026 at 14:54):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jun 15 2026 at 20:43):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jun 15 2026 at 20:43):

erikrose edited PR #12990:

This is an implementation of #1749, specifically @cfallin's roadmap, with the goal of reducing the overhead of checking for the end of epochs.

Paul ran some benchmarks on this (broadly agreeing with our real-world experiments) which tell us:

• Current compare-against-a-deadline -Wepoch-interruption=y is a 14.4% hit versus doing nothing.
• Doing only dead loads in function prologues and loop headers (which were all that was implemented in this patch at the time of the bench) brings that down to a 2.8% hit. There will be some additional hit from the signal handler that actually effects the task switch, but that's on the cold path.

The above numbers are from SpiderMonkey, which I deem the most representative benchmark.

Status:

• [X] Add unmapped-on-interrupt page and pointer to it in vmctx.
• [X] Add method for embedder to call to bring an epoch to a close.
• [X] Add DeadLoadWithContext Cranelift instruction, and use it.
• [x] Have DeadLoadWithContext emit metadata into compiled-artifact tables to tell the signal handler this is an interruption-point load.
• [x] Add logic to signal handler that, when seeing such a PC, updates state to redirect to the stub, saving the original PC (probably in the scratch register).
• [ ] Add that stub, which saves all register state and invokes a hostcall with the recovered vmctx.

If the TLB shootdown arising from the frobbing of privs on the "interrupt page" proves too expensive, we can try a more indirect load instead, where, instead of messing with page privs, we mess with the address we're dead-loading from so it points to either a (statically) allowed or forbidden page. (Chris floated this idea at the 2026-04-08 Cranelift meeting.) Not many of the other mechanics need change.

Wasmtime GitHub notifications bot (Jun 16 2026 at 15:02):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jun 16 2026 at 16:31):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jun 18 2026 at 15:07):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jun 23 2026 at 16:40):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jun 23 2026 at 20:35):

erikrose edited PR #12990:

This is an implementation of #1749, specifically @cfallin's roadmap, with the goal of reducing the overhead of checking for the end of epochs.

Paul ran some benchmarks on this (broadly agreeing with our real-world experiments) which tell us:

• Current compare-against-a-deadline -Wepoch-interruption=y is a 14.4% hit versus doing nothing.
• Doing only dead loads in function prologues and loop headers (which were all that was implemented in this patch at the time of the bench) brings that down to a 2.8% hit. There will be some additional hit from the signal handler that actually effects the task switch, but that's on the cold path.

The above numbers are from SpiderMonkey, which I deem the most representative benchmark.

Status:

• [X] Add unmapped-on-interrupt page and pointer to it in vmctx.
• [X] Add method for embedder to call to bring an epoch to a close.
• [X] Add DeadLoadWithContext Cranelift instruction, and use it.
• [x] Have DeadLoadWithContext emit metadata into compiled-artifact tables to tell the signal handler this is an interruption-point load.
• [x] Add logic to signal handler that, when seeing such a PC, updates state to redirect to the stub, saving the original PC (probably in the scratch register).
• [x] Add that stub, which saves all register state and invokes a hostcall with the recovered vmctx.

If the TLB shootdown arising from the frobbing of privs on the "interrupt page" proves too expensive, we can try a more indirect load instead, where, instead of messing with page privs, we mess with the address we're dead-loading from so it points to either a (statically) allowed or forbidden page. (Chris floated this idea at the 2026-04-08 Cranelift meeting.) Not many of the other mechanics need change.

Wasmtime GitHub notifications bot (Jun 24 2026 at 14:55):

erikrose edited PR #12990:

This is an implementation of #1749, specifically @cfallin's roadmap, with the goal of reducing the overhead of checking for the end of epochs.

Paul ran some benchmarks on this (broadly agreeing with our real-world experiments) which tell us:

• Current compare-against-a-deadline -Wepoch-interruption=y is a 14.4% hit versus doing nothing.
• Doing only dead loads in function prologues and loop headers (which were all that was implemented in this patch at the time of the bench) brings that down to a 2.8% hit. There will be some additional hit from the signal handler that actually effects the task switch, but that's on the cold path.

The above numbers are from SpiderMonkey, which I deem the most representative benchmark.

Status:

• [X] Add unmapped-on-interrupt page and pointer to it in vmctx.
• [X] Add method for embedder to call to bring an epoch to a close.
• [X] Add DeadLoadWithContext Cranelift instruction, and use it.
• [x] Have DeadLoadWithContext emit metadata into compiled-artifact tables to tell the signal handler this is an interruption-point load.
• [x] Add logic to signal handler that, when seeing such a PC, updates state to redirect to the stub, saving the original PC (probably in the scratch register).
• [x] Add that stub, which saves all register state and invokes a hostcall with the recovered vmctx.
• [ ] Make sure nothing happens when --epoch-interruption-via-mmu is off.
• [ ] Make wasmtime run do MMU interrupts at the frequency of WasmOptions::timeout, just as currently happens with classic epoch-interruption.
• [ ] Write about SAFETY everywhere I do unsafe things.

If the TLB shootdown arising from the frobbing of privs on the "interrupt page" proves too expensive, we can try a more indirect load instead, where, instead of messing with page privs, we mess with the address we're dead-loading from so it points to either a (statically) allowed or forbidden page. (Chris floated this idea at the 2026-04-08 Cranelift meeting.) Not many of the other mechanics need change.

Wasmtime GitHub notifications bot (Jun 24 2026 at 16:56):

erikrose edited PR #12990:

This is an implementation of #1749, specifically @cfallin's roadmap, with the goal of reducing the overhead of checking for the end of epochs.

Paul ran some benchmarks on this (broadly agreeing with our real-world experiments) which tell us:

• Current compare-against-a-deadline -Wepoch-interruption=y is a 14.4% hit versus doing nothing.
• Doing only dead loads in function prologues and loop headers (which were all that was implemented in this patch at the time of the bench) brings that down to a 2.8% hit. There will be some additional hit from the signal handler that actually effects the task switch, but that's on the cold path.

The above numbers are from SpiderMonkey, which I deem the most representative benchmark.

Status:

• [X] Add unmapped-on-interrupt page and pointer to it in vmctx.
• [X] Add method for embedder to call to bring an epoch to a close.
• [X] Add DeadLoadWithContext Cranelift instruction, and use it.
• [x] Have DeadLoadWithContext emit metadata into compiled-artifact tables to tell the signal handler this is an interruption-point load.
• [x] Add logic to signal handler that, when seeing such a PC, updates state to redirect to the stub, saving the original PC (probably in the scratch register).
• [x] Add that stub, which saves all register state and invokes a hostcall with the recovered vmctx.
• [ ] Make sure we can't overflow the stack undetected.
• [ ] Make sure nothing happens when --epoch-interruption-via-mmu is off.
• [ ] Make wasmtime run do MMU interrupts at the frequency of WasmOptions::timeout, just as currently happens with classic epoch-interruption.
• [ ] Write about SAFETY everywhere I do unsafe things.

If the TLB shootdown arising from the frobbing of privs on the "interrupt page" proves too expensive, we can try a more indirect load instead, where, instead of messing with page privs, we mess with the address we're dead-loading from so it points to either a (statically) allowed or forbidden page. (Chris floated this idea at the 2026-04-08 Cranelift meeting.) Not many of the other mechanics need change.

Wasmtime GitHub notifications bot (Jun 24 2026 at 20:32):

erikrose edited PR #12990:

This is an implementation of #1749, specifically @cfallin's roadmap, with the goal of reducing the overhead of checking for the end of epochs.

Paul ran some benchmarks on this (broadly agreeing with our real-world experiments) which tell us:

• Current compare-against-a-deadline -Wepoch-interruption=y is a 14.4% hit versus doing nothing.
• Doing only dead loads in function prologues and loop headers (which were all that was implemented in this patch at the time of the bench) brings that down to a 2.8% hit. There will be some additional hit from the signal handler that actually effects the task switch, but that's on the cold path.

The above numbers are from SpiderMonkey, which I deem the most representative benchmark.

Status:

• [X] Add unmapped-on-interrupt page and pointer to it in vmctx.
• [X] Add method for embedder to call to bring an epoch to a close.
• [X] Add DeadLoadWithContext Cranelift instruction, and use it.
• [x] Have DeadLoadWithContext emit metadata into compiled-artifact tables to tell the signal handler this is an interruption-point load.
• [x] Add logic to signal handler that, when seeing such a PC, updates state to redirect to the stub, saving the original PC (probably in the scratch register).
• [x] Add that stub, which saves all register state and invokes a hostcall with the recovered vmctx.

Polishing tasks:

• [ ] Make sure we can't overflow the stack undetected.
• [ ] Make sure nothing happens when --epoch-interruption-via-mmu is off.
• [ ] Make wasmtime run do MMU interrupts at the frequency of WasmOptions::timeout, just as currently happens with classic epoch-interruption.
• [ ] Write about SAFETY everywhere I do unsafe things.

If the TLB shootdown arising from the frobbing of privs on the "interrupt page" proves too expensive, we can try a more indirect load instead, where, instead of messing with page privs, we mess with the address we're dead-loading from so it points to either a (statically) allowed or forbidden page. (Chris floated this idea at the 2026-04-08 Cranelift meeting.) Not many of the other mechanics need change.

Wasmtime GitHub notifications bot (Jun 24 2026 at 20:41):

erikrose edited PR #12990:

This is an implementation of #1749, specifically @cfallin's roadmap, with the goal of reducing the overhead of checking for the end of epochs.

Paul ran some benchmarks on this (broadly agreeing with our real-world experiments) which tell us:

• Current compare-against-a-deadline -Wepoch-interruption=y is a 14.4% hit versus doing nothing.
• Doing only dead loads in function prologues and loop headers (which were all that was implemented in this patch at the time of the bench) brings that down to a 2.8% hit. There will be some additional hit from the signal handler that actually effects the task switch, but that's on the cold path.

The above numbers are from SpiderMonkey, which I deem the most representative benchmark.

Status:

• [X] Add unmapped-on-interrupt page and pointer to it in vmctx.
• [X] Add method for embedder to call to bring an epoch to a close.
• [X] Add DeadLoadWithContext Cranelift instruction, and use it.
• [x] Have DeadLoadWithContext emit metadata into compiled-artifact tables to tell the signal handler this is an interruption-point load.
• [x] Add logic to signal handler that, when seeing such a PC, updates state to redirect to the stub, saving the original PC (probably in the scratch register).
• [x] Add that stub, which saves all register state and invokes a hostcall with the recovered vmctx.

Polishing tasks:

• [ ] Make sure we can't overflow the stack undetected.
• [ ] Make sure nothing happens when --epoch-interruption-via-mmu is off.
• [ ] Make wasmtime run do MMU interrupts after WasmOptions::timeout, just as currently happens with classic epoch-interruption.
• [ ] Write about SAFETY everywhere I do unsafe things.

If the TLB shootdown arising from the frobbing of privs on the "interrupt page" proves too expensive, we can try a more indirect load instead, where, instead of messing with page privs, we mess with the address we're dead-loading from so it points to either a (statically) allowed or forbidden page. (Chris floated this idea at the 2026-04-08 Cranelift meeting.) Not many of the other mechanics need change.

Wasmtime GitHub notifications bot (Jun 24 2026 at 21:54):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jul 06 2026 at 16:50):

erikrose edited PR #12990:

This is an implementation of #1749, specifically @cfallin's roadmap, with the goal of reducing the overhead of checking for the end of epochs.

Paul ran some benchmarks on this (broadly agreeing with our real-world experiments) which tell us:

• Current compare-against-a-deadline -Wepoch-interruption=y is a 14.4% hit versus doing nothing.
• Doing only dead loads in function prologues and loop headers (which were all that was implemented in this patch at the time of the bench) brings that down to a 2.8% hit. There will be some additional hit from the signal handler that actually effects the task switch, but that's on the cold path.

The above numbers are from SpiderMonkey, which I deem the most representative benchmark.

Status:

• [X] Add unmapped-on-interrupt page and pointer to it in vmctx.
• [X] Add method for embedder to call to bring an epoch to a close.
• [X] Add DeadLoadWithContext Cranelift instruction, and use it.
• [x] Have DeadLoadWithContext emit metadata into compiled-artifact tables to tell the signal handler this is an interruption-point load.
• [x] Add logic to signal handler that, when seeing such a PC, updates state to redirect to the stub, saving the original PC (probably in the scratch register).
• [x] Add that stub, which saves all register state and invokes a hostcall with the recovered vmctx.

Polishing tasks:

• [ ] Make sure we can't overflow the stack undetected.
• [ ] Make sure nothing happens when --epoch-interruption-via-mmu is off.
• [x] Make wasmtime run do MMU interrupts after WasmOptions::timeout, just as currently happens with classic epoch-interruption.
• [ ] Write about SAFETY everywhere I do unsafe things.

If the TLB shootdown arising from the frobbing of privs on the "interrupt page" proves too expensive, we can try a more indirect load instead, where, instead of messing with page privs, we mess with the address we're dead-loading from so it points to either a (statically) allowed or forbidden page. (Chris floated this idea at the 2026-04-08 Cranelift meeting.) Not many of the other mechanics need change.

Wasmtime GitHub notifications bot (Jul 06 2026 at 21:39):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jul 07 2026 at 14:14):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jul 07 2026 at 17:24):

erikrose edited PR #12990:

This is an implementation of #1749, specifically @cfallin's roadmap, with the goal of reducing the overhead of checking for the end of epochs.

Paul ran some benchmarks on this (broadly agreeing with our real-world experiments) which tell us:

• Current compare-against-a-deadline -Wepoch-interruption=y is a 14.4% hit versus doing nothing.
• Doing only dead loads in function prologues and loop headers (which were all that was implemented in this patch at the time of the bench) brings that down to a 2.8% hit. There will be some additional hit from the signal handler that actually effects the task switch, but that's on the cold path.

The above numbers are from SpiderMonkey, which I deem the most representative benchmark.

Status:

• [X] Add unmapped-on-interrupt page and pointer to it in vmctx.
• [X] Add method for embedder to call to bring an epoch to a close.
• [X] Add DeadLoadWithContext Cranelift instruction, and use it.
• [x] Have DeadLoadWithContext emit metadata into compiled-artifact tables to tell the signal handler this is an interruption-point load.
• [x] Add logic to signal handler that, when seeing such a PC, updates state to redirect to the stub, saving the original PC (probably in the scratch register).
• [x] Add that stub, which saves all register state and invokes a hostcall with the recovered vmctx.

Polishing tasks:

• [ ] Make sure we can't overflow the stack undetected.
• [ ] Make sure nothing happens when --epoch-interruption-via-mmu is off.
• [x] Make wasmtime run do MMU interrupts after WasmOptions::timeout, just as currently happens with classic epoch-interruption.
• [ ] Write about SAFETY everywhere I do unsafe things.
• [ ] Make --mmu-interruption report an error on unsupported platforms.

If the TLB shootdown arising from the frobbing of privs on the "interrupt page" proves too expensive, we can try a more indirect load instead, where, instead of messing with page privs, we mess with the address we're dead-loading from so it points to either a (statically) allowed or forbidden page. (Chris floated this idea at the 2026-04-08 Cranelift meeting.) Not many of the other mechanics need change.

Wasmtime GitHub notifications bot (Jul 07 2026 at 17:24):

erikrose edited PR #12990:

This is an implementation of #1749, specifically @cfallin's roadmap, with the goal of reducing the overhead of checking for the end of epochs.

Paul ran some benchmarks on this (broadly agreeing with our real-world experiments) which tell us:

• Current compare-against-a-deadline -Wepoch-interruption=y is a 14.4% hit versus doing nothing.
• Doing only dead loads in function prologues and loop headers (which were all that was implemented in this patch at the time of the bench) brings that down to a 2.8% hit. There will be some additional hit from the signal handler that actually effects the task switch, but that's on the cold path.

The above numbers are from SpiderMonkey, which I deem the most representative benchmark.

Status:

• [X] Add unmapped-on-interrupt page and pointer to it in vmctx.
• [X] Add method for embedder to call to bring an epoch to a close.
• [X] Add DeadLoadWithContext Cranelift instruction, and use it.
• [x] Have DeadLoadWithContext emit metadata into compiled-artifact tables to tell the signal handler this is an interruption-point load.
• [x] Add logic to signal handler that, when seeing such a PC, updates state to redirect to the stub, saving the original PC (probably in the scratch register).
• [x] Add that stub, which saves all register state and invokes a hostcall with the recovered vmctx.

Polishing tasks:

• [ ] Make sure we can't overflow the stack undetected.
• [ ] Make sure nothing happens when --epoch-interruption-via-mmu is off.
• [x] Make wasmtime run do MMU interrupts after WasmOptions::timeout, just as currently happens with classic epoch-interruption.
• [ ] Write about SAFETY everywhere I do unsafe things.
• [ ] Make --epoch-interruption-via-mmu report an error on unsupported platforms.

If the TLB shootdown arising from the frobbing of privs on the "interrupt page" proves too expensive, we can try a more indirect load instead, where, instead of messing with page privs, we mess with the address we're dead-loading from so it points to either a (statically) allowed or forbidden page. (Chris floated this idea at the 2026-04-08 Cranelift meeting.) Not many of the other mechanics need change.

Wasmtime GitHub notifications bot (Jul 07 2026 at 17:36):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jul 07 2026 at 18:41):

erikrose edited PR #12990:

This is an implementation of #1749, specifically @cfallin's roadmap, with the goal of reducing the overhead of checking for the end of epochs.

Paul ran some benchmarks on this (broadly agreeing with our real-world experiments) which tell us:

• Current compare-against-a-deadline -Wepoch-interruption=y is a 14.4% hit versus doing nothing.
• Doing only dead loads in function prologues and loop headers (which were all that was implemented in this patch at the time of the bench) brings that down to a 2.8% hit. There will be some additional hit from the signal handler that actually effects the task switch, but that's on the cold path.

The above numbers are from SpiderMonkey, which I deem the most representative benchmark.

Status:

• [X] Add unmapped-on-interrupt page and pointer to it in vmctx.
• [X] Add method for embedder to call to bring an epoch to a close.
• [X] Add DeadLoadWithContext Cranelift instruction, and use it.
• [x] Have DeadLoadWithContext emit metadata into compiled-artifact tables to tell the signal handler this is an interruption-point load.
• [x] Add logic to signal handler that, when seeing such a PC, updates state to redirect to the stub, saving the original PC (probably in the scratch register).
• [x] Add that stub, which saves all register state and invokes a hostcall with the recovered vmctx.

Polishing tasks:

• [ ] Make sure we can't overflow the stack undetected.
• [ ] Make sure nothing happens when --epoch-interruption-via-mmu is off.
• [x] Make wasmtime run do MMU interrupts after WasmOptions::timeout, just as currently happens with classic epoch-interruption.
• [x] Test 2 tasks interrupting each other several times to make sure interruption works more than once.
• [ ] Write about SAFETY everywhere I do unsafe things.
• [ ] Make --epoch-interruption-via-mmu report an error on unsupported platforms.

If the TLB shootdown arising from the frobbing of privs on the "interrupt page" proves too expensive, we can try a more indirect load instead, where, instead of messing with page privs, we mess with the address we're dead-loading from so it points to either a (statically) allowed or forbidden page. (Chris floated this idea at the 2026-04-08 Cranelift meeting.) Not many of the other mechanics need change.

Wasmtime GitHub notifications bot (Jul 07 2026 at 19:14):

erikrose edited PR #12990:

This is an implementation of #1749, specifically @cfallin's roadmap, with the goal of reducing the overhead of checking for the end of epochs.

Paul ran some benchmarks on this (broadly agreeing with our real-world experiments) which tell us:

• Current compare-against-a-deadline -Wepoch-interruption=y is a 14.4% hit versus doing nothing.
• Doing only dead loads in function prologues and loop headers (which were all that was implemented in this patch at the time of the bench) brings that down to a 2.8% hit. There will be some additional hit from the signal handler that actually effects the task switch, but that's on the cold path.

The above numbers are from SpiderMonkey, which I deem the most representative benchmark.

Status:

• [X] Add unmapped-on-interrupt page and pointer to it in vmctx.
• [X] Add method for embedder to call to bring an epoch to a close.
• [X] Add DeadLoadWithContext Cranelift instruction, and use it.
• [x] Have DeadLoadWithContext emit metadata into compiled-artifact tables to tell the signal handler this is an interruption-point load.
• [x] Add logic to signal handler that, when seeing such a PC, updates state to redirect to the stub, saving the original PC (probably in the scratch register).
• [x] Add that stub, which saves all register state and invokes a hostcall with the recovered vmctx.

Polishing tasks:

• [ ] Make sure we can't overflow the stack undetected.
• [x] Make sure nothing happens when --epoch-interruption-via-mmu is off.
• [x] Make wasmtime run do MMU interrupts after WasmOptions::timeout, just as currently happens with classic epoch-interruption.
• [x] Test 2 tasks interrupting each other several times to make sure interruption works more than once.
• [ ] Write about SAFETY everywhere I do unsafe things.
• [ ] Make --epoch-interruption-via-mmu report an error on unsupported platforms.

If the TLB shootdown arising from the frobbing of privs on the "interrupt page" proves too expensive, we can try a more indirect load instead, where, instead of messing with page privs, we mess with the address we're dead-loading from so it points to either a (statically) allowed or forbidden page. (Chris floated this idea at the 2026-04-08 Cranelift meeting.) Not many of the other mechanics need change.

Wasmtime GitHub notifications bot (Jul 07 2026 at 21:52):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jul 07 2026 at 21:52):

erikrose edited PR #12990:

This is an implementation of #1749, specifically @cfallin's roadmap, with the goal of reducing the overhead of checking for the end of epochs.

Paul ran some benchmarks on this (broadly agreeing with our real-world experiments) which tell us:

• Current compare-against-a-deadline -Wepoch-interruption=y is a 14.4% hit versus doing nothing.
• Doing only dead loads in function prologues and loop headers (which were all that was implemented in this patch at the time of the bench) brings that down to a 2.8% hit. There will be some additional hit from the signal handler that actually effects the task switch, but that's on the cold path.

The above numbers are from SpiderMonkey, which I deem the most representative benchmark.

Status:

• [X] Add unmapped-on-interrupt page and pointer to it in vmctx.
• [X] Add method for embedder to call to bring an epoch to a close.
• [X] Add DeadLoadWithContext Cranelift instruction, and use it.
• [x] Have DeadLoadWithContext emit metadata into compiled-artifact tables to tell the signal handler this is an interruption-point load.
• [x] Add logic to signal handler that, when seeing such a PC, updates state to redirect to the stub, saving the original PC (probably in the scratch register).
• [x] Add that stub, which saves all register state and invokes a hostcall with the recovered vmctx.

Polishing tasks:

• [ ] Make sure we can't overflow the stack undetected.
• [x] Make sure nothing happens when --epoch-interruption-via-mmu is off.
• [x] Make wasmtime run do MMU interrupts after WasmOptions::timeout, just as currently happens with classic epoch-interruption.
• [x] Test 2 tasks interrupting each other several times to make sure interruption works more than once.
• [x] Write about SAFETY everywhere I do unsafe things.
• [ ] Make --epoch-interruption-via-mmu report an error on unsupported platforms.

If the TLB shootdown arising from the frobbing of privs on the "interrupt page" proves too expensive, we can try a more indirect load instead, where, instead of messing with page privs, we mess with the address we're dead-loading from so it points to either a (statically) allowed or forbidden page. (Chris floated this idea at the 2026-04-08 Cranelift meeting.) Not many of the other mechanics need change.

Wasmtime GitHub notifications bot (Jul 07 2026 at 23:54):

erikrose edited PR #12990:

This is an implementation of #1749, specifically @cfallin's roadmap, with the goal of reducing the overhead of checking for the end of epochs.

Paul ran some benchmarks on this (broadly agreeing with our real-world experiments) which tell us:

• Current compare-against-a-deadline -Wepoch-interruption=y is a 14.4% hit versus doing nothing.
• Doing only dead loads in function prologues and loop headers (which were all that was implemented in this patch at the time of the bench) brings that down to a 2.8% hit. There will be some additional hit from the signal handler that actually effects the task switch, but that's on the cold path.

The above numbers are from SpiderMonkey, which I deem the most representative benchmark.

Status:

• [X] Add unmapped-on-interrupt page and pointer to it in vmctx.
• [X] Add method for embedder to call to bring an epoch to a close.
• [X] Add DeadLoadWithContext Cranelift instruction, and use it.
• [x] Have DeadLoadWithContext emit metadata into compiled-artifact tables to tell the signal handler this is an interruption-point load.
• [x] Add logic to signal handler that, when seeing such a PC, updates state to redirect to the stub, saving the original PC (probably in the scratch register).
• [x] Add that stub, which saves all register state and invokes a hostcall with the recovered vmctx.

Polishing tasks:

• [ ] Make sure we can't overflow the stack undetected.
• [ ] Call CLI flag (and other mentions of the term in code) "MMU interrupts" rather than "epoch interruption via MMU".
• [x] Make sure nothing happens when --epoch-interruption-via-mmu is off.
• [x] Make wasmtime run do MMU interrupts after WasmOptions::timeout, just as currently happens with classic epoch-interruption.
• [x] Test 2 tasks interrupting each other several times to make sure interruption works more than once.
• [x] Write about SAFETY everywhere I do unsafe things.
• [ ] Make --epoch-interruption-via-mmu report an error on unsupported platforms.

If the TLB shootdown arising from the frobbing of privs on the "interrupt page" proves too expensive, we can try a more indirect load instead, where, instead of messing with page privs, we mess with the address we're dead-loading from so it points to either a (statically) allowed or forbidden page. (Chris floated this idea at the 2026-04-08 Cranelift meeting.) Not many of the other mechanics need change.

Wasmtime GitHub notifications bot (Jul 08 2026 at 14:16):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jul 08 2026 at 14:41):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jul 08 2026 at 15:26):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jul 08 2026 at 15:27):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jul 08 2026 at 15:28):

erikrose edited PR #12990:

This is an implementation of #1749, specifically @cfallin's roadmap, with the goal of reducing the overhead of checking for the end of epochs.

Paul ran some benchmarks on this (broadly agreeing with our real-world experiments) which tell us:

• Current compare-against-a-deadline -Wepoch-interruption=y is a 14.4% hit versus doing nothing.
• Doing only dead loads in function prologues and loop headers (which were all that was implemented in this patch at the time of the bench) brings that down to a 2.8% hit. There will be some additional hit from the signal handler that actually effects the task switch, but that's on the cold path.

The above numbers are from SpiderMonkey, which I deem the most representative benchmark.

Status:

• [X] Add unmapped-on-interrupt page and pointer to it in vmctx.
• [X] Add method for embedder to call to bring an epoch to a close.
• [X] Add DeadLoadWithContext Cranelift instruction, and use it.
• [x] Have DeadLoadWithContext emit metadata into compiled-artifact tables to tell the signal handler this is an interruption-point load.
• [x] Add logic to signal handler that, when seeing such a PC, updates state to redirect to the stub, saving the original PC (probably in the scratch register).
• [x] Add that stub, which saves all register state and invokes a hostcall with the recovered vmctx.

Polishing tasks:

• [ ] Make sure we can't overflow the stack undetected.
• [ ] Call CLI flag (and other mentions of the term in code) "MMU interrupts" rather than "epoch interruption via MMU".
• [x] Make sure nothing happens when --epoch-interruption-via-mmu is off.
• [x] Make wasmtime run do MMU interrupts after WasmOptions::timeout, just as currently happens with classic epoch-interruption.
• [x] Test 2 tasks interrupting each other several times to make sure interruption works more than once.
• [x] Write about SAFETY everywhere I do unsafe things.
• [x] Make --epoch-interruption-via-mmu report an error on unsupported platforms.

If the TLB shootdown arising from the frobbing of privs on the "interrupt page" proves too expensive, we can try a more indirect load instead, where, instead of messing with page privs, we mess with the address we're dead-loading from so it points to either a (statically) allowed or forbidden page. (Chris floated this idea at the 2026-04-08 Cranelift meeting.) Not many of the other mechanics need change.

Wasmtime GitHub notifications bot (Jul 08 2026 at 16:09):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jul 08 2026 at 16:09):

erikrose edited PR #12990:

This is an implementation of #1749, specifically @cfallin's roadmap, with the goal of reducing the overhead of checking for the end of epochs.

Paul ran some benchmarks on this (broadly agreeing with our real-world experiments) which tell us:

• Current compare-against-a-deadline -Wepoch-interruption=y is a 14.4% hit versus doing nothing.
• Doing only dead loads in function prologues and loop headers (which were all that was implemented in this patch at the time of the bench) brings that down to a 2.8% hit. There will be some additional hit from the signal handler that actually effects the task switch, but that's on the cold path.

The above numbers are from SpiderMonkey, which I deem the most representative benchmark.

Status:

• [X] Add unmapped-on-interrupt page and pointer to it in vmctx.
• [X] Add method for embedder to call to bring an epoch to a close.
• [X] Add DeadLoadWithContext Cranelift instruction, and use it.
• [x] Have DeadLoadWithContext emit metadata into compiled-artifact tables to tell the signal handler this is an interruption-point load.
• [x] Add logic to signal handler that, when seeing such a PC, updates state to redirect to the stub, saving the original PC (probably in the scratch register).
• [x] Add that stub, which saves all register state and invokes a hostcall with the recovered vmctx.

Polishing tasks:

• [x] Make sure we can't overflow the stack undetected.
• [ ] Call CLI flag (and other mentions of the term in code) "MMU interrupts" rather than "epoch interruption via MMU".
• [x] Make sure nothing happens when --epoch-interruption-via-mmu is off.
• [x] Make wasmtime run do MMU interrupts after WasmOptions::timeout, just as currently happens with classic epoch-interruption.
• [x] Test 2 tasks interrupting each other several times to make sure interruption works more than once.
• [x] Write about SAFETY everywhere I do unsafe things.
• [x] Make --epoch-interruption-via-mmu report an error on unsupported platforms.

If the TLB shootdown arising from the frobbing of privs on the "interrupt page" proves too expensive, we can try a more indirect load instead, where, instead of messing with page privs, we mess with the address we're dead-loading from so it points to either a (statically) allowed or forbidden page. (Chris floated this idea at the 2026-04-08 Cranelift meeting.) Not many of the other mechanics need change.

Wasmtime GitHub notifications bot (Jul 08 2026 at 20:13):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jul 10 2026 at 15:45):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jul 14 2026 at 03:55):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jul 14 2026 at 03:58):

erikrose edited PR #12990:

This is an implementation of #1749, specifically @cfallin's roadmap, with the goal of reducing the overhead of checking for the end of epochs.

Paul ran some benchmarks on this (broadly agreeing with our real-world experiments) which tell us:

• Current compare-against-a-deadline -Wepoch-interruption=y is a 14.4% hit versus doing nothing.
• Doing only dead loads in function prologues and loop headers (which were all that was implemented in this patch at the time of the bench) brings that down to a 2.8% hit. There will be some additional hit from the signal handler that actually effects the task switch, but that's on the cold path.

The above numbers are from SpiderMonkey, which I deem the most representative benchmark.

Status:

• [X] Add unmapped-on-interrupt page and pointer to it in vmctx.
• [X] Add method for embedder to call to bring an epoch to a close.
• [X] Add DeadLoadWithContext Cranelift instruction, and use it.
• [x] Have DeadLoadWithContext emit metadata into compiled-artifact tables to tell the signal handler this is an interruption-point load.
• [x] Add logic to signal handler that, when seeing such a PC, updates state to redirect to the stub, saving the original PC (probably in the scratch register).
• [x] Add that stub, which saves all register state and invokes a hostcall with the recovered vmctx.

Polishing tasks:

• [x] Make sure we can't overflow the stack undetected.
• [x] Call CLI flag (and other mentions of the term in code) "MMU interrupts" rather than "epoch interruption via MMU".
• [x] Make sure nothing happens when --epoch-interruption-via-mmu is off.
• [x] Make wasmtime run do MMU interrupts after WasmOptions::timeout, just as currently happens with classic epoch-interruption.
• [x] Test 2 tasks interrupting each other several times to make sure interruption works more than once.
• [x] Write about SAFETY everywhere I do unsafe things.
• [x] Make --epoch-interruption-via-mmu report an error on unsupported platforms.

If the TLB shootdown arising from the frobbing of privs on the "interrupt page" proves too expensive, we can try a more indirect load instead, where, instead of messing with page privs, we mess with the address we're dead-loading from so it points to either a (statically) allowed or forbidden page. (Chris floated this idea at the 2026-04-08 Cranelift meeting.) Not many of the other mechanics need change.

Wasmtime GitHub notifications bot (Jul 14 2026 at 04:00):

erikrose edited PR #12990:

This is an implementation of #1749, specifically @cfallin's roadmap, with the goal of reducing the overhead of checking for the end of epochs.

Paul ran some benchmarks on this (broadly agreeing with our real-world experiments) which tell us:

• Current compare-against-a-deadline -Wepoch-interruption=y is a 14.4% hit versus doing nothing.
• Doing only dead loads in function prologues and loop headers (which were all that was implemented in this patch at the time of the bench) brings that down to a 2.8% hit. There will be some additional hit from the signal handler that actually effects the task switch, but that's on the cold path.

The above numbers are from SpiderMonkey, which I deem the most representative benchmark.

Status:

• [X] Add unmapped-on-interrupt page and pointer to it in vmctx.
• [X] Add method for embedder to call to bring an epoch to a close.
• [X] Add DeadLoadWithContext Cranelift instruction, and use it.
• [x] Have DeadLoadWithContext emit metadata into compiled-artifact tables to tell the signal handler this is an interruption-point load.
• [x] Add logic to signal handler that, when seeing such a PC, updates state to redirect to the stub, saving the original PC (probably in the scratch register).
• [x] Add that stub, which saves all register state and invokes a hostcall with the recovered vmctx.

Polishing tasks:

• [x] Make sure we can't overflow the stack undetected.
• [x] Call CLI flag (and other mentions of the term in code) "MMU interrupts" rather than "epoch interruption via MMU".
• [x] Make sure nothing happens when --epoch-interruption-via-mmu is off.
• [x] Make wasmtime run do MMU interrupts after WasmOptions::timeout, just as currently happens with classic epoch-interruption.
• [x] Test 2 tasks interrupting each other several times to make sure interruption works more than once.
• [x] Write about SAFETY everywhere I do unsafe things.
• [x] Make --epoch-interruption-via-mmu report an error on unsupported platforms.
• [ ] Make sure Config-option docs and fuzzing are up to snuff.

If the TLB shootdown arising from the frobbing of privs on the "interrupt page" proves too expensive, we can try a more indirect load instead, where, instead of messing with page privs, we mess with the address we're dead-loading from so it points to either a (statically) allowed or forbidden page. (Chris floated this idea at the 2026-04-08 Cranelift meeting.) Not many of the other mechanics need change.

Wasmtime GitHub notifications bot (Jul 15 2026 at 18:32):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jul 15 2026 at 20:19):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jul 15 2026 at 23:28):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jul 15 2026 at 23:44):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jul 16 2026 at 16:46):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jul 16 2026 at 17:57):

erikrose has marked PR #12990 as ready for review.

Wasmtime GitHub notifications bot (Jul 16 2026 at 17:57):

erikrose requested alexcrichton for a review on PR #12990.

Wasmtime GitHub notifications bot (Jul 16 2026 at 17:57):

erikrose requested wasmtime-compiler-reviewers for a review on PR #12990.

Wasmtime GitHub notifications bot (Jul 16 2026 at 17:57):

erikrose requested wasmtime-wasi-reviewers for a review on PR #12990.

Wasmtime GitHub notifications bot (Jul 16 2026 at 17:57):

erikrose requested wasmtime-core-reviewers for a review on PR #12990.

Wasmtime GitHub notifications bot (Jul 16 2026 at 17:57):

erikrose requested wasmtime-default-reviewers for a review on PR #12990.

Wasmtime GitHub notifications bot (Jul 16 2026 at 22:05):

alexcrichton commented on PR #12990:

@erikrose do you feel this is landable as-is modulo review? I see the review request now and wanted to confirm. If so, before going too deep into this, have you done performance testing in the contexts of where this is expected to provide a benefit?

Wasmtime GitHub notifications bot (Jul 17 2026 at 00:27):

erikrose commented on PR #12990:

Yes. I'm about to commit a more in-depth comment on Config::mmu_interruption() as the bot above requested (perhaps post-dinner), and I'll also look into fuzzing, but I think it's landable as it stands.

The perf testing is interesting: it should bench out the same as when we did it before on the no-interrupt path (14.4% overhead for epochs, 2.8% for this). The yes-interrupt path is probably going to depend on how fully loaded the machine is: things like TLB shoot-downs interacting with multiple cores, all furiously running Wasm guest code. My thinking was to get it landed upstream, update the version of wasmtime we're using, and then try the new flag in some canary contexts to get real-world numbers.

Of course, I'm happy to do Sightglass benchmarks of the yes-interrupt path at various intervals (every .1ms, 1ms, 10ms) and see how those compare, though it'll require some coding. Sightglass is better than nothing, and I was probably going to do that anyway just in case huge surprises come out of it. If you want to gate landing on that, it's fine with me. But I don't think it needs to hold up review.

I probably should have led with this, but we're having Saúl help us out for a bit, and he's going to take a look over the next day or two before he becomes unavailable for a span. @saulecabrera Maybe you want to claim review? I'll let you guys fight it out. Thanks!

Wasmtime GitHub notifications bot (Jul 17 2026 at 00:41):

cfallin commented on PR #12990:

I'm happy to look at the Cranelift side of this. (And Alex should definitely look at the runtime side...)

On first skim, I think the overall shape of the "dead load" seems fine, but a few style and design comments:

• It'd be great not to refer to bits of Wasmtime in the Cranelift instruction definition. Cranelift should stand on its own; dead_load_with_context can have a description that maybe says something like "Useful when implementing virtual memory-triggered interrupts, with context in a fixed register available to the interrupt handler."

• I also see a TODO and a question in the description; we should resolve these.

• The register choices are more or less a function of ABI, so let's (i) assert that the instruction occurs in a function with a signature that has an ABI we control (let's say just tail) and then in the doc-comment on the Tail arm of the CallConv enum, alongside the comments defining which registers are used for exception payload, let's add a note about registers used for dead-load-with-context payload.

• I really don't like the yet-another-kind-of-metadata on MachBuffer (mmu_interrupt) -- again it's a weird Wasmtime-specific thing, and adds complexity and maintenance burden. Is there a reason we can't attach a trap code to the load, like we do other loads, and distinguish this mechanism based on that? That also avoids the need to add a whole new metadata section to the compiled artifact.

• At least from my point of view, I think we should get an implementation on another architecture (let's say aarch64) as well; this shows generality, and we have quite a few aarch64 users too. I think we should also probably have perf results showing that the interruption behavior is not catastrophic (due to TLB shootdowns as you mention): the result of that experiment could change the mechanism (use an indirect approach instead), and/or could indicate that the whole thing is not worthwhile.

Wasmtime GitHub notifications bot (Jul 17 2026 at 00:42):

cfallin edited a comment on PR #12990:

I'm happy to look at the Cranelift side of this. (And Alex should definitely look at the runtime side...)

On first skim, I think the overall shape of the "dead load" seems fine, but a few style and design comments:

• It'd be great not to refer to bits of Wasmtime in the Cranelift instruction definition. Cranelift should stand on its own; dead_load_with_context can have a description that maybe says something like "Useful when implementing virtual memory-triggered interrupts, with context in a fixed register available to the interrupt handler."

• I also see a TODO and a question in the description; we should resolve these.

• The register choices are more or less a function of ABI, so let's (i) assert that the instruction occurs in a function with a signature that has an ABI we control (let's say just tail) and then (ii) in the doc-comment on the Tail arm of the CallConv enum, alongside the comments defining which registers are used for exception payload, let's add a note about registers used for dead-load-with-context payload.

• I really don't like the yet-another-kind-of-metadata on MachBuffer (mmu_interrupt) -- again it's a weird Wasmtime-specific thing, and adds complexity and maintenance burden. Is there a reason we can't attach a trap code to the load, like we do other loads, and distinguish this mechanism based on that? That also avoids the need to add a whole new metadata section to the compiled artifact.

• At least from my point of view, I think we should get an implementation on another architecture (let's say aarch64) as well; this shows generality, and we have quite a few aarch64 users too. I think we should also probably have perf results showing that the interruption behavior is not catastrophic (due to TLB shootdowns as you mention): the result of that experiment could change the mechanism (use an indirect approach instead), and/or could indicate that the whole thing is not worthwhile.

Wasmtime GitHub notifications bot (Jul 17 2026 at 00:43):

cfallin edited a comment on PR #12990:

I'm happy to look at the Cranelift side of this. (And Alex should definitely look at the runtime side...)

On first skim, I think the overall shape of the "dead load" seems fine, but a few style and design comments:

• It'd be great not to refer to bits of Wasmtime in the Cranelift instruction definition. Cranelift should stand on its own; dead_load_with_context can have a description that maybe says something like "Useful when implementing virtual memory-triggered interrupts, with context in a fixed register available to the interrupt handler."

• I also see a TODO and a question in the description; we should resolve these.

• The register choices are more or less a function of ABI, so let's (i) assert that the instruction occurs in a function with a signature that has an ABI we control (let's say just tail) and then (ii) in the doc-comment on the Tail arm of the CallConv enum, alongside the comments defining which registers are used for exception payload, let's add a note about registers used for dead-load-with-context payload.

• I really don't like the yet-another-kind-of-metadata on MachBuffer (mmu_interrupt) -- again it's a weird Wasmtime-specific thing, and adds complexity and maintenance burden. Is there a reason we can't attach a trap code to the load, like we do other loads, and distinguish this mechanism based on that? That also avoids the need to add a whole new metadata section to the compiled artifact.

• At least from my point of view, I think we should get an implementation on another architecture (let's say aarch64) as well; this shows generality, and we have quite a few aarch64 users too. It also shows that the mechanism works on an ISA with another style of call linkage (namely, the link register rather than the stack).

  I think we should also probably have perf results showing that the interruption behavior is not catastrophic (due to TLB shootdowns as you mention): the result of that experiment could change the mechanism (use an indirect approach instead), and/or could indicate that the whole thing is not worthwhile.

Wasmtime GitHub notifications bot (Jul 17 2026 at 04:17):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Jul 17 2026 at 11:48):

:memo: saulecabrera submitted PR review.

Wasmtime GitHub notifications bot (Jul 17 2026 at 11:48):

:speech_balloon: saulecabrera created PR review comment:

I think this approach has the added advantage that it checks for async support, which my previous comment states, but does not check at the engine level.

Wasmtime GitHub notifications bot (Jul 17 2026 at 11:48):

:speech_balloon: saulecabrera created PR review comment:

I just realized that you had introduced has_mmu_interruption, I believe this can be replaced with

if !cfg!(has_mmu_interruption) { ... }

This totally my fault for not digging deeper here.

With this change, I _think_ we can get rid of the host.architecture check.

Wasmtime GitHub notifications bot (Jul 20 2026 at 15:48):

erikrose commented on PR #12990:

Is there a reason we can't attach a trap code to the load, like we do other loads, and distinguish this mechanism based on that?

@cfallin Oh, sorry; I misinterpreted your "we can add to the module metadata, produced alongside e.g. trap codes" to mean a new piece of metadata was needed. However, it may yet be so: it turned out I needed not only to flag the interruption-check instruction offsets but also their lengths so I could compute where to resume after the interruption. The x64 mov instructions can be either 3 bytes or 4, depending on the source register—R12 (and RSP) make it 4.

So I see a couple of possibilities:

1. Keep as is.
2. Switch to using trap codes. Just note that I'd have to eat 2 separate ones: one signifying "3 bytes long" and another "4 bytes long".
3. Maybe there's something I can say to regalloc to assert I don't want my source reg to be R12, thus sparing one of the trap codes. I'm not sure how precious trap codes are. My intuition is that they're less precious than regalloc degrees of freedom.

What's your favorite? Numbers 2 and 3 have the advantage of glomming onto the LEB and delta-compression fanciness of trap codes, which I wasn't aware of until just now. My leaning is toward 2.

Wasmtime GitHub notifications bot (Jul 20 2026 at 15:49):

:speech_balloon: erikrose created PR review comment:

Nope, I introduced that after your commit! But thanks for noticing; I'll have a look.

Wasmtime GitHub notifications bot (Jul 20 2026 at 15:50):

:memo: erikrose submitted PR review.

Wasmtime GitHub notifications bot (Jul 20 2026 at 16:02):

erikrose edited a comment on PR #12990:

Is there a reason we can't attach a trap code to the load, like we do other loads, and distinguish this mechanism based on that?

@cfallin Oh, sorry; I misinterpreted your "we can add to the module metadata, produced alongside e.g. trap codes" to mean a new piece of metadata was needed. However, it may yet be so: it turned out I needed not only to flag the interruption-check instruction offsets but also their lengths so I could compute where to resume after the interruption. The x64 mov instructions can be either 3 bytes or 4, depending on the source register—R12 (and RSP) make it 4.

So I see a few possibilities:

1. Keep as is.
2. Switch to using trap codes. Just note that I'd have to eat 2 separate ones: one signifying "3 bytes long" and another "4 bytes long".
3. Maybe there's something I can say to regalloc to assert I don't want my source reg to be R12, thus sparing one of the trap codes. I'm not sure how precious trap codes are. My intuition is that they're less precious than regalloc degrees of freedom.

What's your favorite? Numbers 2 and 3 have the advantage of glomming onto the LEB and delta-compression fanciness of trap codes, which I wasn't aware of until just now. My leaning is toward 2.

Wasmtime GitHub notifications bot (Jul 20 2026 at 16:08):

erikrose edited a comment on PR #12990:

Is there a reason we can't attach a trap code to the load, like we do other loads, and distinguish this mechanism based on that?

@cfallin Oh, sorry; I misinterpreted your "we can add to the module metadata, produced alongside e.g. trap codes" to mean a new piece of metadata was needed. However, it may yet be so: it turned out I needed not only to flag the interruption-check instruction offsets but also their lengths so I could compute where to resume after the interruption. The x64 mov instructions can be either 3 bytes or 4, depending on the source register—R12 (and RSP) make it 4.

So I see a few possibilities:

1. Keep as is.
2. Switch to using trap codes. Just note that I'd have to eat 2 separate ones: one signifying "3 bytes long" and another "4 bytes long".
3. Maybe there's something I can say to regalloc to assert I don't want my source reg to be R12, thus sparing one of the trap codes. I'm not sure how precious trap codes are. My intuition is that they're less precious than regalloc degrees of freedom. [Ed: Looks like we have only 3 of them so far and pack them in a u8, so likely not precious at all.]

What's your favorite? Numbers 2 and 3 have the advantage of glomming onto the LEB and delta-compression fanciness of trap codes, which I wasn't aware of until just now. My leaning is toward 2.

Wasmtime GitHub notifications bot (Jul 20 2026 at 16:12):

cfallin commented on PR #12990:

I don't like any of those options actually -- and it gets at my general concern with the way that this change conflates abstractions from a number of logically-separate places in a way that makes things really brittle. The proposed solutions are (i) x64-specific, and (ii) combine distributed knowledge from instruction emission and regalloc into an invariant encoded in the trap table and then used by the signal handler (!!). The alternative is to have a very specific and weird "MMU interruption" concept in Cranelift. A compiler should not know what MMU interruption is; it understands loads, stores, and instruction metadata.

Let me ask more deeply: why do we need to know about the instruction encoding at all, to advance past the instruction? Why can't we resume into the guest and let it redo the load, with the page mapped back in? Unless we burn a virtual address on every interruption and never reuse it, it will have to be mapped back in eventually, so I don't think that's an issue?

If we do need to resume past the load, then we should not bake assumptions about possible lengths into the compiler. So "one trap code for 3 bytes, one for 4 bytes" is a bad idea: what happens when we get a new architecture that requires 5 bytes? Or add an optimization to the x64 backend to encode it in 2 bytes? Or ...

Instead what you're really trying to build is the concept of resumable traps. So in the trap table, we have a kind of entry that indicates "resumable" and gives the PC-offset to correct the captured PC by. One could encode that with a sparse array alongside trap codes. That might end up looking something like your current custom section, but named in a much less confusing and specific way, and reusable for any other purpose that requires resumable traps in the future (e.g. hardware debug-break opcodes, or missing-instruction emulation, or ...).

So: I prefer that we don't build a distributed, brittle invariant that is ISA-specific like this; but if we have to, let's reify the concept as a more fundamental thing.

Wasmtime GitHub notifications bot (Jul 20 2026 at 16:58):

alexcrichton commented on PR #12990:

Personally I think it would be best to prioritize performance numbers for this. I agree with everything @cfallin is saying review-wise, but I think it would be best to get on the same page about the overall shape of this feature first as well since that will inform the implementation which will have knock-on effects on what will be reviewed. Of the two possibilities here -- flipping virtual memory permissions vs having an indirect load that is dynamically switched to null -- there's still quite a lot of shared implementation/mechanisms between the two, but I also feel they're different enough we should try to settle on one before landing.

I personally have a strong hunch that what's implemented here, flipping virtual memory permissions, will be a significant performance regression over today's implementation of epochs, specifically because there are more TLB shootdowns (repeatedly unmapping previously-mapped pages). I understand that getting performance numbers on this isn't easy, and I also understand that it would be easiest to land everything here, gated, and then get performance numbers. This is a significant change, however, and I believe we're going to want more confidence before landing it, even gated.

The performance numbers that I'm specifically interested in is the performance of a multi-threaded program with wasm execution in a lot of threads. I'd like to see the performance effect of flipping pages being accessible/inaccessible for the running wasms. This would end-to-end exercise the Cranelift bits here, how the page flipping is implemented, how the list of pages to flip are managed, etc. In-repo the closest equivalent to this is benchmarking wasmtime serve under heavy load. While that's not a perfect benchmark it would at least give me personally more confidence in this change one way or another.

One thing I'll also explicitly say is that I'm specifically not too interested in the single-threaded overhead of epochs nor the impact of virtual mapping changes in a single-threaded program. I understand some benchmarks were done, but Sightglass does not exercise anything related to TLB shootdowns so it's the wrong benchmark corpus for what I'm interested in. While it's good to provie that this has lower overhead than epochs that's also pretty naturally expected given the reduction in the size of the generated code. The specific concerns I have lie in the TLB shootdown behavior, which surface only in a multithreaded environment.

Wasmtime GitHub notifications bot (Jul 20 2026 at 17:37):

erikrose commented on PR #12990:

@cfallin First, thank you for bringing your wide and lengthy wasmtime perspective to bear on this!

Why can't we resume into the guest and let it redo the load, with the page mapped back in?

We could, if we don't mind making resumption 1 instruction more expensive. It's the cold path, but I'd be lying to claim knowledge of how significant this would be. Actually, it should be pretty easy to get performance numbers—decent ones, if I make wasmtime serve MMU-interrupt-savvy like Alex suggests above. If it's much <1%, I'm inclined to just eat it and enjoy the simplicity.

There remains, however, a pathological case which troubles me:

1. The load is hit.
2. The interrupt happens.
3. The fiber-yielding routine runs and unprotects the page.
4. The interrupter thread (whose existence I'm positing in an embedder) re-protects the page.
5. The wasm code resumes, redoing the load and not making any progress.

This loop could happen indefinitely, hopefully unlikely but conceivably not, depending on how many fibers are competing, their scheduling algorithm, and how quickly the interrupter thread (in which I envision some adjustable sleep interval) spins. The epoch-deadline approach doesn't have this problem; it always makes forward progress.

So "one trap code for 3 bytes, one for 4 bytes" is a bad idea: what happens when we get a new architecture that requires 5 bytes? Or add an optimization to the x64 backend to encode it in 2 bytes? Or

That's only if we want to represent MMU interrupt locations as trap codes, of course. Otherwise, we have complete freedom to express instruction length in any kind of forward-compatible, ISA-agnostic way. Your suggestion to get aarch64 going before merging has the advantage of ferreting out these sorts of ISA-specific assumptions. So chalk up one in its column.

One could encode that with a sparse array alongside trap codes. That might end up looking something like your current custom section, but named in a much less confusing and specific way…

I appreciate your reimagining of this as a future-looking resumable-traps mechanism, and I'm intrigued. To clarify, are you proposing to add the sparse array of lengths to the trap section or to bail out into a separate section after all to avoid further complicating traps?

How do you and @alexcrichton feel about this course of action?

1. Make wasmtime serve optionally use the MMU interrupt mechanism like wasmtime run does. (I punted on this initially because it's going to be much harder than run was.)
2. Measure perf on that, as heavy as I can load it.
3. If shootdowns consume the whole performance gain, try the additional-indirection approach.
4. If they don't, bench again with the repeated-load approach if we can assuage my pathological no-forward-progress concern above.

(In this plan, I'm leaving a few of your earlier comments on the table until we have an idea of multithreaded numbers.)

Wasmtime GitHub notifications bot (Jul 20 2026 at 19:17):

alexcrichton commented on PR #12990:

That sounds good to me. Personally I wouldn't be too concerned about live-locking of sorts with resuming and always faulting at the load-that-traps. I also don't think that the performance of executing a load twice will be noticable at all. Which is to say, I would agree with @cfallin that the initial state here is to run the load again. In the absolute worst case if live-locking is a problem in practice I'd prefer to go the route of rewriting the address the dead-load instruction loads from to be known-valid during resumption since that's easier to do cross-platform than skipping instructions.

Wasmtime GitHub notifications bot (Jul 22 2026 at 02:39):

cfallin commented on PR #12990:

We could, if we don't mind making resumption 1 instruction more expensive. It's the cold path, but I'd be lying to claim knowledge of how significant this would be. Actually, it should be pretty easy to get performance numbers—decent ones, if I make wasmtime serve MMU-interrupt-savvy like Alex suggests above. If it's much <1%, I'm inclined to just eat it and enjoy the simplicity.

Stated this way I think it's pretty clear that it should be negligible: one more machine instruction, on the cold path, after a resume, which happens once a millisecond or so, is no big deal to any computer from the past 40 years (order-of-magnitude: modern core runs ~1B insts/second; a millisecond is a million instructions).

There remains, however, a pathological case which troubles me:

This only happens if interruption occurs so frequently that the guest cannot even be scheduled back onto the CPU before the next interruption. Arguably, the mechanism is even working correctly in this case: the interrupter has asserted the interrupt signal again, so we yield again. In other words, it's a faithful mechanism to the purpose. It's up to the overall system not to interrupt too frequently.

Given all this, I feel pretty strongly that let's not try to do any instruction-skipping magic; hence, no extra metadata sections at all.

Strong +1 to: let's get data from a full multithreaded benchmark with frequent interrupts (wasmtime serving wasi-http should be fine, with a concurrent load generator) to see how the TLB shootdowns impact perf.

Wasmtime GitHub notifications bot (Jul 31 2026 at 18:25):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Aug 01 2026 at 15:20):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Aug 14 2026 at 19:09):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Aug 27 2026 at 20:58):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Aug 28 2026 at 02:17):

github-actions[bot] added the label cranelift:area:aarch64 on PR #12990.

Wasmtime GitHub notifications bot (Sep 04 2026 at 17:53):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Sep 04 2026 at 18:53):

erikrose commented on PR #12990:

Benchmark time! We're got various realistic interrupt frequencies, a spectrum of TLB contention, and a whole sweaterload of threads. I concentrated on (1) seeing whether MMU interruption falls off a cliff due to TLB shootdowns and (2) seeing where it delivers a performance benefit.

All of these measurements were done on an otherwise idle Xeon W-2295 machine with 18 physical cores, 36 logical, sporting a 1536-entry shared TLB per physical core. Turbo was disabled, via echo 1 > /sys/devices/system/cpu/intel_pstate/no_turbo. As the goal was to model or exceed production stresses, I applied a 30-second load of 72 concurrent requests from a secondary box, a 5.4ms average ping away over a 1Gb network. 72 concurrent was obviously enough to keep 36 cores busy; the overshoot ensured there was always a pending task for a core to switch to, for the workloads that include yields. Having more wasms in flight also increased TLB contention, as we'll see later.

Mechanically, the load was applied through a bespoke wrapper driving the oha load-testing tool. (Still deciding whether I should deposit these scripts in the benches/ folder.) It was configured to wait for all in-flight requests to finish after a 30-second test period was up. This avoided inadvertently slowing the next workload while the previous drained. Consequently, queries per second was chosen as the main evaluative metric, being unaffected by the unsaturated fill and drain periods. The tradeoff is that latencies see some faster-than-typical results during fill and drain, so we evaluate only p50 and p90. (I found p99 was either corroborative or else noise and almost by definition outliers, so I elide it from the graphs.) I did not interleave epoch and MMU runs, but I've observed the numbers to be undistorted when spot-checked against solo runs.

The load itself was a single wasm component, a release build compiled from Rust, implementing a tight loop. It exposed 3 parameters:

• i: number of iterations to take through the loop
• y: how many iterations to go between yields (using a sleep(0) call)
• p: how many pages of RAM it would keep live, by both reading and writing one page per iteration, in round-robin order

By adjusting those parameters, this program can manifest as a CPU busy loop, various frequencies of waiting upon IO, and various levels of TLB saturation for the interrupt page to contend with. For concision, parameter sets are expressed as "recipes", e.g. 123p456y789i, which are attached as reproducibility breadcrumbs to some graphs. Note that the purpose of the benchmark program is not to match a realistic load like Spidermonkey but to generate a spectrum of best- to worst-case results for MMU interruption.

On the server side, I run a release build of wasmtime serve with -Wtimeout=99s so the interruption machinery is activated but never actually used to abort any request. Before the first workload, wasmtime is warmed for 2 seconds with the same wasm, at 72 concurrent requests. This allows it to do JITting and fill the instance pool so as to not disadvantage the first workload.

I edit wasmtime's serve.rs to make epoch ticks occur every 0.5ms but deadlines every 5ms (so every 10 ticks). This is representative of our settings in production. For a fair comparison, MMU interruption times its triggering loop to interrupt every running fiber every 5ms. Both MMU and epoch are implemented with sleep() loops, and both simply sleep for 5ms (or smaller slices adding up to that, in the case of MMU) and then do their work, rather than integrating feedback to approximate a realtime schedule. The epoch loop increments the epoch each iteration, while MMU spins around a list of fibers and calls mprotect() once. MMU has to spin faster and thus sleep shorter, but the sleep duration with this core count (1s / 36 = 278µs) is still comfortably above the min sleep duration of ≈169µs on this box. (An industrial embedding may do multiple interruptions per loop to keep up on much higher core counts.)

For each of the 30-second workloads, I run 3 repetitions and graph the mean. This emphasizes amortized wins over variation in any given rep. I also graphed a set of medians (not shown), and they didn't look much different. Most of the graphs have a neighboring scatter plot of the raw values (zoom in), to give an idea of variance.

The stage set, let's see some numbers!

QPS across all workloads

This graph makes a broad point before we dive into fiddly details: MMU delivers more queries per second for all workloads but one, where it loses by only .01 QPS. Whether the wins are substantial or slight, it clearly takes no measurable step back in throughput. Even the whiskers (which here represent the other 2 non-median reps) rarely overlap. We'll examine each of these workloads in detail below.
<img width="1408" height="2539" alt="0" src="https://github.com/user-attachments/assets/899282c6-04ca-44d3-8870-95e0a857dc33" />

CPU busy loops

CPU busy loops are, as expected, where MMU shines. These workloads run, without yielding, through a ≈6 LOC loop for various numbers of iterations (clocked here in ms as measured on a single very long request on an unloaded box and then reduced proportionally). Because the loop is so short and thus the epoch checks so frequent, we see a large average effect of 40-62%, rather than the ≈12% seen under Spidermonkey in Sightglass. The spread of the raw values is narrower toward the top, where requests are longer and thus instantiation and other startup time are dwarfed by actual looping. Again, with 0y, yields do not figure in this scenario.
<img width="1689" height="691" alt="1" src="https://github.com/user-attachments/assets/0df7c91c-b320-4e01-822d-228382a103f4" />

Let's examine latency under the same runs. We find MMU delivers latency improvements as well, across all request lengths.
<img width="1689" height="968" alt="2" src="https://github.com/user-attachments/assets/d1ea1b27-5702-423b-b857-1a2a153a6edb" />

Adding yield points

Now we add a simulated async call into the busy loop, causing it to yield every 2 calibrated ms (to one of the other in-flight 71 requests oversubscribing our cores). Compare it to the previous blue-barred graph, which doesn't yield. The longer requests are still consistently better under MMU, though by less. Then, as requests get shorter, there's sometimes a loss in one rep, which brings down the average. Currently, MMU interruption interrupts running fibers in round-robin order. An optimization in which each is left unmolested until it completes its timeslice might fix the losses on short requests.
<img width="1689" height="691" alt="3" src="https://github.com/user-attachments/assets/23a801a4-5f5d-4139-955e-763348e7b752" />

Here are the latencies for the above. The average (esp. of p50s) is an improvement, but there are a couple of bad reps in the scatter. The bad ones correspond to QPS losses, which makes me doubly suspicious the short requests are getting interrupted before their initial timeslice. (Anything <5ms should be able to finish in a single timeslice.) Or it could simply be that the rescheduling delay after any yields doesn't disappear as readily into the shorter request durations.
<img width="1703" height="968" alt="4" src="https://github.com/user-attachments/assets/e1ad1618-ad8f-476a-913b-dc97d159ebd7" />

This repeats the above runs but interrupts twice as often: every 1ms. MMU always wins here, but we see some reps garner _quite_ the advantage, of similar magnitude to the best runs of the 2ms case. As often, we see the most variance on short requests.
<img width="1711" height="691" alt="5" src="https://github.com/user-attachments/assets/febfdbb0-c150-436d-8ad8-c09fee2ebc90" />

Latencies for the above. MMU is usually at an advantage. As before, it shows more variance on short requests.
<img width="1689" height="968" alt="6" src="https://github.com/user-attachments/assets/9dd0c90d-cc38-4514-9fe0-f32840a33103" />

QPS under rising TLB load

MMU wins slightly but consistently even as the TLB becomes saturated. Here we keep between 1 and 1024 pages hot in a round-robin fashion, with a byte in 1 of the pages read and written per iteration. 512 is where the TLB becomes saturated (512 * 2 cores * 2 instances competing for each core (in the best case) > the 1536 entries in the TLB). We take care to add some realistic yields so 72 instances get a chance to map in their memory, not just 36. We see that TLB thrashing hurts epoch and MMU interruption throughput about equally.
<img width="1172" height="875" alt="0a" src="https://github.com/user-attachments/assets/a1f90b59-4a97-49c9-ad7e-fad1669738d5" />

Here's the same experiment but with a percent-better view. We can more clearly see that MMU actually has a single-digit-percent advantage in the 3 scenarios worth considering (1, 100, maybe 256). After that, the MMU is way past saturation, QPS falls by a factor of 5 for both interruption mechanisms (see the previous chart), and we see noisy thrashing. Few would find it fruitful to push a production box this far. We may further consider discounting 512 and 1024 (and perhaps even some of the smaller p values) as unrealistic because the memory access pattern is so pathological: sparse page access, all pages equally hot, till the box is full. p=100 would mean 400K of RAM kept uniformly hot per request; real programs exhibit locality of reference. But our goal here was to stress MMU interruption to the point of breaking, and we did, seeing epoch break at the same point.
<img width="1695" height="644" alt="7" src="https://github.com/user-attachments/assets/5e7a92ce-4945-4701-aa85-99e742dfb295" />

MMU wins the ma
[message truncated]

Wasmtime GitHub notifications bot (Sep 04 2026 at 20:02):

erikrose edited a comment on PR #12990:

Benchmark time! We're got various realistic interrupt frequencies, a spectrum of TLB contention, and a whole sweaterload of threads. I concentrated on (1) seeing whether MMU interruption falls off a cliff due to TLB shootdowns and (2) seeing where it delivers a performance benefit.

All of these measurements were done on an otherwise idle Xeon W-2295 machine with 18 physical cores, 36 logical, sporting a 1536-entry shared TLB per physical core. Turbo was disabled, via echo 1 > /sys/devices/system/cpu/intel_pstate/no_turbo. As the goal was to model or exceed production stresses, I applied a 30-second load of 72 concurrent requests from a secondary box, a 5.4ms average ping away over a 1Gb network. 72 concurrent was obviously enough to keep 36 cores busy; the overshoot ensured there was always a pending task for a core to switch to, for the workloads that include yields. Having more wasms in flight also increased TLB contention, as we'll see later.

Mechanically, the load was applied through a bespoke wrapper driving the oha load-testing tool. (Still deciding whether I should deposit these scripts in the benches/ folder.) It was configured to wait for all in-flight requests to finish after a 30-second test period was up. This avoided inadvertently slowing the next workload while the previous drained. Consequently, queries per second was chosen as the main evaluative metric, being unaffected by the unsaturated fill and drain periods. The tradeoff is that latencies see some faster-than-typical results during fill and drain, so we evaluate only p50 and p90. (I found p99 was either corroborative or else noise and almost by definition outliers, so I elide it from the graphs.) I did not interleave epoch and MMU runs, but I've observed the numbers to be undistorted when spot-checked against solo runs.

The load itself was a single wasm component, a release build compiled from Rust, implementing a tight loop. It exposed 3 parameters:

• i: number of iterations to take through the loop
• y: how many iterations to go between yields (using a sleep(0) call)
• p: how many pages of RAM it would keep live, by both reading and writing one page per iteration, in round-robin order

By adjusting those parameters, this program can manifest as a CPU busy loop, various frequencies of waiting upon IO, and various levels of TLB saturation for the interrupt page to contend with. For concision, parameter sets are expressed as "recipes", e.g. 123p456y789i, which are attached as reproducibility breadcrumbs to some graphs. Note that the purpose of the benchmark program is not to match a realistic load like Spidermonkey but to generate a spectrum of best- to worst-case results for MMU interruption.

On the server side, I run a release build of wasmtime serve with -Wtimeout=99s so the interruption machinery is activated but never actually used to abort any request. Before the first workload, wasmtime is warmed for 2 seconds with the same wasm, at 72 concurrent requests. This allows it to do JITting and fill the instance pool so as to not disadvantage the first workload.

I edit wasmtime's serve.rs to make epoch ticks occur every 0.5ms but deadlines every 5ms (so every 10 ticks). This is representative of our settings in production. For a fair comparison, MMU interruption times its triggering loop to interrupt every running fiber every 5ms. Both MMU and epoch are implemented with sleep() loops, and both simply sleep for 5ms (or smaller slices adding up to that, in the case of MMU) and then do their work, rather than integrating feedback to approximate a realtime schedule. The epoch loop increments the epoch each iteration, while MMU spins around a list of fibers and calls mprotect() once. MMU has to spin faster and thus sleep shorter, but the sleep duration with this core count (1s / 36 = 278µs) is still comfortably above the min sleep duration of ≈169µs on this box. (An industrial embedding may do multiple interruptions per loop to keep up on much higher core counts.)

For each of the 30-second workloads, I run 3 repetitions and graph the mean. This emphasizes amortized wins over variation in any given rep. I also graphed a set of medians (not shown), and they didn't look much different. Most of the graphs have a neighboring scatter plot of the raw values (zoom in), to give an idea of variance.

The stage set, let's see some numbers!

QPS across all workloads

This graph makes a broad point before we dive into fiddly details: MMU delivers more queries per second for all workloads but one, where it loses by only .01 QPS. Whether the wins are substantial or slight, it clearly takes no measurable step back in throughput. Even the whiskers (which here represent the other 2 non-median reps) rarely overlap. We'll examine each of these workloads in detail below.
<img width="1408" height="2539" alt="0" src="https://github.com/user-attachments/assets/899282c6-04ca-44d3-8870-95e0a857dc33" />

CPU busy loops

CPU busy loops are, as expected, where MMU shines. These workloads run, without yielding, through a ≈6 LOC loop for various numbers of iterations (clocked here in ms as measured on a single very long request on an unloaded box and then reduced proportionally). Because the loop is so short and thus the epoch checks so frequent, we see a large average effect of 40-62%, rather than the ≈12% seen under Spidermonkey in Sightglass. The spread of the raw values is narrower toward the top, where requests are longer and thus instantiation and other startup time are dwarfed by actual looping. Again, with 0y, yields do not figure in this scenario.
<img width="1689" height="691" alt="1" src="https://github.com/user-attachments/assets/0df7c91c-b320-4e01-822d-228382a103f4" />

Let's examine latency under the same runs. We find MMU delivers latency improvements as well, across all request lengths.
<img width="1689" height="968" alt="2" src="https://github.com/user-attachments/assets/d1ea1b27-5702-423b-b857-1a2a153a6edb" />

Adding yield points

Now we add a simulated async call into the busy loop, causing it to yield every 2 calibrated ms (to one of the other in-flight 71 requests oversubscribing our cores). Compare it to the previous blue-barred graph, which doesn't yield. The longer requests are still consistently better under MMU, though by less. Then, as requests get shorter, there's sometimes a loss in one rep, which brings down the average. Currently, MMU interruption interrupts running fibers in round-robin order. An optimization in which each is left unmolested until it completes its timeslice might fix the losses on short requests.
<img width="1689" height="691" alt="3" src="https://github.com/user-attachments/assets/23a801a4-5f5d-4139-955e-763348e7b752" />

Here are the latencies for the above. The average (esp. of p50s) is an improvement, but there are a couple of bad reps in the scatter. The bad ones correspond to QPS losses, which makes me doubly suspicious the short requests are getting interrupted before their initial timeslice. (Anything <5ms should be able to finish in a single timeslice.) Or it could simply be that the rescheduling delay after any yields doesn't disappear as readily into the shorter request durations.
<img width="1703" height="968" alt="4" src="https://github.com/user-attachments/assets/e1ad1618-ad8f-476a-913b-dc97d159ebd7" />

This repeats the above runs but interrupts twice as often: every 1ms. MMU always wins here, but we see some reps garner _quite_ the advantage, of similar magnitude to the best runs of the 2ms case. As often, we see the most variance on short requests.
<img width="1711" height="691" alt="5" src="https://github.com/user-attachments/assets/febfdbb0-c150-436d-8ad8-c09fee2ebc90" />

Latencies for the above. MMU is usually at an advantage. As before, it shows more variance on short requests.
<img width="1689" height="968" alt="6" src="https://github.com/user-attachments/assets/9dd0c90d-cc38-4514-9fe0-f32840a33103" />

QPS under rising TLB load

MMU wins slightly but consistently even as the TLB becomes saturated. Here we keep between 1 and 1024 pages hot in a round-robin fashion, with a byte in 1 of the pages read and written per iteration. 512 is where the TLB becomes saturated (512 * 2 cores * 2 instances competing for each core (in the best case) > the 1536 entries in the TLB). We take care to add some realistic yields so 72 instances get a chance to map in their memory, not just 36. We see that TLB thrashing hurts epoch and MMU interruption throughput about equally.
<img width="1172" height="875" alt="0a" src="https://github.com/user-attachments/assets/a1f90b59-4a97-49c9-ad7e-fad1669738d5" />

Here's the same experiment but with a percent-better view. We can more clearly see that MMU actually has a single-digit-percent advantage in the 3 scenarios worth considering (1, 100, maybe 256). After that, the MMU is way past saturation, QPS falls by a factor of 5 for both interruption mechanisms (see the previous chart), and we see noisy thrashing. Few would find it fruitful to push a production box this far. We may further consider discounting 512 and 1024 (and perhaps even some of the smaller p values) as unrealistic because the memory access pattern is so pathological: sparse page access, all pages equally hot, till the box is full. p=100 would mean 400K of RAM kept uniformly hot per request; real programs exhibit locality of reference. But our goal here was to stress MMU interruption to the point of breaking, and we did, seeing epoch break at the same point.
<img width="1695" height="644" alt="7" src="https://github.com/user-attachments/assets/5e7a92ce-4945-4701-aa85-99e742dfb295" />

MMU wins
[message truncated]

Wasmtime GitHub notifications bot (Sep 05 2026 at 03:26):

erikrose edited a comment on PR #12990:

Benchmark time! We're got various realistic interrupt frequencies, a spectrum of TLB contention, and a whole sweaterload of threads. I concentrated on (1) seeing whether MMU interruption falls off a cliff due to TLB shootdowns and (2) seeing where it delivers a performance benefit.

All of these measurements were done on an otherwise idle Xeon W-2295 machine with 18 physical cores, 36 logical, sporting a 1536-entry shared TLB per physical core. Turbo was disabled, via echo 1 > /sys/devices/system/cpu/intel_pstate/no_turbo. As the goal was to model or exceed production stresses, I applied a 30-second load of 72 concurrent requests from a secondary box, a 5.4ms average ping away over a 1Gb network. 72 concurrent was obviously enough to keep 36 cores busy; the overshoot ensured there was always a pending task for a core to switch to, for the workloads that include yields. Having more wasms in flight also increased TLB contention, as we'll see later.

Mechanically, the load was applied through a bespoke wrapper driving the oha load-testing tool. (Still deciding whether I should deposit these scripts in the benches/ folder.) It was configured to wait for all in-flight requests to finish after a 30-second test period was up. This avoided inadvertently slowing the next workload while the previous drained. Consequently, queries per second was chosen as the main evaluative metric, being unaffected by the unsaturated fill and drain periods. The tradeoff is that latencies see some faster-than-typical results during fill and drain, so we evaluate only p50 and p90. (I found p99 was either corroborative or else noise and almost by definition outliers, so I elide it from the graphs.) I did not interleave epoch and MMU runs, but I've observed the numbers to be undistorted when spot-checked against solo runs.

The load itself was a single wasm component, a release build compiled from Rust, implementing a tight loop. It exposed 3 parameters:

• i: number of iterations to take through the loop
• y: how many iterations to go between yields (using a sleep(0) call)
• p: how many pages of RAM it would keep live, by both reading and writing one page per iteration, in round-robin order

By adjusting those parameters, this program can manifest as a CPU busy loop, various frequencies of waiting upon IO, and various levels of TLB saturation for the interrupt page to contend with. For concision, parameter sets are expressed as "recipes", e.g. 123p456y789i, which are attached as reproducibility breadcrumbs to some graphs. Note that the purpose of the benchmark program is not to match a realistic load like Spidermonkey but to generate a spectrum of best- to worst-case results for MMU interruption.

On the server side, I run a release build of wasmtime serve with -Wtimeout=99s so the interruption machinery is activated but never actually used to abort any request. Before the first workload, wasmtime is warmed for 2 seconds with the same wasm, at 72 concurrent requests. This allows it to do JITting and fill the instance pool so as to not disadvantage the first workload.

I edit wasmtime's serve.rs to make epoch ticks occur every 0.5ms but deadlines every 5ms (so every 10 ticks). This is representative of our settings in production. For a fair comparison, MMU interruption times its triggering loop to interrupt every running fiber every 5ms. Both MMU and epoch are implemented with sleep() loops, and both simply sleep for 5ms (or smaller slices adding up to that, in the case of MMU) and then do their work, rather than integrating feedback to approximate a realtime schedule. The epoch loop increments the epoch each iteration, while MMU spins around a list of fibers and calls mprotect() once. MMU has to spin faster and thus sleep shorter, but the sleep duration with this core count (1s / 36 = 278µs) is still comfortably above the min sleep duration of ≈169µs on this box. (An industrial embedding may do multiple interruptions per loop to keep up on much higher core counts.)

For each of the 30-second workloads, I run 10 repetitions for each treatment (mmu and epoch) and graph their mean (or the percent improvement, epoch mean to mmu mean). This emphasizes amortized wins over variation in any given rep. Most of the graphs have a neighboring scatter plot of the raw values (zoom in for better viewing), to give an idea of variance.

[Edit: I increased the rep count from 3 to 10 a few hours after posting this, redoing all the graphs and updating the interpretations. I had time to re-run everything and wanted to make sure we weren't just getting lucky.]

The stage set, let's see some numbers!

QPS across all workloads

This graph makes a broad point before we dive into fiddly details: MMU delivers more queries per second for all workloads. Whether the wins are substantial or slight, it clearly takes no measurable step back in throughput. Even the whiskers (which here represent the range of all reps) rarely overlap. We'll examine each of these workloads in detail below.
<img width="1408" height="2539" alt="0" src="https://github.com/user-attachments/assets/5bb2eb5d-228f-4a6e-bab7-7754cc15bf55" />

CPU busy loops

CPU busy loops are, as expected, where MMU shines. These workloads run, without yielding, through a ≈6 LOC loop for various numbers of iterations (clocked here in ms as measured on a single very long request on an unloaded box and then reduced proportionally). Because the loop is so short and thus the epoch checks so frequent, we see a large average effect of 29-57%, rather than the ≈12% seen under Spidermonkey in Sightglass. The spread of the raw values appears narrower toward the top, where requests are longer and thus instantiation and other startup time are dwarfed by actual looping. Again, with 0y, yields do not figure in this scenario.
<img width="1689" height="691" alt="1" src="https://github.com/user-attachments/assets/dbea3029-057e-42aa-a3a9-7de18e0e6532" />

Let's examine latency under the same runs. We find MMU delivers latency improvements as well, across all request lengths.
<img width="1689" height="968" alt="2" src="https://github.com/user-attachments/assets/476b2972-f639-4626-8f1c-604ef711b6f6" />

Adding yield points

Now we add a simulated async call into the busy loop, causing it to yield every 2 calibrated ms (to one of the other in-flight 71 requests oversubscribing our cores). Compare it to the previous blue-barred graph, which doesn't yield. All requests are still consistently better under MMU, though by less. The longest ones take a disproportionate dive; it's not clear why.
<img width="1689" height="691" alt="3" src="https://github.com/user-attachments/assets/aee1d5ec-70ea-4a7f-983b-f241c6bafd3e" />

Here are the latencies for the above. It's still an improvement across the board, though the magnitude drops. However, the short requests retain a large improvement in the worst-ish (p90) case.
<img width="1708" height="968" alt="4" src="https://github.com/user-attachments/assets/429b5e0a-bf50-43ca-95bb-7a816c2332bc" />

This repeats the above runs but interrupts twice as often: every 1ms. Again, MMU always wins, and the character of the results isn't readily distinguishable from the above, except that epoch suddenly suffers a large variance in its 1.3ms results, causing MMU to double its lead. As above, we see more variance as requests shorten. I would expect this under MMU but am surprised to see it on epoch as well.
<img width="1689" height="691" alt="5" src="https://github.com/user-attachments/assets/6431ae5e-c77d-47f9-86f4-122672171d39" />

Latencies for the above. MMU wins except for p90 on the longest requests.
<img width="1712" height="968" alt="6" src="https://github.com/user-attachments/assets/5d2a1c2b-39da-4d8f-9d0b-a0aa752baf4b" />

QPS under rising TLB load

MMU wins slightly but consistently even as the TLB becomes saturated. Here we keep between 1 and 1024 pages hot in a round-robin fashion, with a byte in 1 of the pages read and written per iteration. 512 is where the TLB becomes saturated (512 * 2 cores * 2 instances competing for each core (in the best case) > the 1536 entries in the TLB). We take care to add some realistic yields so 72 instances get a chance to map in their memory, not just 36. We see that TLB thrashing hurts epoch and MMU interruption throughput about equally.
<img width="1172" height="875" alt="0a" src="https://github.com/user-attachments/assets/a3eb9d77-aacd-4515-9319-3ff9c7da74ab" />

Here's the same experiment but with a percent-better view. We can more clearly see that MMU has a single-digit-percent advantage in the 3 scenarios worth considering (1, 100, maybe 256). After that, the MMU is way past saturation, QPS falls by a factor of 5 for both interruption mechanisms (see the previous chart), and we see noisy thrashing. Few would find it fruitful to push a production box this far. We may further consider discounting 512 and 1024 (and perhaps even some of the smaller p values) as unrealistic because the memory access pattern is so pathological: sparse page access, all pages equally hot, till the box is full. p=100 would mean 400K of RAM kept uniformly hot per request; real programs exhibit locality of reference. But our goal here was to stress MMU interruption to the point of breaking, and we did, seeing epoch break at the same point.
<img width="1695" height="644" alt="7" src="https://github.com/user-attachments/assets/a3295023-de82-49b8-8cc4-7795f41e25c9" />

MMU wins on the majority of requests (centering around p50) but loses slightly at p90. As above, 512p and up are pathological and can be discounted.
<img width="1708" height="875" alt="8" src="https://github.
[message truncated]

Wasmtime GitHub notifications bot (Sep 05 2026 at 04:32):

erikrose edited a comment on PR #12990:

Benchmark time! We're got various realistic interrupt frequencies, a spectrum of TLB contention, and a whole sweaterload of threads. I concentrated on (1) seeing whether MMU interruption falls off a cliff due to TLB shootdowns and (2) seeing where it delivers a performance benefit.

All of these measurements were done on an otherwise idle Xeon W-2295 machine with 18 physical cores, 36 logical, sporting a 1536-entry shared TLB per physical core. Turbo was disabled, via echo 1 > /sys/devices/system/cpu/intel_pstate/no_turbo. As the goal was to model or exceed production stresses, I applied a 30-second load of 72 concurrent requests from a secondary box, a 5.4ms average ping away over a 1Gb network. 72 concurrent was obviously enough to keep 36 cores busy; the overshoot ensured there was always a pending task for a core to switch to, for the workloads that include yields. Having more wasms in flight also increased TLB contention, as we'll see later.

Mechanically, the load was applied through a bespoke wrapper driving the oha load-testing tool. (Still deciding whether I should deposit these scripts in the benches/ folder.) It was configured to wait for all in-flight requests to finish after a 30-second test period was up. This avoided inadvertently slowing the next workload while the previous drained. Consequently, queries per second was chosen as the main evaluative metric, being unaffected by the unsaturated fill and drain periods. The tradeoff is that latencies see some faster-than-typical results during fill and drain, so we evaluate only p50 and p90. (I found p99 was either corroborative or else noise and almost by definition outliers, so I elide it from the graphs.) I did not interleave epoch and MMU runs, but I've observed the numbers to be undistorted when spot-checked against solo runs.

The load itself was a single wasm component, a release build compiled from Rust, implementing a tight loop. It exposed 3 parameters:

• i: number of iterations to take through the loop
• y: how many iterations to go between yields (using a sleep(0) call)
• p: how many pages of RAM it would keep live, by both reading and writing one page per iteration, in round-robin order

By adjusting those parameters, this program can manifest as a CPU busy loop, various frequencies of waiting upon IO, and various levels of TLB saturation for the interrupt page to contend with. For concision, parameter sets are expressed as "recipes", e.g. 123p456y789i, which are attached as reproducibility breadcrumbs to some graphs. Note that the purpose of the benchmark program is not to match a realistic load like Spidermonkey but to generate a spectrum of best- to worst-case results for MMU interruption.

On the server side, I run a release build of wasmtime serve with -Wtimeout=99s so the interruption machinery is activated but never actually used to abort any request. Before the first workload, wasmtime is warmed for 2 seconds with the same wasm, at 72 concurrent requests. This allows it to do JITting and fill the instance pool so as to not disadvantage the first workload.

I edit wasmtime's serve.rs to make epoch ticks occur every 0.5ms but deadlines every 5ms (so every 10 ticks). This is representative of our settings in production. For a fair comparison, MMU interruption times its triggering loop to interrupt every running fiber every 5ms. Both MMU and epoch are implemented with sleep() loops, and both simply sleep for 5ms (or smaller slices adding up to that, in the case of MMU) and then do their work, rather than integrating feedback to approximate a realtime schedule. The epoch loop increments the epoch each iteration, while MMU spins around a list of fibers and calls mprotect() once. MMU has to spin faster and thus sleep shorter, but the sleep duration with this core count (1s / 36 = 278µs) is still comfortably above the min sleep duration of ≈169µs on this box. (An industrial embedding may do multiple interruptions per loop to keep up on much higher core counts.)

I test 22 unique 30-second workloads, performed in a different random order for each of 10 repetitions, to average out any inter-workload effects. This happens for mmu and again for epochs. I then graph both means (or the percent improvement, epoch mean to mmu mean). This emphasizes amortized wins over variation in any given rep. Most of the graphs have a neighboring scatter plot of the raw values (zoom in for better viewing), to give an idea of variance.

[Edit: I increased the rep count from 3 to 10 a few hours after posting this, redoing all the graphs and updating the interpretations. I had time to re-run everything and wanted to make sure we weren't just getting lucky.]

The stage set, let's see some numbers!

QPS across all workloads

This graph makes a broad point before we dive into fiddly details: MMU delivers more queries per second for all workloads. Whether the wins are substantial or slight, it clearly takes no measurable step back in throughput. Even the whiskers (which here represent the range of all reps) rarely overlap. We'll examine each of these workloads in detail below.
<img width="1408" height="2539" alt="0" src="https://github.com/user-attachments/assets/5bb2eb5d-228f-4a6e-bab7-7754cc15bf55" />

CPU busy loops

CPU busy loops are, as expected, where MMU shines. These workloads run, without yielding, through a ≈6 LOC loop for various numbers of iterations (clocked here in ms as measured on a single very long request on an unloaded box and then reduced proportionally). Because the loop is so short and thus the epoch checks so frequent, we see a large average effect of 29-57%, rather than the ≈12% seen under Spidermonkey in Sightglass. The spread of the raw values appears narrower toward the top, where requests are longer and thus instantiation and other startup time are dwarfed by actual looping. Again, with 0y, yields do not figure in this scenario.
<img width="1689" height="691" alt="1" src="https://github.com/user-attachments/assets/dbea3029-057e-42aa-a3a9-7de18e0e6532" />

Let's examine latency under the same runs. We find MMU delivers latency improvements as well, across all request lengths.
<img width="1689" height="968" alt="2" src="https://github.com/user-attachments/assets/476b2972-f639-4626-8f1c-604ef711b6f6" />

Adding yield points

Now we add a simulated async call into the busy loop, causing it to yield every 2 calibrated ms (to one of the other in-flight 71 requests oversubscribing our cores). Compare it to the previous blue-barred graph, which doesn't yield. All requests are still consistently better under MMU, though by less. The longest ones take a disproportionate dive; it's not clear why.
<img width="1689" height="691" alt="3" src="https://github.com/user-attachments/assets/aee1d5ec-70ea-4a7f-983b-f241c6bafd3e" />

Here are the latencies for the above. It's still an improvement across the board, though the magnitude drops. However, the short requests retain a large improvement in the worst-ish (p90) case.
<img width="1708" height="968" alt="4" src="https://github.com/user-attachments/assets/429b5e0a-bf50-43ca-95bb-7a816c2332bc" />

This repeats the above runs but interrupts twice as often: every 1ms. Again, MMU always wins, and the character of the results isn't readily distinguishable from the above, except that epoch suddenly suffers a large variance in its 1.3ms results, causing MMU to double its lead. As above, we see more variance as requests shorten. I would expect this under MMU but am surprised to see it on epoch as well.
<img width="1689" height="691" alt="5" src="https://github.com/user-attachments/assets/6431ae5e-c77d-47f9-86f4-122672171d39" />

Latencies for the above. MMU wins except for p90 on the longest requests.
<img width="1712" height="968" alt="6" src="https://github.com/user-attachments/assets/5d2a1c2b-39da-4d8f-9d0b-a0aa752baf4b" />

QPS under rising TLB load

MMU wins slightly but consistently even as the TLB becomes saturated. Here we keep between 1 and 1024 pages hot in a round-robin fashion, with a byte in 1 of the pages read and written per iteration. 512 is where the TLB becomes saturated (512 * 2 cores * 2 instances competing for each core (in the best case) > the 1536 entries in the TLB). We take care to add some realistic yields so 72 instances get a chance to map in their memory, not just 36. We see that TLB thrashing hurts epoch and MMU interruption throughput about equally.
<img width="1172" height="875" alt="0a" src="https://github.com/user-attachments/assets/a3eb9d77-aacd-4515-9319-3ff9c7da74ab" />

Here's the same experiment but with a percent-better view. We can more clearly see that MMU has a single-digit-percent advantage in the 3 scenarios worth considering (1, 100, maybe 256). After that, the MMU is way past saturation, QPS falls by a factor of 5 for both interruption mechanisms (see the previous chart), and we see noisy thrashing. Few would find it fruitful to push a production box this far. We may further consider discounting 512 and 1024 (and perhaps even some of the smaller p values) as unrealistic because the memory access pattern is so pathological: sparse page access, all pages equally hot, till the box is full. p=100 would mean 400K of RAM kept uniformly hot per request; real programs exhibit locality of reference. But our goal here was to stress MMU interruption to the point of breaking, and we did, seeing epoch break at the same point.
<img width="1695" height="644" alt="7" src="https://github.com/user-attachments/assets/a3295023-de82-49b8-8cc4-7795f41e25c9" />

MMU wins on the majority of requests (centering around p50) but loses slightly at p90. As above, 512p and up
[message truncated]

Wasmtime GitHub notifications bot (Sep 05 2026 at 12:31):

erikrose edited a comment on PR #12990:

Benchmark time! We're got various realistic interrupt frequencies, a spectrum of TLB contention, and a whole sweaterload of threads. I concentrated on (1) seeing whether MMU interruption falls off a cliff due to TLB shootdowns and (2) seeing where it delivers a performance benefit.

All of these measurements were done on an otherwise idle Xeon W-2295 machine with 18 physical cores, 36 logical, sporting a 1536-entry shared TLB per physical core. Turbo was disabled, via echo 1 > /sys/devices/system/cpu/intel_pstate/no_turbo. As the goal was to model or exceed production stresses, I applied a 30-second load of 72 concurrent requests from a secondary box, a 5.4ms average ping away over a 1Gb network. 72 concurrent was obviously enough to keep 36 cores busy; the overshoot ensured there was always a pending task for a core to switch to, for the workloads that include yields. Having more wasms in flight also increased TLB contention, as we'll see later.

Mechanically, the load was applied through a bespoke wrapper driving the oha load-testing tool. (Still deciding whether I should deposit these scripts in the benches/ folder.) It was configured to wait for all in-flight requests to finish after a 30-second test period was up. This avoided inadvertently slowing the next workload while the previous drained. Consequently, queries per second was chosen as the main evaluative metric, being unaffected by the unsaturated fill and drain periods. The tradeoff is that latencies see some faster-than-typical results during fill and drain, so we evaluate only p50 and p90. (I found p99 was either corroborative or else noise and almost by definition outliers, so I elide it from the graphs.) I did not interleave epoch and MMU runs, but I've observed the numbers to be undistorted when spot-checked against solo runs.

The load itself was a single wasm component, a release build compiled from Rust, implementing a tight loop. It exposed 3 parameters:

• i: number of iterations to take through the loop
• y: how many iterations to go between yields (using a sleep(0) call)
• p: how many pages of RAM it would keep live, by both reading and writing one page per iteration, in round-robin order

By adjusting those parameters, this program can manifest as a CPU busy loop, various frequencies of waiting upon IO, and various levels of TLB saturation for the interrupt page to contend with. For concision, parameter sets are expressed as "recipes", e.g. 123p456y789i, which are attached as reproducibility breadcrumbs to some graphs. Note that the purpose of the benchmark program is not to match a realistic load like Spidermonkey but to generate a spectrum of best- to worst-case results for MMU interruption.

On the server side, I run a release build of wasmtime serve with -Wtimeout=99s so the interruption machinery is activated but never actually used to abort any request. Before the first workload, wasmtime is warmed for 2 seconds with the same wasm, at 72 concurrent requests. This allows it to do JITting and fill the instance pool so as to not disadvantage the first workload.

I edit wasmtime's serve.rs to make epoch ticks occur every 0.5ms but deadlines every 5ms (so every 10 ticks). This is representative of our settings in production. For a fair comparison, MMU interruption times its triggering loop to interrupt every running fiber every 5ms. Both MMU and epoch are implemented with sleep() loops, and both simply sleep for 5ms (or smaller slices adding up to that, in the case of MMU) and then do their work, rather than integrating feedback to approximate a realtime schedule. The epoch loop increments the epoch each iteration, while MMU spins around a list of fibers and calls mprotect() once. MMU has to spin faster and thus sleep shorter, but the sleep duration with this core count (1s / 36 = 278µs) is still comfortably above the min sleep duration of ≈169µs on this box. (An industrial embedding may do multiple interruptions per loop to keep up on much higher core counts.)

I test 22 unique 30-second workloads, performed in a different random order for each of 10 repetitions, to average out any inter-workload effects. This happens for mmu and again for epochs. I then graph both means (or the percent improvement, epoch mean to mmu mean). This emphasizes amortized wins over variation in any given rep. Most of the graphs have a neighboring scatter plot of the raw values (zoom in for better viewing), to give an idea of variance and magnitude.

[Edit: I increased the rep count from 3 to 10 a few hours after posting this, redoing all the graphs and updating the interpretations. I had time to re-run everything and wanted to make sure we weren't just getting lucky.]

The stage set, let's see some numbers!

QPS across all workloads

This graph makes a broad point before we dive into fiddly details: MMU delivers more queries per second for all workloads. Whether the wins are substantial or slight (the log scale compresses differences), it clearly takes no measurable step back in throughput. Even the whiskers (which here represent the range of all reps) rarely overlap. We'll examine each of these workloads in detail below.
<img width="1408" height="2539" alt="0" src="https://github.com/user-attachments/assets/5bb2eb5d-228f-4a6e-bab7-7754cc15bf55" />

CPU busy loops

CPU busy loops are, as expected, where MMU shines. These workloads run, without yielding, through a ≈6 LOC loop for various numbers of iterations (clocked here in ms as measured on a single very long request on an unloaded box and then reduced proportionally). Because the loop is so short and thus the epoch checks so frequent, we see a large average effect of 29-57%, rather than the ≈12% seen under Spidermonkey in Sightglass. The spread of the raw values appears narrower toward the top, where requests are longer and thus instantiation and other startup time are dwarfed by actual looping. Again, with 0y, yields do not figure in this scenario.
<img width="1689" height="691" alt="1" src="https://github.com/user-attachments/assets/dbea3029-057e-42aa-a3a9-7de18e0e6532" />

Let's examine latency under the same runs. We find MMU delivers latency improvements as well, across all request lengths.
<img width="1689" height="968" alt="2" src="https://github.com/user-attachments/assets/476b2972-f639-4626-8f1c-604ef711b6f6" />

Adding yield points

Now we add a simulated async call into the busy loop, causing it to yield every 2 calibrated ms (to one of the other in-flight 71 requests oversubscribing our cores). Compare it to the previous blue-barred graph, which doesn't yield. All requests are still consistently better under MMU, though by less. The longest ones take a disproportionate dive; it's not clear why.
<img width="1689" height="691" alt="3" src="https://github.com/user-attachments/assets/aee1d5ec-70ea-4a7f-983b-f241c6bafd3e" />

Here are the latencies for the above. It's still an improvement across the board, though the magnitude drops. However, the short requests retain a large improvement in the worst-ish (p90) case.
<img width="1708" height="968" alt="4" src="https://github.com/user-attachments/assets/429b5e0a-bf50-43ca-95bb-7a816c2332bc" />

This repeats the above runs but interrupts twice as often: every 1ms. Again, MMU always wins, and the character of the results isn't readily distinguishable from the above, except that epoch suddenly suffers a large variance in its 1.3ms results, causing MMU to double its lead. As above, we see more variance as requests shorten. I would expect this under MMU but am surprised to see it on epoch as well.
<img width="1689" height="691" alt="5" src="https://github.com/user-attachments/assets/6431ae5e-c77d-47f9-86f4-122672171d39" />

Latencies for the above. MMU wins except for p90 on the longest requests.
<img width="1712" height="968" alt="6" src="https://github.com/user-attachments/assets/5d2a1c2b-39da-4d8f-9d0b-a0aa752baf4b" />

QPS under rising TLB load

MMU wins slightly but consistently even as the TLB becomes saturated. Here we keep between 1 and 1024 pages hot in a round-robin fashion, with a byte in 1 of the pages read and written per iteration. 512 is where the TLB becomes saturated (512 * 2 cores * 2 instances competing for each core (in the best case) > the 1536 entries in the TLB). We take care to add some realistic yields so 72 instances get a chance to map in their memory, not just 36. We see that TLB thrashing hurts epoch and MMU interruption throughput about equally.
<img width="1172" height="875" alt="0a" src="https://github.com/user-attachments/assets/a3eb9d77-aacd-4515-9319-3ff9c7da74ab" />

Here's the same experiment but with a percent-better view. We can more clearly see that MMU has a single-digit-percent advantage in the 3 scenarios worth considering (1, 100, maybe 256). After that, the MMU is way past saturation, QPS falls by a factor of 5 for both interruption mechanisms (see the previous chart), and we see noisy thrashing. Few would find it fruitful to push a production box this far. We may further consider discounting 512 and 1024 (and perhaps even some of the smaller p values) as unrealistic because the memory access pattern is so pathological: sparse page access, all pages equally hot, till the box is full. p=100 would mean 400K of RAM kept uniformly hot per request; real programs exhibit locality of reference. But our goal here was to stress MMU interruption to the point of breaking, and we did, seeing epoch break at the same point.
<img width="1695" height="644" alt="7" src="https://github.com/user-attachments/assets/a3295023-de82-49b8-8cc4-7795f41e25c9" />

MMU wins on the majority of requests (centering around p
[message truncated]

Wasmtime GitHub notifications bot (Sep 08 2026 at 13:46):

erikrose edited a comment on PR #12990:

Benchmark time! We're got various realistic interrupt frequencies, a spectrum of TLB contention, and a whole sweaterload of threads. I concentrated on (1) seeing whether MMU interruption falls off a cliff due to TLB shootdowns and (2) seeing where it delivers a performance benefit.

All of these measurements were done on an otherwise idle Xeon W-2295 machine with 18 physical cores, 36 logical, sporting a 1536-entry shared TLB per physical core. Turbo was disabled, via echo 1 > /sys/devices/system/cpu/intel_pstate/no_turbo. As the goal was to model or exceed production stresses, I applied a 30-second load of 72 concurrent requests from a secondary box, a 5.4ms average ping away over a 1Gb network. 72 concurrent was obviously enough to keep 36 cores busy; the overshoot ensured there was always a pending task for a core to switch to, for the workloads that include yields. Having more wasms in flight also increased TLB contention, as we'll see later.

Mechanically, the load was applied through a bespoke wrapper driving the oha load-testing tool. (Still deciding whether I should deposit these scripts in the benches/ folder.) It was configured to wait for all in-flight requests to finish after a 30-second test period was up. This avoided inadvertently slowing the next workload while the previous drained. Consequently, queries per second was chosen as the main evaluative metric, being unaffected by the unsaturated fill and drain periods. The tradeoff is that latencies see some faster-than-typical results during fill and drain, so we evaluate only p50 and p90. (I found p99 was either corroborative or else noise and almost by definition outliers, so I elide it from the graphs.) I did not interleave epoch and MMU runs, but I've observed the numbers to be undistorted when spot-checked against solo runs.

The load itself was a single wasm component, a release build compiled from Rust, implementing a tight loop. It exposed 3 parameters:

• i: number of iterations to take through the loop
• y: how many iterations to go between yields (using a sleep(0) call)
• p: how many pages of RAM it would keep live, by both reading and writing one page per iteration, in round-robin order

By adjusting those parameters, this program can manifest as a CPU busy loop, various frequencies of waiting upon IO, and various levels of TLB saturation for the interrupt page to contend with. For concision, parameter sets are expressed as "recipes", e.g. 123p456y789i, which are attached as reproducibility breadcrumbs to some graphs. Note that the purpose of the benchmark program is not to match a realistic load like Spidermonkey but to generate a spectrum of best- to worst-case results for MMU interruption.

On the server side, I run a release build of wasmtime serve with -Wtimeout=99s so the interruption machinery is activated but never actually used to abort any request. Before the first workload, wasmtime is warmed for 2 seconds with the same wasm, at 72 concurrent requests. This allows it to do JITting and fill the instance pool so as to not disadvantage the first workload.

I edit wasmtime's serve.rs to make epoch ticks occur every 0.5ms but deadlines every 5ms (so every 10 ticks). This is representative of our settings in production. For a fair comparison, MMU interruption times its triggering loop to interrupt every running fiber every 5ms. Both MMU and epoch are implemented with sleep() loops, and both simply sleep for 5ms (or smaller slices adding up to that, in the case of MMU) and then do their work, rather than integrating feedback to approximate a realtime schedule. The epoch loop increments the epoch each iteration, while MMU spins around a list of fibers and calls mprotect() once. MMU has to spin faster and thus sleep shorter, but the sleep duration with this core count (1s / 36 = 278µs) is still comfortably above the min sleep duration of ≈169µs on this box. (An industrial embedding may do multiple interruptions per loop to keep up on much higher core counts.)

I test 22 unique 30-second workloads, performed in a different random order for each of 10 repetitions, to average out any inter-workload effects. This happens for mmu and again for epochs. I then graph both means (or the percent improvement, epoch mean to mmu mean). This emphasizes amortized wins over variation in any given rep. Most of the graphs have a neighboring scatter plot of the raw values (zoom in for better viewing), to give an idea of variance and magnitude.

[Edit: I increased the rep count from 3 to 10 a few hours after posting this, redoing all the graphs and updating the interpretations. I had time to re-run everything and wanted to make sure we weren't just getting lucky.]

The stage set, let's see some numbers!

QPS across all workloads

This graph makes a broad point before we dive into fiddly details: MMU delivers more queries per second for all workloads. Whether the wins are substantial or slight (the log scale compresses differences), it clearly takes no measurable step back in throughput. Even the whiskers (which here represent the range of all reps) rarely overlap. We'll examine each of these workloads in detail below.
<img width="1408" height="2539" alt="0" src="https://github.com/user-attachments/assets/5bb2eb5d-228f-4a6e-bab7-7754cc15bf55" />

CPU busy loops

CPU busy loops are, as expected, where MMU shines. These workloads run, without yielding, through a ≈6 LOC loop for various numbers of iterations (clocked here in ms as measured on a single very long request on an unloaded box and then reduced proportionally). Because the loop is so short and thus the epoch checks so frequent, we see a large average effect of 29-57%, rather than the ≈12% seen under Spidermonkey in Sightglass. The spread of the raw values appears narrower toward the top, where requests are longer and thus instantiation and other startup time are dwarfed by actual looping. Again, with 0y, yields do not figure in this scenario.
<img width="1689" height="691" alt="1" src="https://github.com/user-attachments/assets/b4f8f4b3-1519-48e8-8986-bb8e2feb6ef3" />

Let's examine latency under the same runs. We find MMU delivers latency improvements as well, across all request lengths.
<img width="1689" height="968" alt="2" src="https://github.com/user-attachments/assets/476b2972-f639-4626-8f1c-604ef711b6f6" />

Adding yield points

Now we add a simulated async call into the busy loop, causing it to yield every 2 calibrated ms (to one of the other in-flight 71 requests oversubscribing our cores). Compare it to the previous blue-barred graph, which doesn't yield. All requests are still consistently better under MMU, though by less. The longest ones take a disproportionate dive; it's not clear why.
<img width="1689" height="691" alt="3" src="https://github.com/user-attachments/assets/346ae01c-367b-4bd3-aa0d-98ece19d75d6" />

Here are the latencies for the above. It's still an improvement across the board, though the magnitude drops. However, the short requests retain a large improvement in the worst-ish (p90) case.
<img width="1708" height="968" alt="4" src="https://github.com/user-attachments/assets/429b5e0a-bf50-43ca-95bb-7a816c2332bc" />

This repeats the above runs but interrupts twice as often: every 1ms. Again, MMU always wins, and the character of the results isn't readily distinguishable from the above, except that epoch suddenly suffers a large variance in its 1.3ms results, causing MMU to double its lead. As above, we see more variance as requests shorten. I would expect this under MMU but am surprised to see it on epoch as well.
<img width="1689" height="691" alt="5" src="https://github.com/user-attachments/assets/6a6efcd7-dca3-4b66-8316-dab561f214b2" />

Latencies for the above. MMU wins except for p90 on the longest requests.
<img width="1712" height="968" alt="6" src="https://github.com/user-attachments/assets/5d2a1c2b-39da-4d8f-9d0b-a0aa752baf4b" />

QPS under rising TLB load

MMU wins slightly but consistently even as the TLB becomes saturated. Here we keep between 1 and 1024 pages hot in a round-robin fashion, with a byte in 1 of the pages read and written per iteration. 512 is where the TLB becomes saturated (512 * 2 cores * 2 instances competing for each core (in the best case) > the 1536 entries in the TLB). We take care to add some realistic yields so 72 instances get a chance to map in their memory, not just 36. We see that TLB thrashing hurts epoch and MMU interruption throughput about equally.
<img width="1172" height="875" alt="0a" src="https://github.com/user-attachments/assets/a3eb9d77-aacd-4515-9319-3ff9c7da74ab" />

Here's the same experiment but with a percent-better view. We can more clearly see that MMU has a single-digit-percent advantage in the 3 scenarios worth considering (1, 100, maybe 256). After that, the MMU is way past saturation, QPS falls by a factor of 5 for both interruption mechanisms (see the previous chart), and we see noisy thrashing. Few would find it fruitful to push a production box this far. We may further consider discounting 512 and 1024 (and perhaps even some of the smaller p values) as unrealistic because the memory access pattern is so pathological: sparse page access, all pages equally hot, till the box is full. p=100 would mean 400K of RAM kept uniformly hot per request; real programs exhibit locality of reference. But our goal here was to stress MMU interruption to the point of breaking, and we did, seeing epoch break at the same point.
<img width="1695" height="644" alt="7" src="https://github.com/user-attachments/assets/af9c9345-b0ed-4414-836f-183dcdd270c1" />

MMU wins on the majority of requests (centering around p
[message truncated]

Wasmtime GitHub notifications bot (Sep 08 2026 at 16:47):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Sep 08 2026 at 21:22):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Sep 09 2026 at 20:00):

:memo: alexcrichton submitted PR review:

Thanks for gathering all that data and collating it all together, that clearly took a significant amount of time and work, and I agree with the conclusion that it motivates this well to the point of "let's experiment with this".

I've done an initial skim of this PR and left some higher-level comments below in addition to the ones that others have left previously. Would you be able to come to the next Wasmtime meeting to present this as well? I think it'd be good to get a shared consensus on whether or not to move forward with this at a high-level and then finer details can be discussed about how exactly to land this (e.g. I'd prefer to not a +6kloc -3kloc PR as-one, but instead split this up, but there's other things too)

Wasmtime GitHub notifications bot (Sep 09 2026 at 20:00):

:speech_balloon: alexcrichton created PR review comment:

FWIW this is not safe -- there's nothing preventing MmuInterrupter from living longer than the store, and were that to happen the store's VMStoreContext pointer would be deallocated and lead to a UAF if this were called. This MmuInterrupter struct as-is will require keeping something in the store alive while this is alive.

Wasmtime GitHub notifications bot (Sep 09 2026 at 20:00):

:speech_balloon: alexcrichton created PR review comment:

This might be related to my other comment, but this comment should not be necessary. Without unsafe anywhere it should be by definition that pointers can't get mixed up and mis-used -- a lock should not be necessary to uphold an otherwises-safe API contract.

Wasmtime GitHub notifications bot (Sep 09 2026 at 20:00):

:speech_balloon: alexcrichton created PR review comment:

If you can finagle it I think this would be best to live within the platform-agnostic test_if_trap method or similar. That might need some refactoring to handle the outcome of an mmu interrupt, but that would help reduce duplication here around things like lookup_code and such. Additionally that'd help pave a path towards a cross-platform implementation which we'll inevitably want with this.

Wasmtime GitHub notifications bot (Sep 09 2026 at 20:00):

:speech_balloon: alexcrichton created PR review comment:

I think it'd be best to manage this in Store rather than in VMStoreContext which otherwise doesn't have a destructor. That way the page can be managed in a new/drop combo more idiomatically and the pointer can be reflected into here during construction.

Wasmtime GitHub notifications bot (Sep 09 2026 at 20:00):

:speech_balloon: alexcrichton created PR review comment:

Can this contain an else { panic!() } or similar to help catch bugs where it's enabled on a platform that doesn't have support?

Wasmtime GitHub notifications bot (Sep 09 2026 at 20:00):

:speech_balloon: alexcrichton created PR review comment:

Generally we try to keep the architecture-specific bits as contained as possible in the workspace as opposed to sprinkling them in various files. These functions seem like good candidates for functions in wasmtime-internal-fiber perhaps?

Wasmtime GitHub notifications bot (Sep 09 2026 at 20:00):

:speech_balloon: alexcrichton created PR review comment:

Alternatively folding this into store_handler_in_ucontext with some refactoring would assist with the portability story

Wasmtime GitHub notifications bot (Sep 09 2026 at 20:00):

:speech_balloon: alexcrichton created PR review comment:

Could this use get_trap_registers instead of duplicating it?

Wasmtime GitHub notifications bot (Sep 09 2026 at 20:00):

:speech_balloon: alexcrichton created PR review comment:

I realize what I'm about to ask is a relatively large change, but the semantic correctness of this function is more-or-less essential for safety and is something that I'm concerned about. This function is basically duplicating the trampoline logic we have in the compilation part of Wasmtime where we previously have the guarantee that all exits from wasm happen through compiled code and this is the first exit happening from not-compiled code. That means that this can't reuse the logic that compiled code all shares and this is duplicating functionality.

Could this, instead, be modeled as a cranelift-compiled function? For example every module with mmu interruption could have a cranelift-compiled function which takes a VMContext parameter, a return address parameter, and basically does what this does except from compiled code? That could then reuse all the exit-trampoline logic and additionally could hook into all the libcall infrastructure (e.g. this could literally call new_epoch for example). That would additionally avoid needing to make a number of internals throughout this PR as pub(crate).

My thinking is that the lookup of "is this an mmu trap instruction" would return "ok and if so here's the function to resume to". I also believe that we could remove all the register load/restore logic in the inline/naked asm below and defer that to Cranelift's ABI handling since this is effectively a preserve_all ABI of sorts.

Wasmtime GitHub notifications bot (Sep 09 2026 at 20:00):

:speech_balloon: alexcrichton created PR review comment:

I know this has been discussed before, and I forget if this is still planned to be done, but I think it'd help to simplify a fair bit to remove the knowledge of the size of a dead-load-with-context instruction. Resuming at the same instruction that faulted should be sufficient and would remove the need for a fair bit of infrastructure like this (although this'd still be a test-if-this-is-an-mmu-interrupt of course)

Wasmtime GitHub notifications bot (Sep 09 2026 at 20:45):

:memo: erikrose submitted PR review.

Wasmtime GitHub notifications bot (Sep 09 2026 at 20:45):

:speech_balloon: erikrose created PR review comment:

Yes! That's what I'm doing right now, along with removing the custom ELF section in favor of existing trap codes. :-)

Wasmtime GitHub notifications bot (Sep 09 2026 at 20:45):

:speech_balloon: erikrose edited PR review comment.

Wasmtime GitHub notifications bot (Sep 09 2026 at 21:06):

erikrose commented on PR #12990:

@alexcrichton

Thanks for gathering all that data and collating it all together, that clearly took a significant amount of time and work, and I agree with the conclusion that it motivates this well to the point of "let's experiment with this".

It was useful for me as well. The loads uncovered a few bugs, and the measurements uncovered some bottlenecks, now all squashed.

I've done an initial skim of this PR and left some higher-level comments below in addition to the ones that others have left previously. Would you be able to come to the next Wasmtime meeting to present this as well?

Thank you! Absolutely. I've added it to tomorrow's agenda.

Wasmtime GitHub notifications bot (Sep 11 2026 at 14:56):

erikrose commented on PR #12990:

@cfallin Here are the additional short-request graphs you requested. Can you remind me what hypothesis you were trying to test?

No yields:
<img width="1689" height="783" alt="1" src="https://github.com/user-attachments/assets/9613bfd1-1b27-4808-8dce-dd62506804ff" />
Yield every 2ms:
<img width="1706" height="783" alt="3" src="https://github.com/user-attachments/assets/d3b42401-90c5-4044-9aa8-4ade41d9c3b9" />
Yield every 1ms:
<img width="1707" height="783" alt="5" src="https://github.com/user-attachments/assets/13247c50-aae0-42e4-ae02-8560b0cf5f53" />

Wasmtime GitHub notifications bot (Sep 11 2026 at 18:32):

cfallin commented on PR #12990:

@erikrose thanks -- the question arose basically out of a question about whether instantiation rate was being stressed at all. Historically TLB-shootdown troubles have come when frequent instantiation occurs; so a short execution time per instance pushes more on that corner of the evaluation space.

I think that a "hello world" in JS on wasi-http is probably a good benchmark to run too. You should be able to pretty easily write that (~10 lines of JS then run it through StarlingMonkey) to run in wasmtime serve and I think it'd give much more confidence in the reality of the QPS numbers.

Wasmtime GitHub notifications bot (Sep 11 2026 at 21:46):

erikrose commented on PR #12990:

At yesterday's wasmtime meeting, someone suggest "interrupt groups", which would pool a number of Stores on a single interrupt page. This would allow an embedder to trade off fine-grained-ness of interruptions for lower mprotect frequency by assocating more Stores with a page—yay! Sitting down to implement it, I realized it can't work. When the page is protected, some Store (maybe a few if you're lucky) will be interrupted, and then the signal handler will unprotect the page. Some number of other Stores in the group will fail to be interrupted, because the page will be unprotected again before they ever execute a load from it. As long as we want a guaranteed interrupt of each Store every so often, the groups are a no-go.

Wasmtime GitHub notifications bot (Sep 12 2026 at 00:17):

cfallin commented on PR #12990:

@erikrose yes, that was my suggestion, and I guess there is a way to salvage it: a notion of phases or epochs (no, not those ones). Basically, instead of re-permitting access to the page, you put it on some "dirty/used" queue -- maybe you eventually realize all stores that are in the group have been interrupted at least once since so you're free to reclaim it. But it stays unmapped as long as some guests still are needing to be interrupted. To unblock a guest once interrupted, you update its pointer to a new page that is accessible. So a given group at any time may have an "unmapped, old epoch" page and a "mapped, new epoch" page.

The broader need/requirement I was trying to get at is: it seems very noisy-neighborish in terms of TLB shootdown impact to do two mprotect's per Store per interrupt. The actual goal is to interrupt some subset of Stores every so often. Why not make that interrupt event a single mprotect (or pair) -- batch up the impact in an asymptotically-better way.

Wasmtime GitHub notifications bot (Sep 14 2026 at 21:35):

erikrose commented on PR #12990:

Thanks! That's a fine idea, and I had a couple of variants all sketched out and ready to try: timer-wheel-shaped ones, ones that just poked Stores from the outside and thus needed no mprotect traffic at all.... That is, until I realized the rub: "unprotecting" a store by subbing out its page ptr is not so simple. As with epochs, part of MMU's speed comes from caching something in a local, very likely promoted to a register. For epochs, it's the Engine's counter; for MMU, the page ptr. So we'd need not only update VMStoreContext but somehow the local/register value as well. Otherwise, interuptions would continue to occur at every opportunity within any function already on the stack.

I have a couple ideas that could help:

1. Give dead_load_with_context an additional job: providing a def containing the new page ptr. We could squirrel it out of the signal handler in the context register it already pins and clobbers and expose it to the caller, who could then arrange (in func_environ.rs) for it to get written to the local. This costs an additional write, but it's probably to a register and thus cheap. (We'd continue to update VMStoreContext, of course, so any newly called functions pick up the right value.)

2. mprotect coalescing. The trouble isn't the number of pages; we have the vsize and the rsize to spare. It's the mprotects, each of which fires off an IPI. In particular, it's the mprotects that tigthen protections, not the ones that loosen them. (At least on Intel, TLB entries aren't made for PROT_NONE pages, so there's nothing to flush when we change them to PROT_READ. Linux 6.1 and up is smart enough to elide these flushes.) However, we can mprotect a whole swath of n contiguous pages in one call, which should collapse the number of IPIs to 1/n if we perfectly avoid fragmentation. I'll have to think about feasible it will be to keep fragmentation to a reasonable level.

I'm going to go rub my sketches against those and see if they do any good.

Wasmtime GitHub notifications bot (Sep 15 2026 at 00:38):

erikrose edited a comment on PR #12990:

Thanks! That's a fine idea, and I had a couple of variants all sketched out and ready to try: timer-wheel-shaped ones, ones that just poked Stores from the outside and thus needed no mprotect traffic at all.... That is, until I realized the rub: "unprotecting" a store by subbing out its page ptr is not so simple. As with epochs, part of MMU's speed comes from caching something in a local, very likely promoted to a register. For epochs, it's the Engine's counter; for MMU, the page ptr. So we'd need not only update VMStoreContext but somehow the local/register value as well. Otherwise, interuptions would continue to occur at every opportunity within any function already on the stack.

I have a couple ideas that could help:

1. Give dead_load_with_context an additional job: providing a def containing the new page ptr. We could squirrel it out of the signal handler in the context register it already pins and clobbers and expose it to the caller, who could then arrange (in func_environ.rs) for it to get written to the local. This costs an additional write, but it's probably to a register and thus cheap. (We'd continue to update VMStoreContext, of course, so any newly called functions pick up the right value.)

2. mprotect coalescing. The trouble isn't the number of pages; we have the vsize and the rsize to spare. It's the mprotects, each of which fires off an IPI. In particular, it's the mprotects that tigthen protections, not the ones that loosen them. (At least on Intel, TLB entries aren't made for PROT_NONE pages, so there's nothing to flush when we change them to PROT_READ. Linux 6.1 and up is smart enough to elide these flushes.) However, we can mprotect a whole swath of n contiguous pages in one call, which should collapse the number of IPIs to 1/n if we perfectly avoid fragmentation. I'll have to think about how feasible it will be to keep fragmentation to a reasonable level.

I'm going to go rub my sketches against those and see if they do any good.

Wasmtime GitHub notifications bot (Sep 15 2026 at 14:40):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Sep 15 2026 at 15:38):

erikrose edited a comment on PR #12990:

Thanks! That's a fine idea, and I had a couple of variants all sketched out and ready to try: timer-wheel-shaped ones, ones that just poked Stores from the outside and thus needed no mprotect traffic at all.... That is, until I realized the rub: "unprotecting" a store by subbing out its page ptr is not so simple. As with epochs, part of MMU's speed comes from caching something in a local, very likely promoted to a register. For epochs, it's the Engine's counter; for MMU, the page ptr. So we'd need not only update VMStoreContext but somehow the local/register value as well. Otherwise, interuptions would continue to occur at every opportunity within any function already on the stack.

I have a couple ideas that could help:

1. Give dead_load_with_context an additional job: providing a def containing the new page ptr. We could squirrel it out of the signal handler in the r10 register it already pins and clobbers and expose it to the caller, who could then arrange (in func_environ.rs) for it to get written to the local. This costs an additional write, but it's probably to a register and thus cheap. (We'd continue to update VMStoreContext, of course, so any newly called functions pick up the right value.)

2. mprotect coalescing. The trouble isn't the number of pages; we have the vsize and the rsize to spare. It's the mprotects, each of which fires off an IPI. In particular, it's the mprotects that tigthen protections, not the ones that loosen them. (At least on Intel, TLB entries aren't made for PROT_NONE pages, so there's nothing to flush when we change them to PROT_READ. Linux 6.1 and up is smart enough to elide these flushes.) However, we can mprotect a whole swath of n contiguous pages in one call, which should collapse the number of IPIs to 1/n if we perfectly avoid fragmentation. I'll have to think about how feasible it will be to keep fragmentation to a reasonable level.

I'm going to go rub my sketches against those and see if they do any good.

Wasmtime GitHub notifications bot (Sep 15 2026 at 19:42):

erikrose commented on PR #12990:

Hmm, never mind about number 1: it doesn't do any good for ancestor stack frames, which retain the old cached page ptr. So callers will spuriously re-interrupt until they return. Thinking continues…

Wasmtime GitHub notifications bot (Sep 15 2026 at 19:47):

erikrose commented on PR #12990:

Actually, a given stack frame will spuriously interrupt only once. In doing so, it will update its own cached ptr. So hmm, the bound is less bad than I thought. I don't have an intuitive sense of whether it'd be a good tradeoff.

Wasmtime GitHub notifications bot (Sep 16 2026 at 16:24):

erikrose updated PR #12990.

Wasmtime GitHub notifications bot (Sep 16 2026 at 17:06):

:memo: erikrose submitted PR review.

Wasmtime GitHub notifications bot (Sep 16 2026 at 17:06):

:speech_balloon: erikrose created PR review comment:

Done in 49f4e41.

Wasmtime GitHub notifications bot (Sep 18 2026 at 21:41):

erikrose updated PR #12990.

Last updated: Sep 20 2026 at 18:08 UTC