Stream: git-wasmtime

Topic: wasmtime / issue #14312 Cranelift: Regression in memory b...

Wasmtime GitHub notifications bot (Sep 10 2026 at 19:48):

stevendore opened issue #14312:

We embed wasmtime in a high-throughput HTTP proxy and bisected a throughput regression to #14230 (b5b8f49a04, "Make LastStores a proper lattice"). A memory-bound guest loop runs 8 to 18% slower depending on engine config, and the stock config is unaffected, which is likely why upstream benchmarks did not catch it. Full data, the bisect, and a byte-reproducible source repro are below.

.clif Test Case

This is a runtime performance regression (~8-18% on a memory-bound wasm loop), not a miscompile or crash, so there is no reduced .clif that "fails". The regression is a lost redundant-load elimination across the whole hot function, and clif-util bugpoint doesn't apply to perf deltas.

The hot function is a plain bubble sort (Rust source inline below, wasm build is byte-reproducible with the pinned toolchain). Its CLIF can be dumped from either revision with:

    wasmtime compile --emit-clif <dir> \
        -O signals-based-traps=n -O memory-reservation=0x800000 \
        -O memory-may-move=y bench_workload.wasm

Happy to attach the pre/post-#14230 CLIF or disassembly for the sort function if that's useful for triage.

Steps to Reproduce

• Build the guest component (byte-deterministic with the pinned toolchain - full source inline below: WIT world + ~170-line lib.rs + Cargo.toml/Cargo.lock):

  ```sh
  rustup toolchain install 1.94.1 --profile minimal
  rustup target add wasm32-wasip2 --toolchain 1.94.1
  cargo +1.94.1 build --target wasm32-wasip2 --release
  md5sum target/wasm32-wasip2/release/bench_workload.wasm

  cebda31ff7808793ae03be52e211cc80 - the exact artifact all numbers

  below were measured on

  ```

  (.cargo/config.toml sets -C link-arg=-zstack-size=131072, and the standard WASI 0.2.0 WIT packages are vendored under wit/deps/: cli, clocks, filesystem, io, random, and sockets - e.g. from the WebAssembly/wasi-cli 0.2.0 release.)

• Build the A/B runner (single binary linking both wasmtime revisions via cargo package renaming - source in the appendix below): wt_old = rev bc2f967927, wt_new = rev 774dec1c5e.

• Run it against the component, pinned, on a quiet machine:

  sh taskset -c 8 cargo run --release -- bench_workload.wasm

  The runner instantiates the component with all imports stubbed as traps (Linker::define_unknown_imports_as_traps - the benched path calls no imports) and times invoke("bench_sort_large", []) - a bubble sort of 7680 descending i32s in linear memory - over 3 alternating rounds per config.

Expected Results

Comparable per-call latency between bc2f967927 and 774dec1c5e for the same wasm and the same Config, on all configs.

Actual Results

Per-call latency regresses when memory accesses compile to explicit bounds-check control flow. The stock config is unaffected (medians of 21 calls):

config	old (bc2f967927)	new (774dec1c5e)	delta
stock Config	24.91 ms	25.04 ms	+0.5%
signals_based_traps(false)	33.64 ms	36.32 ms	+8.0%
signals_based_traps(false) + memory_reservation(8<<20) + memory_may_move(true)	36.60 ms	43.21 ms	+18.1%

git bisect run over the 91 commits (bc2f967927..774dec1c5e) on the 18% config was perfectly bimodal (every good rev within ±1% of old, every bad rev within ±1% of new, no ambiguous steps):

First bad commit: b5b8f49a04 - "Make LastStores a proper lattice" (#14230).

We see the same delta end-to-end through our production host (HTTP-benchmarked p50 36.9 -> 43.3 ms for the same guest call).

We understand #14230 fixes real soundness/determinism problems (order-dependent forwarding, illegal forwarding across control-flow joins) and are not asking for a revert - we're reporting the cost in case some precision is recoverable for the explicit-bounds-check pattern. We embed wasmtime in a host that owns its signal handlers, so signals_based_traps(false) is not optional for us.

Versions and Environment

Cranelift version or commit: regression window bc2f967927 (good) -> 774dec1c5e (bad), first bad commit b5b8f49a04 (#14230, cranelift-codegen 0.137.0-dev / wasmtime 50.0.0-dev line). Both built from source, default-features = false, features = ["cranelift", "runtime", "std", "parallel-compilation", "component-model"].

Operating system: Rocky Linux 9.3 (primary numbers), also reproduced (noisier) on Ubuntu 22.04.

Architecture: x86_64 (Intel Xeon Gold 6330, taskset-pinned). Host rustc 1.97.1, guest toolchain pinned to rustc 1.94.1 as above.

Extra Info

The magnitude tracks the exact emitted code shape - both the producing toolchain and the component glue move it (same host, same config, same workload source):

• rustc 1.94.1 build: +18.1%
• identical source rebuilt with rustc 1.97.1: +8.1%
• glue-trimmed variant world (wasi imports only), rustc 1.97.1: +4.3%
• import-free world, rustc 1.97.1: -1.9%
• bare core module (wasm32-unknown-unknown): ±1-2%

So a minimized repro must preserve the pipeline and toolchain (hence the pinned build), and the exposed population is existing deployed binaries, which embedders cannot recompile away. Our working hypothesis is that the extra branches/blocks from explicit bounds checks interact with the more conservative LastStores meet, but we haven't isolated the exact CLIF - happy to help narrow it.

---

Appendix: A/B runner source

[package]
name = "laststores-perf-repro"
version = "0.1.0"
edition = "2021"

[dependencies]
wt_old = { package = "wasmtime", git = "https://github.com/bytecodealliance/wasmtime", rev = "bc2f967927", default-features = false, features = ["cranelift", "runtime", "std", "parallel-compilation", "component-model"] }
wt_new = { package = "wasmtime", git = "https://github.com/bytecodealliance/wasmtime", rev = "774dec1c5e", default-features = false, features = ["cranelift", "runtime", "std", "parallel-compilation", "component-model"] }

use std::time::Instant;

macro_rules! bench {
    ($wt:ident, $slow_cfg:expr, $bytes:expr) => {{
        use $wt::component::{Component, Linker, Val};
        let mut c = $wt::Config::new();
        c.wasm_component_model(true);
        if $slow_cfg {
            c.signals_based_traps(false);
            c.memory_reservation(8 << 20);
            c.memory_may_move(true);
        }
        let engine = $wt::Engine::new(&c).unwrap();
        let component = Component::new(&engine, $bytes).unwrap();
        let mut linker: Linker<()> = Linker::new(&engine);
        linker.define_unknown_imports_as_traps(&component).unwrap();
        let mut store = $wt::Store::new(&engine, ());
        let inst = linker.instantiate(&mut store, &component).unwrap();
        let f = inst.get_func(&mut store, "invoke").unwrap();
        let args = [Val::String("bench_sort_large".into()), Val::List(vec![])];
        let mut ms = Vec::new();
        for i in 0..9 {
            let mut results = [Val::Bool(false)];
            let t = Instant::now();
            f.call(&mut store, &args, &mut results).unwrap();
            if i >= 2 { ms.push(t.elapsed().as_secs_f64() * 1e3); }
            assert!(matches!(&results[0], Val::Result(Ok(_))));
        }
        ms
    }};
}

fn median(v: &mut Vec<f64>) -> f64 {
    v.sort_by(|a, b| a.partial_cmp(b).unwrap());
    v[v.len() / 2]
}

fn main() {
    let bytes = std::fs::read(std::env::args().nth(1).expect("path to component")).unwrap();
    for (label, slow) in [("stock", false), ("no-signals+8M-reservation", true)] {
        // Alternate old/new rounds so CPU frequency drift hits both sides;
        // run pinned (taskset) on a quiet machine.
        let (mut old_ms, mut new_ms) = (Vec::new(), Vec::new());
        for _ in 0..3 {
            old_ms.extend(bench!(wt_old, slow, &bytes));
            new_ms.extend(bench!(wt_new, slow, &bytes));
        }
        let (old, new) = (median(&mut old_ms), median(&mut new_ms));
        println!("{label:>26}: old {old:7.2} ms  new {new:7.2} ms  ({:+.1}%)", (new / old - 1.0) * 100.0);
    }
}

Appendix: guest source

Cargo.lock

<details><summary>Details</summary>
<p>

# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4

[[package]]
name = "anyhow"
version = "1.0.102"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c"

[[package]]
name = "bench-workload"
version = "0.1.0"
dependencies = [
 "wit-bindgen",
]

[[package]]
name = "bitflags"
version = "2.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4388bee8683e3d04af747c73422af53102d2bd24d9eadb6cbc100baef4b43f8"

[[package]]
name = "equivalent"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"

[[package]]
name = "foldhash"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2"

[[package]]
name = "futures"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b147ee9d1f6d097cef9ce628cd2ee62288d963e16fb287bd9286455b241382d"
dependencies = [
 "futures-channel",
 "futures-core",
 "futures-executor",
 "futures-io",
 "futures-sink",
 "futures-task",
 "futures-util",
]

[[package]]
name = "futures-channel"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "07bbe89c50d7a535e539b8c17bc0b49bdb77747034daa8087407d655f3f7cc1d"
dependencies = [
 "futures-core",
 "futures-sink",
]

[[package]]
name = "futures-core"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7e3450815272ef58cec6d564423f6e755e25
[message truncated]

Wasmtime GitHub notifications bot (Sep 10 2026 at 19:48):

stevendore added the bug label to Issue #14312.

Wasmtime GitHub notifications bot (Sep 10 2026 at 19:48):

stevendore added the cranelift label to Issue #14312.

Wasmtime GitHub notifications bot (Sep 10 2026 at 19:56):

cfallin commented on issue #14312:

cc @fitzgen

Wasmtime GitHub notifications bot (Sep 14 2026 at 18:48):

fitzgen commented on issue #14312:

Can you attach a .wasm test case in addition to the sources? Thank you.

Wasmtime GitHub notifications bot (Sep 15 2026 at 02:46):

stevendore commented on issue #14312:

The .wasm file is here: sort-bench.component.wasm.zip. It is the exact one used for these measurements. Using a zip file in case github doesn't like plain binary. The md5sum of the unzipped is cebda31ff7808793ae03be52e211cc80.

Last updated: Sep 20 2026 at 18:08 UTC