Stream: git-wasmtime

Topic: wasmtime / PR #14849 x64: Lower single-source `shuffle` o...

Wasmtime GitHub notifications bot (Oct 19 2026 at 20:27):

EvanCarroll opened PR #14849 from EvanCarroll:x64-single-source-shuffle to bytecodealliance:main:

x64: Lower single-source shuffle of distinct operands to one pshufb

Fixes #14848

A shuffle whose mask only selects lanes from one operand got a single pshufb
only when both operands were the same value. With distinct operands it fell
through to vpermi2b or to pshufb + pshufb + por, both of which read the
operand the mask never selects. This adds two rules, ahead of the vpermi2b
rules, that shuffle only the selected operand when every lane comes from one side.

Changes

• lower.isle: two rules guarded on SSSE3, like the existing shuffle _ a a
  rule:

  • shuffle _ a _ with an lhs-only mask becomes pshufb a;
  • shuffle _ _ b with an rhs-only mask becomes pshufb b.

  ISLE can't prove the new extractors disjoint from the a a rule, so they get
  their own priorities (2 and 1), and the three rules below shift down to 0, -1
  and -2. The twelve rules above are unchanged.

• lower/isle.rs: shuffle_lhs_only / shuffle_rhs_only extractors. The masks
  reuse shuffle_0_15_mask / shuffle_16_31_mask, which already produce exactly
  the right constant when a mask reads only one side.

• The SSSE3 guard keeps a target with AVX512VBMI but no SSSE3 on vpermi2b
  instead of a pshufb libcall. No real CPU has that combination, but the
  existing shuffle-avx512.clif target does, so the new VBMI tests are in their
  own file with a realistic target.

Tests

• isa/x64/shuffle.clif (SSE4.1) and new isa/x64/shuffle-avx512-single-source.clif
  (SSSE3 + VBMI): lhs-only and rhs-only masks over distinct operands. Against
  the old lowering they fail with pshufb/pshufb/por and vpermi2b
  respectively.

• runtests/simd-shuffle.clif: the same two masks, on every target in the file,
  including baseline x86_64 without SSSE3.

• All 1378 filetests and the 158 SIMD spec tests (cargo test --test wast -- simd)
  pass. No existing filetest output changes.

Results

i7-1165G7 (AVX512VBMI), single-thread:

	wasmtime 49.0.3	this PR
shuffle-false-dep.wat from the issue	3.37 s (3.4 ns/iteration)	0.85 s (0.85 ns/iteration), 4.0× faster
tract wasm_i32_4x1, 512×512 int8 GEMV, vs scalar generic_i32_4x1 in the same build	0.70–0.79×	1.48×

The second row compares the kernel against a scalar kernel in the same run,
which cancels out machine load. The two builds also have different bases (49.0.3
against main plus this PR). In absolute terms, on a single 49.0.3 binary,
changing only that shuffle operand in the module takes the kernel from 56.7 µs
to 27.2 µs.

Wasmtime GitHub notifications bot (Oct 19 2026 at 20:27):

EvanCarroll requested cfallin for a review on PR #14849.

Wasmtime GitHub notifications bot (Oct 19 2026 at 20:27):

EvanCarroll removed the review request for wasmtime-compiler-reviewers from PR #14849.

Wasmtime GitHub notifications bot (Oct 19 2026 at 20:27):

EvanCarroll requested wasmtime-compiler-reviewers for a review on PR #14849.

Wasmtime GitHub notifications bot (Oct 19 2026 at 22:44):

github-actions[bot] added the label cranelift:area:x64 on PR #14849.

Wasmtime GitHub notifications bot (Oct 19 2026 at 22:44):

github-actions[bot] added the label cranelift on PR #14849.

Wasmtime GitHub notifications bot (Oct 19 2026 at 22:44):

github-actions[bot] added the label isle on PR #14849.

Wasmtime GitHub notifications bot (Oct 19 2026 at 22:45):

github-actions[bot] commented on PR #14849:

Subscribe to Label Action

cc @avanhatt, @cfallin, @fitzgen, @mmcloughlin

<details>
This issue or pull request has been labeled: "cranelift", "cranelift:area:x64", "isle"

Thus the following users have been cc'd because of the following labels:

• avanhatt: isle
• cfallin: isle
• fitzgen: isle
• mmcloughlin: isle

To subscribe or unsubscribe from this label, edit the <code>.github/subscribe-to-label.json</code> configuration file.

Learn more.
</details>

Wasmtime GitHub notifications bot (Oct 20 2026 at 06:15):

EvanCarroll commented on PR #14849:

Some background on how this turned up, plus a clean before/after measurement of the code that found it.

How it was found. I'm porting a translation model (opus-mt, run with tract) to WebAssembly, and wrote an int8 matrix-vector kernel for it: wasm_i32_4x1. Its inner loop runs one i8x16.shuffle per 16 bytes (from i8x16_shuffle::<0, 4, 1, 5, …>(a16, a16), L386), then i32x4.dot_i16x8_s. Under V8 the kernel is 2.8–4.3× faster than tract's scalar generic_i32_4x1. Under Wasmtime, with the same .wasm and the same microbenchmark, it was 15–25% slower than the scalar kernel.

The cause was the shuffle. LLVM emits it with the loop accumulator as the unused second operand, and on this machine (AVX512VBMI) Cranelift lowers it to vpermi2b, which reads that operand, so each iteration's shuffle waits on the previous iteration's add. Changing only that operand in the module, so that the existing shuffle a a rule matches, halved the kernel's time on 49.0.3. This PR does that in the lowering instead.

Before/after on the microbenchmark. The same module throughout (built from the commit linked above, sha256 9b381e03…f9d504), on three Wasmtime builds:

• stock: release 49.0.3
• base: this PR's base, dcfbea188d
• pr: this PR, e08d044365

Both source builds use crates.io regalloc2 0.15.2, with no patches. Each configuration ran 5 rounds, interleaved, rotating the order, pinned to one core, on an i7-1165G7 (load average 1.2–3.1). Medians in ns per call (min–max):

m × k	wasm_i32_4x1 stock	wasm_i32_4x1 base	wasm_i32_4x1 PR	PR vs base	scalar generic_i32_4x1 stock / base / PR
16 × 96	579 (569–598)	582 (565–663)	440 (428–510)	1.32×	569 / 570 / 567
64 × 64	1,429 (1,415–1,493)	1,387 (1,345–1,663)	1,096 (1,073–1,099)	1.27×	1,451 / 1,424 / 1,440
64 × 256	3,923 (3,896–4,105)	3,928 (3,805–4,833)	2,444 (2,334–2,767)	1.61×	3,427 / 3,365 / 3,333
256 × 256	15,721 (15,448–19,084)	15,603 (14,968–17,794)	9,040 (8,738–9,804)	1.73×	13,210 / 13,131 / 12,886
1024 × 256	62,802 (61,056–92,200)	61,934 (59,275–71,235)	34,943 (34,425–35,780)	1.77×	52,461 / 51,984 / 50,680
512 × 512	58,862 (57,348–78,755)	58,183 (56,735–67,651)	30,944 (30,343–31,330)	1.88×	46,365 / 46,778 / 45,628
256 × 1536	83,715 (83,046–117,030)	83,151 (82,114–98,475)	41,609 (41,402–42,580)	2.00×	63,399 / 63,894 / 63,202
2048 × 512	232,149 (231,300–310,398)	234,916 (227,389–265,062)	123,569 (123,441–124,753)	1.90×	184,763 / 184,725 / 182,738
512 × 2048	222,993 (221,020–311,282)	223,783 (217,946–290,383)	110,397 (107,887–111,378)	2.03×	167,067 / 167,148 / 166,449

The scalar kernel runs in the same process and contains no shuffle, so it works as a control: it is the same on all three builds. Stock and base also match each other, so the difference comes from this PR alone. The kernel goes from about 0.75× the scalar kernel's speed to about 1.5×. The 512 × 512, 2048 × 512 and 512 × 2048 rows are the shapes of a transformer decoder's weight matmuls.

The kernel doesn't quite reach the hand-patched module's 27.2 µs at 512 × 512. That remaining gap is a separate register-allocation spill in the same loop, which I'm reporting to regalloc2 on its own.

To reproduce:

git clone -b opus-mt-wasm-parity https://github.com/EvanCarroll/tract && cd tract
git checkout 7d43f384b8e56a3ab587b0d3f952d514b7b80635
RUSTFLAGS='-C target-feature=+simd128' cargo bench --no-run --target wasm32-wasip1 -p tract-linalg --bench wasm
taskset -c 7 wasmtime --env RUST_TEST_NOCAPTURE=1 -- target/wasm32-wasip1/release/deps/wasm-*.wasm --bench
# the "int8 matvec (n=1)" section

Wasmtime GitHub notifications bot (Oct 20 2026 at 16:22):

:thumbs_up: alexcrichton submitted PR review:

Thanks!

Wasmtime GitHub notifications bot (Oct 20 2026 at 16:22):

alexcrichton added PR #14849 x64: Lower single-source shuffle of distinct operands to one pshufb to the merge queue

Wasmtime GitHub notifications bot (Oct 20 2026 at 17:01):

alexcrichton removed PR #14849 x64: Lower single-source shuffle of distinct operands to one pshufb from the merge queue

Wasmtime GitHub notifications bot (Oct 20 2026 at 17:01):

:check: alexcrichton merged PR #14849.

Last updated: Oct 31 2026 at 23:03 UTC