Theo Butler (Oct 07 2026 at 16:24): Theo Butler (Oct 07 2026 at 16:24):Hello, I just opened a PR #14607. This is my first PR to this project, and what I'm proposing is pretty specific, so I figured I should add some more context here.
I found this suboptimal lowering of amode operations resulting in unnecessary register pressure while comparing the performance of a ChaCha20 WASM implementation against Rust directly targeting my machine. I originally had 2 findings that were causing the majority of the throughput difference. But the first issue was largely sidestepped by ongoing work that increased MAX_SPLITS_PER_SPILLSET from 2 to 3. So this PR includes a fix for the remaining issue that improves the throughput with MAX_SPLITS_PER_SPILLSET set to 2 and 3.
I'm currently looking into some potential regressions that came up in CI.
Theo Butler (Oct 07 2026 at 16:56):I've fixed the regression, but now the change does not result in the claimed performance improvement. Setting the PR to draft while I figure this out.
Theo Butler (Oct 07 2026 at 18:41):Fixed now, and PR comment updated. It turns out handling this case separately for i32 and i64 values makes sense on a 64-bit target. Sorry for all the noise.
Alex Crichton (Oct 07 2026 at 21:52):Thanks for the PR! In general AFAIK this is a really tricky problem where ideally we'd use regalloc to feed into lowering decisions, but I think that makes lowering nigh-undecidable and likely unmaintainable. Without doing that we're left with rigging up the best heuristics we can think of, and they'll inevitably be good for some cases and not great for others.
One thing you could do for this -- could you perhaps play around with different shapes here and measure the impact via sightglass? That's the benchmark suite we try to use to measure the impact of changes like this
Theo Butler (Oct 07 2026 at 21:59):Yeah, that makes sense. Sure, I'll dig into sightglass! That should help determine how much this might be over-fitting to my ChaCha example program.
Theo Butler (Oct 08 2026 at 03:59):Results are in! Seems like a win overall for execution. Compilation and instantiation are a bit of a mixed bag.
crypto-results.csv
crypto-results.svg
default-results.csv
default-results.svg
Chris Fallin (Oct 08 2026 at 05:02):A quick note on results: instantiation time should be almost constant across this change (there is a tiny bit of compiled code that runs for the generated instantiation entry point but it's trivial in Sightglass benchmarks' cases; the actual benchmark is all in Execution). So with the large error bars on your results I think this is more indicative of a noisy machine / not enough iterations. Could you try turning the iteration count up?
Also, the "default" suite is now kinda misnamed, since @fitzgen (he/him) recently did the work of statistically deriving a "good" full suite -- try pca.suite instead, probably?
FWIW it seems reasonable to me that the heuristic you have here is good, independently (it should also be net-positive for reg pressure -- one value (the sum) carried downward instead of two (the summands)). So IMHO it's a reasonable change. Just wanted to make sure we get good statistical evidence if we're going to go off of it :-)
Theo Butler (Oct 10 2026 at 15:57):Sorry for the delayed response. I'm travelling for a friend's wedding. I'll be able to get updated measurements using the pca.suite in a few days
fitzgen (he/him) (Oct 10 2026 at 17:47):FYI the default suite has been updated and is now PCA-based, you can use it for your new benchmarking
Theo Butler (Oct 12 2026 at 17:26):Alright, I'm back. Here are the results using the latest sightglass defaults.
(redoing to put the main branch as baseline)
Theo Butler (Oct 12 2026 at 18:47):updated results, using the latest sightglass and the main branch commit 65abe37 (parent of PR commit) as baseline:
report.html
sightglass-data.csv
fitzgen (he/him) (Oct 13 2026 at 16:51):can you paste the human readable sightglass-cli effect-size output? having to download and open an html report or run that command ourselves is a bit annoying
Theo Butler (Oct 13 2026 at 17:11):Sure! Here's that output (omitting non-execution results, and results with no difference in performance):
execution :: cycles :: gcc-loops
Δ = 131512.20 ± 2612.26 (confidence = 95%)
branch-a84af50.so is 1.02x to 1.02x faster than main-65abe37.so!
┌─────────┬─────────┬────────────┬─────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├─────────┼─────────┼────────────┼─────────┼───────────────────┤
│ 6079260 │ 6131970 │ 6106712.10 │ 6107190 │ branch-a84af50.so │
├─────────┼─────────┼────────────┼─────────┼───────────────────┤
│ 6213840 │ 6281670 │ 6238224.30 │ 6238440 │ main-65abe37.so │
└─────────┴─────────┴────────────┴─────────┴───────────────────┘
execution :: cycles :: libsodium-siphashx24
Δ = 66884.40 ± 43226.93 (confidence = 95%)
branch-a84af50.so is 1.00x to 1.02x faster than main-65abe37.so!
┌─────────┬─────────┬────────────┬─────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├─────────┼─────────┼────────────┼─────────┼───────────────────┤
│ 5746320 │ 5942220 │ 5775468.60 │ 5765190 │ branch-a84af50.so │
├─────────┼─────────┼────────────┼─────────┼───────────────────┤
│ 5744220 │ 6486390 │ 5842353.00 │ 5763360 │ main-65abe37.so │
└─────────┴─────────┴────────────┴─────────┴───────────────────┘
execution :: cycles :: starlingmonkey-markdown
Δ = 77557.50 ± 47457.66 (confidence = 95%)
main-65abe37.so is 1.00x to 1.02x faster than branch-a84af50.so!
┌─────────┬─────────┬────────────┬─────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├─────────┼─────────┼────────────┼─────────┼───────────────────┤
│ 7045950 │ 8051460 │ 7226131.20 │ 7170300 │ branch-a84af50.so │
├─────────┼─────────┼────────────┼─────────┼───────────────────┤
│ 7030020 │ 7947510 │ 7148573.70 │ 7124940 │ main-65abe37.so │
└─────────┴─────────┴────────────┴─────────┴───────────────────┘
execution :: cycles :: libsodium-hash3
Δ = 46854.30 ± 9263.58 (confidence = 95%)
branch-a84af50.so is 1.01x to 1.01x faster than main-65abe37.so!
┌─────────┬─────────┬────────────┬─────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├─────────┼─────────┼────────────┼─────────┼───────────────────┤
│ 5626680 │ 5836110 │ 5701653.90 │ 5698500 │ branch-a84af50.so │
├─────────┼─────────┼────────────┼─────────┼───────────────────┤
│ 5683410 │ 5838090 │ 5748508.20 │ 5748570 │ main-65abe37.so │
└─────────┴─────────┴────────────┴─────────┴───────────────────┘
execution :: cycles :: intgemm-simd
Δ = 57844.20 ± 36581.23 (confidence = 95%)
branch-a84af50.so is 1.00x to 1.01x faster than main-65abe37.so!
┌─────────┬─────────┬────────────┬─────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├─────────┼─────────┼────────────┼─────────┼───────────────────┤
│ 7220010 │ 7774950 │ 7403976.30 │ 7402290 │ branch-a84af50.so │
├─────────┼─────────┼────────────┼─────────┼───────────────────┤
│ 7267890 │ 7921620 │ 7461820.50 │ 7471920 │ main-65abe37.so │
└─────────┴─────────┴────────────┴─────────┴───────────────────┘
execution :: cycles :: kotlin-richards
Δ = 280364.40 ± 193153.58 (confidence = 95%)
main-65abe37.so is 1.00x to 1.01x faster than branch-a84af50.so!
┌──────────┬──────────┬─────────────┬──────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├──────────┼──────────┼─────────────┼──────────┼───────────────────┤
│ 37883250 │ 42894810 │ 38581208.70 │ 38252670 │ branch-a84af50.so │
├──────────┼──────────┼─────────────┼──────────┼───────────────────┤
│ 37808820 │ 40460460 │ 38300844.30 │ 38119950 │ main-65abe37.so │
└──────────┴──────────┴─────────────┴──────────┴───────────────────┘
execution :: cycles :: rust-protobuf
Δ = 177459.60 ± 155680.40 (confidence = 95%)
branch-a84af50.so is 1.00x to 1.01x faster than main-65abe37.so!
┌──────────┬──────────┬─────────────┬──────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├──────────┼──────────┼─────────────┼──────────┼───────────────────┤
│ 24065610 │ 27070680 │ 25654810.50 │ 25625490 │ branch-a84af50.so │
├──────────┼──────────┼─────────────┼──────────┼───────────────────┤
│ 23997060 │ 27484110 │ 25832270.10 │ 25802010 │ main-65abe37.so │
└──────────┴──────────┴─────────────┴──────────┴───────────────────┘
execution :: cycles :: tinygo-regex
Δ = 83829.30 ± 70488.34 (confidence = 95%)
branch-a84af50.so is 1.00x to 1.01x faster than main-65abe37.so!
┌──────────┬──────────┬─────────────┬──────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├──────────┼──────────┼─────────────┼──────────┼───────────────────┤
│ 13294950 │ 16173540 │ 13451327.10 │ 13377570 │ branch-a84af50.so │
├──────────┼──────────┼─────────────┼──────────┼───────────────────┤
│ 13387860 │ 14133330 │ 13535156.40 │ 13482990 │ main-65abe37.so │
└──────────┴──────────┴─────────────┴──────────┴───────────────────┘
execution :: cycles :: hashset
Δ = 39568.80 ± 27401.02 (confidence = 95%)
main-65abe37.so is 1.00x to 1.01x faster than branch-a84af50.so!
┌──────────┬──────────┬─────────────┬──────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├──────────┼──────────┼─────────────┼──────────┼───────────────────┤
│ 10253430 │ 10677750 │ 10433605.20 │ 10430820 │ branch-a84af50.so │
├──────────┼──────────┼─────────────┼──────────┼───────────────────┤
│ 10199580 │ 11127630 │ 10394036.40 │ 10379370 │ main-65abe37.so │
└──────────┴──────────┴─────────────┴──────────┴───────────────────┘
execution :: cycles :: shootout-xchacha20
Δ = 11470.80 ± 7929.07 (confidence = 95%)
main-65abe37.so is 1.00x to 1.00x faster than branch-a84af50.so!
┌─────────┬─────────┬────────────┬─────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├─────────┼─────────┼────────────┼─────────┼───────────────────┤
│ 5913900 │ 6207480 │ 5957086.20 │ 5950140 │ branch-a84af50.so │
├─────────┼─────────┼────────────┼─────────┼───────────────────┤
│ 5912580 │ 6042030 │ 5945615.40 │ 5938650 │ main-65abe37.so │
└─────────┴─────────┴────────────┴─────────┴───────────────────┘
thanks! looks promising
Theo Butler said:
omitting non-execution results
were there any statistically significant compilation results?
Chris Fallin (Oct 13 2026 at 17:18):omitting non-execution results
Also, please include whatever the tool says about instantiation -- that's the signal that indicated your earlier runs may have been too noisy
Theo Butler (Oct 13 2026 at 17:24):Here are the relevant compilation and instantiation outputs:
compilation :: cycles :: Sum Total
Δ = 58049625.00 ± 15633925.73 (confidence = 95%)
main-65abe37.so is 1.00x to 1.00x faster than branch-a84af50.so!
┌─────────────┬─────────────┬────────────────┬─────────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├─────────────┼─────────────┼────────────────┼─────────────┼───────────────────┤
│ 25124565480 │ 25415452320 │ 25216860674.40 │ 25199541210 │ branch-a84af50.so │
├─────────────┼─────────────┼────────────────┼─────────────┼───────────────────┤
│ 25069587720 │ 25349387520 │ 25158811049.40 │ 25156585590 │ main-65abe37.so │
└─────────────┴─────────────┴────────────────┴─────────────┴───────────────────┘
compilation :: cycles :: shootout-keccak
Δ = 597369.60 ± 452616.40 (confidence = 95%)
main-65abe37.so is 1.00x to 1.01x faster than branch-a84af50.so!
┌──────────┬──────────┬─────────────┬──────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├──────────┼──────────┼─────────────┼──────────┼───────────────────┤
│ 76410240 │ 83379810 │ 77973418.80 │ 77263950 │ branch-a84af50.so │
├──────────┼──────────┼─────────────┼──────────┼───────────────────┤
│ 76267080 │ 84486750 │ 77376049.20 │ 76783590 │ main-65abe37.so │
└──────────┴──────────┴─────────────┴──────────┴───────────────────┘
compilation :: cycles :: rust-json
Δ = 1252661.40 ± 475001.68 (confidence = 95%)
main-65abe37.so is 1.00x to 1.01x faster than branch-a84af50.so!
┌───────────┬───────────┬──────────────┬───────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├───────────┼───────────┼──────────────┼───────────┼───────────────────┤
│ 313845750 │ 322706310 │ 316467581.10 │ 316223040 │ branch-a84af50.so │
├───────────┼───────────┼──────────────┼───────────┼───────────────────┤
│ 312230940 │ 320737110 │ 315214919.70 │ 314734050 │ main-65abe37.so │
└───────────┴───────────┴──────────────┴───────────┴───────────────────┘
compilation :: cycles :: dotnet-raytracer
Δ = 5859756.00 ± 1631383.87 (confidence = 95%)
main-65abe37.so is 1.00x to 1.00x faster than branch-a84af50.so!
┌────────────┬────────────┬───────────────┬────────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├────────────┼────────────┼───────────────┼────────────┼───────────────────┤
│ 1646554110 │ 1683733860 │ 1664268547.80 │ 1663464750 │ branch-a84af50.so │
├────────────┼────────────┼───────────────┼────────────┼───────────────────┤
│ 1647578700 │ 1677804720 │ 1658408791.80 │ 1657478130 │ main-65abe37.so │
└────────────┴────────────┴───────────────┴────────────┴───────────────────┘
compilation :: cycles :: rust-protobuf
Δ = 550005.30 ± 217789.99 (confidence = 95%)
main-65abe37.so is 1.00x to 1.00x faster than branch-a84af50.so!
┌───────────┬───────────┬──────────────┬───────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├───────────┼───────────┼──────────────┼───────────┼───────────────────┤
│ 203163090 │ 207539910 │ 204531495.30 │ 204358890 │ branch-a84af50.so │
├───────────┼───────────┼──────────────┼───────────┼───────────────────┤
│ 202660860 │ 206228910 │ 203981490.00 │ 203883570 │ main-65abe37.so │
└───────────┴───────────┴──────────────┴───────────┴───────────────────┘
compilation :: cycles :: gcc-loops
Δ = 1318936.50 ± 835771.01 (confidence = 95%)
main-65abe37.so is 1.00x to 1.00x faster than branch-a84af50.so!
┌───────────┬───────────┬──────────────┬───────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├───────────┼───────────┼──────────────┼───────────┼───────────────────┤
│ 510895590 │ 524336280 │ 514702021.20 │ 513824880 │ branch-a84af50.so │
├───────────┼───────────┼──────────────┼───────────┼───────────────────┤
│ 509883720 │ 523149120 │ 513383084.70 │ 512691840 │ main-65abe37.so │
└───────────┴───────────┴──────────────┴───────────┴───────────────────┘
compilation :: cycles :: starlingmonkey-markdown
Δ = 41972231.40 ± 10839581.45 (confidence = 95%)
main-65abe37.so is 1.00x to 1.00x faster than branch-a84af50.so!
┌─────────────┬─────────────┬────────────────┬─────────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├─────────────┼─────────────┼────────────────┼─────────────┼───────────────────┤
│ 16691246610 │ 16914455790 │ 16783178511.90 │ 16776330300 │ branch-a84af50.so │
├─────────────┼─────────────┼────────────────┼─────────────┼───────────────────┤
│ 16666068060 │ 16817148570 │ 16741206280.50 │ 16744634940 │ main-65abe37.so │
└─────────────┴─────────────┴────────────────┴─────────────┴───────────────────┘
compilation :: cycles :: hex-simd
Δ = 345904.50 ± 174748.74 (confidence = 95%)
main-65abe37.so is 1.00x to 1.00x faster than branch-a84af50.so!
┌───────────┬───────────┬──────────────┬───────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├───────────┼───────────┼──────────────┼───────────┼───────────────────┤
│ 144335730 │ 147462270 │ 145241099.70 │ 145027140 │ branch-a84af50.so │
├───────────┼───────────┼──────────────┼───────────┼───────────────────┤
│ 143766420 │ 147482340 │ 144895195.20 │ 144836550 │ main-65abe37.so │
└───────────┴───────────┴──────────────┴───────────┴───────────────────┘
compilation :: cycles :: intgemm-simd
Δ = 1145013.30 ± 943957.01 (confidence = 95%)
main-65abe37.so is 1.00x to 1.00x faster than branch-a84af50.so!
┌───────────┬───────────┬──────────────┬───────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├───────────┼───────────┼──────────────┼───────────┼───────────────────┤
│ 510926400 │ 526739070 │ 516143306.40 │ 515600640 │ branch-a84af50.so │
├───────────┼───────────┼──────────────┼───────────┼───────────────────┤
│ 510748140 │ 527888910 │ 514998293.10 │ 514131570 │ main-65abe37.so │
└───────────┴───────────┴──────────────┴───────────┴───────────────────┘
compilation :: cycles :: rust-wasmprinter
Δ = 1334841.60 ± 1123494.83 (confidence = 95%)
main-65abe37.so is 1.00x to 1.00x faster than branch-a84af50.so!
┌───────────┬───────────┬──────────────┬───────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├───────────┼───────────┼──────────────┼───────────┼───────────────────┤
│ 681778410 │ 702041100 │ 690060733.50 │ 689261970 │ branch-a84af50.so │
├───────────┼───────────┼──────────────┼───────────┼───────────────────┤
│ 681356280 │ 699337890 │ 688725891.90 │ 687811560 │ main-65abe37.so │
└───────────┴───────────┴──────────────┴───────────┴───────────────────┘
instantiation :: cycles :: rust-json
Δ = 7255.50 ± 6270.77 (confidence = 95%)
branch-a84af50.so is 1.01x to 1.09x faster than main-65abe37.so!
┌────────┬────────┬───────────┬────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├────────┼────────┼───────────┼────────┼───────────────────┤
│ 112260 │ 218130 │ 148100.70 │ 144600 │ branch-a84af50.so │
├────────┼────────┼───────────┼────────┼───────────────────┤
│ 108810 │ 212280 │ 155356.20 │ 151320 │ main-65abe37.so │
└────────┴────────┴───────────┴────────┴───────────────────┘
instantiation :: cycles :: hoot-earley-boyer
Δ = 30990.90 ± 17458.37 (confidence = 95%)
branch-a84af50.so is 1.01x to 1.05x faster than main-65abe37.so!
┌────────┬─────────┬────────────┬─────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├────────┼─────────┼────────────┼─────────┼───────────────────┤
│ 988110 │ 1131150 │ 1049338.50 │ 1040850 │ branch-a84af50.so │
├────────┼─────────┼────────────┼─────────┼───────────────────┤
│ 990000 │ 1572900 │ 1080329.40 │ 1076670 │ main-65abe37.so │
└────────┴─────────┴────────────┴─────────┴───────────────────┘
instantiation :: cycles :: splay
Δ = 4546.80 ± 4347.35 (confidence = 95%)
branch-a84af50.so is 1.00x to 1.05x faster than main-65abe37.so!
┌────────┬────────┬───────────┬────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├────────┼────────┼───────────┼────────┼───────────────────┤
│ 120420 │ 223560 │ 172805.40 │ 171600 │ branch-a84af50.so │
├────────┼────────┼───────────┼────────┼───────────────────┤
│ 109590 │ 214920 │ 177352.20 │ 178440 │ main-65abe37.so │
└────────┴────────┴───────────┴────────┴───────────────────┘
instantiation :: cycles :: dotnet-raytracer
Δ = 9015.60 ± 5967.94 (confidence = 95%)
main-65abe37.so is 1.01x to 1.03x faster than branch-a84af50.so!
┌────────┬────────┬───────────┬────────┬───────────────────┐
│ Min │ Max │ Mean │ Median │ Engine │
├────────┼────────┼───────────┼────────┼───────────────────┤
│ 498240 │ 654090 │ 543438.00 │ 544320 │ branch-a84af50.so │
├────────┼────────┼───────────┼────────┼───────────────────┤
│ 484380 │ 590730 │ 534422.40 │ 536160 │ main-65abe37.so │
└────────┴────────┴───────────┴────────┴───────────────────┘
Hmm, the instantiation data is still quite noisy where it should be flat (unless I'm out-of-date here -- I know we refactored to put some more module init into Cranelift-compiled code, but I think most of the logic should still be in the Wasmtime runtime). What kind of system are you running on -- desktop? laptop? cloud VM? Any special measure to reduce noise? Can you see what happens if you increase iteration count?
Theo Butler (Oct 13 2026 at 17:33):I ran this on a desktop, with the command chrt -f 80 taskset -c 4-7 ./target/release/sightglass-cli benchmark -e ./main-65abe37.so -e ./branch-a84af50.so. And no other applications like browsers, etc, running.
Chris Fallin (Oct 13 2026 at 17:34):OK, increasing iteration count to see if the instantiation data becomes less noisy may be a good thing to try, then?
Theo Butler (Oct 13 2026 at 17:35):Alright, I'll run it again with a higher iteration count.
fitzgen (he/him) (Oct 13 2026 at 17:40):instantiation is always noisy and part of that is that we have some little bugs in our statistical analyses, which I am working on
I wouldn't read too much into the instantiation numbers
Theo Butler (Oct 13 2026 at 18:57):Updated results, now with SMT disabled and --iterations-per-process=40
effect-size.txt
report.html
sightglass-data.csv
Chris Fallin (Oct 13 2026 at 18:59):IMHO at least, there seems to be enough signal in positive (faster) execution-time results that I'd be happy seeing this change go in, since we also have a qualitative "theory of operation" for why it should be better as well
Chris Fallin (Oct 13 2026 at 19:00):(Alex is still the reviewer of record so I'll defer to him, but I'm happy to take over and read through in detail if he'd prefer)
Theo Butler (Oct 13 2026 at 19:11):I'm also running again now with --measure=callgrind, to see if that gives less noisy results. Even if just to know if that's what I should use in the future.
Chris Fallin (Oct 13 2026 at 19:31):(as a side-note, we're still dialing in our optimal benchmark framework setup -- Nick has been working hard on this lately -- so please don't feel like this churn is you doing anything wrong! it's hard to get reliable reads sometimes with our suite)
Alex Crichton (Oct 13 2026 at 19:38):oh I'll defer this to you Chris lemme go find the PR...
fitzgen (he/him) (Oct 14 2026 at 15:09):neat: the WIP continuous benchmarking flagged the revision range that this landed in as a minor regression to compilation time (which, as per this discussion, was expected and accepted)
https://github.com/bytecodealliance/are-we-fast-yet/issues/1
fitzgen (he/him) (Oct 14 2026 at 15:17):(eventually the issues will be filed on wasmtime's repo, but I want to make sure things are generally working correctly before switching over)
Theo Butler (Oct 14 2026 at 15:39):nice!
Last updated: Oct 31 2026 at 23:03 UTC