Paul Osborne (Jun 26 2026 at 16:47): Paul Osborne (Jun 26 2026 at 16:47):I captured these results prior to v46 but assume they are largely in line; moving to a newer wasmtime shows largely positive results. Crypto I knew would improve from 9a6aa39c94fdce8420e1694c33bd4c519d55769e awhile back.
The biggest slowdowns measured were on the following benchmarks:
I've started to work on tooling for sightglass to aid in automated bisection of perf regressions but I don't have conclusive results. The benchmarking in this report uses a config different than the default (for both engine versions) but I have shown the differences hold with default configurations as well.
Paul Osborne (Jun 26 2026 at 16:47):wasmtime-v39-to-v45-ugprade-benchmarks.html
Paul Osborne (Jun 26 2026 at 16:52):Should add: x86_64 linux host for these benchmarks.
Paul Osborne (Jul 23 2026 at 16:37):@Chris Fallin @fitzgen (he/him) I was able to get my bisection on top of sighglass to isolate the change which introduced perf regressions (and improvements) which was the regalloc2 bump in https://github.com/bytecodealliance/wasmtime/pull/12596
It isn't surprising that this might move around perf, of course, but wanted to share the results to see if it shows anything surprising. Summary notes (note that my initial categorization of sieve was backwards and quicksort didn't show up as having a significant difference in this run):
Slower:
Faster:
Paul Osborne (Jul 23 2026 at 16:37):regalloc2-upgrade-performance-delta.html
fitzgen (he/him) (Jul 23 2026 at 16:44):wait -- looking at the html report, it seems like more than just two got faster to me?
Chris Fallin (Jul 23 2026 at 16:58):Interesting -- that PR pulled in this RA2 change which alters the way that register probing works, for simplicity / compile-time efficiency. At the time we hypothesized it shouldn't matter but it might be worth revisiting
Chris Fallin (Jul 23 2026 at 16:59):It would be good to verify that the slowdowns are legitimate by looking at disassemblies of hot blocks/functions -- do we get an additional spill somewhere because the probes in a different order happened to stack costs in just such a way that that was the best option?
Chris Fallin (Jul 23 2026 at 17:00):(these sorts of things are hard to "fix" fwiw because at the local maximum that we're more-or-less at, regalloc is a pile of heuristics to solve an NP-hard problem; different search orders might find better answers on some problems and worse on others, I guess as those results show)
Paul Osborne (Jul 23 2026 at 17:24):wait -- looking at the html report, it seems like more than just two got faster to me?
Yes, the summary I was just listing ones >5% difference. Didn't mean to gloss over the smaller deltas.
Paul Osborne (Jul 23 2026 at 18:57):Comparing the cwasm for gimli, it does appear that the perf regression is related to register spill. The hot loop has a few more instructions with some additional stack stores and loads and an additional 16 bytes allocated in the function prologue.
Chris Fallin (Jul 23 2026 at 22:38):I don't have cycles to look at this any time soon but if you're interested, you could look at the PR I linked above and the discussion about the heuristic -- IIRC we relaxed/modified the iteration order to avoid the need for an "Nth next bit" rather than "next set bit" when we shifted to a bitmask approach. It's probably not bad to switch back, just takes a bit of careful thought
Paul Osborne (Jul 23 2026 at 23:23):Thanks Chris, totally understood. This may not end up being a big priority for us to change either; we had just noticed the change in perf when doing upgrades and wanted to undertsand the source so we could try to reason about how it might impact more diverse workloads.
It seems like, at least in this context, the ordering imposed before was likely not intentional either way, so I'm not considering this as a bug. I may try to see if I come up with a heuristic to improve codegen; I have a hack impl that shows some promise but I need to work with it more and test it more broadly before it goes anywhere.
Chris Fallin (Jul 23 2026 at 23:28):yes, that's right, the intuitive explanation is something like: the randomized order in which regalloc tries to allocate registers shifted (new, simpler heuristic), and because of the different order, different liveranges conflicted and different decisions were made. Some things got faster, some things got slower as a result
Last updated: Jul 29 2026 at 05:03 UTC