gaaraw opened issue #11296:
Describe the bug
Hello, I caught the following performance anomalies while using
wasmtime. The specific performance is as follows:<img width="414" height="127" alt="Image" src="https://github.com/user-attachments/assets/b2447f47-b354-431e-97b9-02562124977d" />
The data is in seconds, and each data is the result of ten executions and averages.
Test Case
Steps to Reproduce
# wasm2wat or wat2wasm wasm2wat -f test_case.wasm -o test_case.wat wat2wasm test_case.wat -o test_case.wasm # Execute the wasm file and collect data perf stat -r 10 -e 'task-clock' /path/to/wasmer run test_case.wasm perf stat -r 10 -e 'task-clock' /path/to/wasmtime test_case.wasm perf stat -r 10 -e 'task-clock' /path/to/wasmedge --enable-jit test_case.wasm perf stat -r 10 -e 'task-clock' /path/to/build_fast_jit/iwasm test_case.wasm perf stat -r 10 -e 'task-clock' /path/to/build_llvm_jit/iwasm test_case.wasmExpected and actual Results
test_case.wasmcauses large execution time differences between several runtimes, withwasmtimeandwasmerbeing particularly pronounced. The execution time ofwasmeris about 5x-6x, and the execution time ofwasmtimeis about 7x-9x.
I did an analysis oftase_case.watand found that when I deleted the code like the following for 50-52 lines, the execution time is as shown inmodified.wasm. At this point, the results of each runtime are relatively normal. From this I think the following directives may be causing performance anomalies in both runtime tools.(drop (memory.grow (i32.const 1)))Versions and Environment
The runtime tools are all built on release and use JIT mode.
Wasmeruses theCraneliftbackend, and the execution time of thellvmbackend for this test case is basically the same as that ofCranelift.
- wasmer:wasmer 6.0.1
- wasmtime:wasmtime 35.0.0 (9c2e6f17c 2025-06-17)
- wasmedge:version 0.14.1
- WAMR:iwasm 2.4.0
- wabt:1.0.27
- llvm:18.1.8
- Host OS:Ubuntu 22.04.5 LTS x64
- CPU:12th Gen Intel® Core™ i7-12700 × 20
- rustc:rustc 1.87.0 (17067e9ac 2025-05-09)
binary: rustc
commit-hash: 17067e9ac6d7ecb70e50f92c1944e545188d2359
commit-date: 2025-05-09
host: x86_64-unknown-linux-gnu
release: 1.87.0
LLVM version: 20.1.1Extra Info
I also submitted an issue about the phenomenon to
wasmer.
If you need any other relevant information, please let me know and I will do my best to provide it. Looking forward to your reply! Thank you!
gaaraw added the bug label to Issue #11296.
alexcrichton commented on issue #11296:
Can you expand more on what this benchmark is doing? It seems like the time taken is primarily spent in
memory.grow, but what is this benchmarking about memory growth? Is this just executingmemory.growin a loop?Can you test other engines such as V8 via Node.js? Locally that has about the same runtime as Wasmtime's main branch for me.
Additionally the return value of
memory.growis being ignored, are you sure this is behaving the same across all runtimes? For example Wasmtime allows growing memory to the 4G limit of 32-bit linear memories by default, but other runtimes may not. I would recommend double-checking that all runtimes are actually doing the same thing before comparing runtimes as otherwise this could be an apples-to-oranges comparison.
alexcrichton removed the bug label from Issue #11296.
gaaraw commented on issue #11296:
Thanks for your reply!
Can you expand more on what this benchmark is doing?
I'm fuzzing the wasm runtime tools for performance pressure-related, so the program is the result of a mutation. If you want to trace the origin of the program, it comes from the LLVM test suite.
Can you test other engines such as V8 via Node.js?
V8 behaves as follows:
<img width="1089" height="213" alt="Image" src="https://github.com/user-attachments/assets/7a9777c5-b313-4c1c-b1ce-ce9da56971e6" />
<details>
<summary>run_wasi.js</summary>const fs = require('fs'); const path = require('path'); const { WASI } = require('node:wasi'); const { argv, env } = process; const wasi = new WASI({ version: 'preview1', args: argv, env, preopens: { '/': './' } }); (async () => { const wasmPath = path.resolve(__dirname, 'test_case.wasm'); const wasmBuffer = fs.readFileSync(wasmPath); const module = await WebAssembly.compile(wasmBuffer); const instance = await WebAssembly.instantiate(module, { wasi_snapshot_preview1: wasi.wasiImport }); wasi.start(instance); })();</details>
Additionally the return value of memory.grow is being ignored, are you sure this is behaving the same across all runtimes?
I analyzed the program further and found that it didn't seem to be related to the memory limits of the individual runtime tools. I set the maximum number of pages in memory of wasmedge to 65536 using the command line parameter
--memory-page-limit, and the result did not change.Analysis
test_case.watshowed that the local variable decreased by 0. When it is equal to 64, it will exit the loop. (The source program did not initialize 0, I set address 1056 to 0 during the analysis, so that the local variable 0 was initialized to 0. Does not affect the results.) So the cycle was done about 2<sup>32</sup> times. In fact, from the secondmemory.growonwards, the grow command will not succeed, because I only increase the maximum value of memory by 1.(memory (; 0; ) 258 259).In summary, the next pressure on the runtime tools are to deal with a large number of failed
memory.growinstructions, which is why the time of each runtime tool increases.
cfallin commented on issue #11296:
@gaaraw thanks for the issue; I wanted to note a meta-point: please do not post information in bugs as screenshots (your initial table and your later terminal screenshot). Text-as-images means the text can't be searched, it isn't visible by default in many email clients (including mine), its appearance cannot be adjusted (e.g. font size or color for anyone with visual difficulties), and it is not readable by most screen-reader or other accessibility software. It is much friendlier to us maintainers and to anyone who wants to find information in the future if you take a few minutes to copy-and-paste the text directly. Thanks!
Last updated: Dec 06 2025 at 07:03 UTC