cfallin requested alexcrichton for a review on PR #12341.
cfallin opened PR #12341 from cfallin:icmp-40.0 to bytecodealliance:release-40.0.0:
- Cranelift: x64: fix user-controlled recursion in cmp emission.
We had a set of rules introduced in #11097 that attempted to optimize the case of testing the result of an
icmpfor a nonzero value. This allowed optimization of, for example,(((x == 0) == 0) == 0 ...)to a single level, eitherx == 0orx != 0depending on even/odd nesting depth.Unfortunately this kind of recursion in the backend has a depth bounded only by the user input, hence creates a DoS vulnerability: the wrong kind of compiler input can cause a stack overflow in Cranelift at compilation time. This case is reachable from Wasmtime's Wasm frontend via the
i32.eqzoperator (for example) as well.Ideally, this kind of deep rewrite is best done in our mid-end optimizer, where we think carefully about bounds for recursive rewrites. The left-hand sides for the backend rules should really be fixed shapes that correspond to machine instructions, rather than ad-hoc peephole optimizations in their own right.
This fix thus simply removes the recursion case that causes the blowup. The patch includes two tests: one with optimizations disabled, showing correct compilation (without the fix, this case fails to compile with a stack overflow), and one with optimizations enabled, showing that the mid-end properly cleans up the nested expression and we get the expected one-level result anyway.
- Preserve codegen on branches.
This change works by splitting a rule so that the entry point used by
briflowering can still peel off one layer oficmpand emit it directly, without entering the unbounded structural recursion.It also adds a mid-end rule to catch one case that we were previously catching in the backend only:
fcmp(...) != 0.<!--
Please make sure you include the following information:
If this work has been discussed elsewhere, please include a link to that
conversation. If it was discussed in an issue, just mention "issue #...".Explain why this change is needed. If the details are in an issue already,
this can be brief.Our development process is documented in the Wasmtime book:
https://docs.wasmtime.dev/contributing-development-process.htmlPlease ensure all communication follows the code of conduct:
https://github.com/bytecodealliance/wasmtime/blob/main/CODE_OF_CONDUCT.md
-->
cfallin requested wasmtime-compiler-reviewers for a review on PR #12341.
alexcrichton submitted PR review.
cfallin has enabled auto merge for PR #12341.
cfallin merged PR #12341.
Last updated: Jan 29 2026 at 13:25 UTC