fitzgen opened PR #14230 from fitzgen:alias-analysis-lattice to bytecodealliance:main:
There were two ways in which alias analysis's
LastStoresstate was not a proper lattice, which made the order we processed the worklist and calledLastStores::meetobservable:
We didn't have a single, canonical bottom value for the last store to a region. We were taking the first instruction in a block as an identifier for control-flow join points so that we would get different
MemoryLocs for different control-flow joins, which is necessary to avoid illegally forwarding a value loaded inside one control-flow join to a load in another, different control-flow join. However, this meant that we effectively had multiple bottom elements, which made the path we descended through the "lattice" observable. The solution here was to create a separateLastStoredataflow value that has a single, canonical bottom element, and a distinctMemoryVersionvalue that is the same asLastStorebut replaces its bottom value with a variant that identifies the associated control-flow join point. We useLastStorein ourLastStoreslattice, when we need a bottom element, and we useMemoryVersionin ourMemoryLockeys, to distinguish between different regions where we don't know anything about the contents of memory.We computed the observed-stores set while we computed the fixpoint of the initial
LastStoresinputs to each block. This was incorrect, however, because aLastStorescould transiently contain aLastStore::Instthat disappears in later iterations of the fixpoint, and which instructions do or don't transiently appear inLastStoresin that way depends on the order in which we callLastStores::meet. Therefore, observing stores while computing the fixpoint might or might not observe an instruction depending on the worklist processing order. The solution in this case was to only compute the observed-stores set after we've computed theLastStoresfixpoint, at which point there are no transientLastStore::Insts anymore.<!--
Please make sure you include the following information:
If this work has been discussed elsewhere, please include a link to that
conversation. If it was discussed in an issue, just mention "issue #...".Explain why this change is needed. If the details are in an issue already,
this can be brief.Our development process is documented in the Wasmtime book:
https://docs.wasmtime.dev/contributing-development-process.htmlPlease review the Bytecode Alliance's AI tool usage policy at
https://github.com/bytecodealliance/governance/blob/main/AI_TOOL_POLICY.mdPlease ensure all communication follows the code of conduct:
https://github.com/bytecodealliance/wasmtime/blob/main/CODE_OF_CONDUCT.md
-->
fitzgen requested wasmtime-compiler-reviewers for a review on PR #14230.
fitzgen requested wasmtime-core-reviewers for a review on PR #14230.
fitzgen requested cfallin for a review on PR #14230.
github-actions[bot] added the label cranelift on PR #14230.
fitzgen updated PR #14230.
:memo: cfallin submitted PR review:
Thanks for thinking through this carefully -- definitely very subtle.
Some comments as I read below, but more importantly I have a high-level feeling of "this may be getting too complex", and I'm wondering how much of it we really need for our stated purposes. In particular, what I am seeing is that there are a whole bunch of specific complexities coming out of the one core decision to support inter-block merged versions better than "unknown":
- The distinction between
MemoryVersionandLastStore, and conversion between them;- The pre-pass to find "merge blocks", and the subtle semantics around the lattice value that refers to this (external) table. In particular I'm having trouble thinking about "merge token"-version values that flow across other merge blocks (e.g. within a subregion) -- why is this correct?
- The observe-pass thing with
Option<HashMap>and confusing control flow.I wonder if we could benchmark the alternative where we fix the lattice the other way -- no
loc-keyed new version that occurs at a meet-point, just send that to bottom instead. That's a much smaller fix that results in an enormously simpler analysis, and I think it would get at least the straight-line trampoline cases we immediately care about, and many of the intra-block and even simple inter-block opts for e.g. GC fields with separate regions. The only case it misses is where we have a merge point, no store (to do a strong-update and overwrite the identity) but instead a load, and we can't RLE a second load. But actually even in that case we could cache the loaded result on a key with the load's instruction, no? So I'm not seeing where we would actually lose anything with the simpler analysis. Happy to see counterexamples or hear your reasoning on this of course!
:speech_balloon: cfallin created PR review comment:
I'm not sure it makes sense to even call
observewhenobserved_storesisNone-- doesn't that leave the door open to accidentally missing something (we callobserveat the wrong phase and it's just dropped on the floor)? It seems nicer to take a borrow to the underlyingFxHashMaphere, and then do the unwrap once when we do the post-pass to compute observed stores.
:speech_balloon: cfallin created PR review comment:
cut-off sentence? ("then the descent path through the...")
:speech_balloon: cfallin created PR review comment:
XXXusually denotes unfinished stuff to me -- maybe "N.B" or similar?
fitzgen commented on PR #14230:
Some comments as I read below, but more importantly I have a high-level feeling of "this may be getting too complex", and I'm wondering how much of it we really need for our stated purposes. In particular, what I am seeing is that there are a whole bunch of specific complexities coming out of the one core decision to support inter-block merged versions better than "unknown":
It is not that we want to support better merged versions than "unknown", it is that for correctness we need to distinguish between these different control-flow join points so that we get different
MemoryLocs for different control-flow join points and do not accidentally forward loads from one join point to loads in another join point. FWIW, this machinery to distinguish different join points already existed before I ever started touching alias analysis: this is the whole "if we don't agree on the last store, choose the first instruction in the block as a tombstone marker" logic inLastStores::meetthat has been there since alias analysis first landed. That is, this is not new complexity, and, if anything, I'd argue it is getting less complex by explicitly putting this case into the types / its own enum variant, rather than implicitly reusing a random instruction as the last store, and thereby removing incidental complexity to more-directly model the inherent complexity.* The distinction between `MemoryVersion` and `LastStore`, and conversion between them;Again, I believe this is inherent complexity, and is best dealt with by being honest about that and modeling it in the types:
- In one case, we cannot have multiple "bottom" elements (the lattice case)
- In another case, we must have multiple "bottom" elements for correctness (the
MemoryLockey for known memory values case)Trying to have one thing satisfy two conflicting requirements simultaneously is going to lead to bugs in practice (as we have already seen, by realizing that the worklist traversal order is observable).
* The pre-pass to find "merge blocks", and the subtle semantics around the lattice value that refers to this (external) table. In particular I'm having trouble thinking about "merge token"-version values that flow across other merge blocks (e.g. within a subregion) -- why is this correct?I'm not sure I follow the question. Merge blocks do not flow across other merge points. A block's merge block is the first predecessor (or itself) that is either (a) the function entry or (b) has
len(predecessors) != 1. If there was another merge point between a block and its merge block that is not the merge block, then that is a violation of the premise.* The observe-pass thing with `Option<HashMap>` and confusing control flow.We have some options here, and I don't particularly love any of them. We could do any of:
- take an
Optionand only callobservewhen it isSome(what we do now, with some.is_some()checks to avoid iterating over collections when all we do while iterating over them is call otherobserve_*methods), or- make functions compile-time generic over whether they are calling
observeor not, or- always call
observe_*methods even whenobserved_stores.is_none()to avoid the special-casing from (1) and its additional control flow, or- duplicate the definitions of all these functions to have one version that takes
Observationsand callsobserveand one that doesn't takeObservationsand doesn't call `observe.We are balancing code duplication, complexity, and whether or not we skip unnecessary work. I don't know how we can optimize all at once, though.
I wonder if we could benchmark the alternative where we fix the lattice the other way -- no
loc-keyed new version that occurs at a meet-point, just send that to bottom instead.As mentioned above, that would be unsound, because it means that we could have:
- block A, which is a control-flow join point, and therefore has bottom as its last store
- a load in block A
- block B, which is also a control-flow join point, and therefore also has bottom as its last store
- a load of the same address/type/etc... in block B
If we process block A first, then we will insert
MemoryLoc { last_store: None, // or whatever bottom is represented as address, offset, ty, extending_opcode, endianness, }as the key for the known value that we just loaded. Then, when we process block B, it will look up the exact same
MemoryLoc, and, finding an entry, will forward the known memory value here and replace the redundant load. But that is invalid! We are just RLE'ing values from unrelated control-flow join points to each other! (And things are identical if we do block B and then A).Zooming out, the only other way to avoid the merge blocks, without salting our
MemoryLockeys with some kind of token that similarly represents our scope, is to remove entries fromAliasAnalysis::mem_valueswhen they become invalid, which means switching to a scoped hash map of some sort (no big deal) but also changing the interface toAliasAnalysis. That interface change is a bit more annoying: it adds correctness requirements to block visitation order, rather than just enabling better optimization if done in dom tree pre-order, and requires additional push/poppubmethods that need to be called at the exact right times. That is all doable, but the changes leak out from just an internal implementation detail ofalias_analysis.rsand into new invariants to uphold and API calls to make for code that usesAliasAnalysis, and this doesn't really seem any simpler (especially since it becomes non-local) to me than what this PR proposes...
fitzgen edited a comment on PR #14230:
Some comments as I read below, but more importantly I have a high-level feeling of "this may be getting too complex", and I'm wondering how much of it we really need for our stated purposes. In particular, what I am seeing is that there are a whole bunch of specific complexities coming out of the one core decision to support inter-block merged versions better than "unknown":
It is not that we want to support better merged versions than "unknown", it is that for correctness we need to distinguish between these different control-flow join points so that we get different
MemoryLocs for different control-flow join points and do not accidentally forward loads from one join point to loads in another join point. FWIW, this machinery to distinguish different join points already existed before I ever started touching alias analysis: this is the whole "if we don't agree on the last store, choose the first instruction in the block as a tombstone marker" logic inLastStores::meetthat has been there since alias analysis first landed. That is, this is not new complexity, and, if anything, I'd argue it is getting less complex by explicitly putting this case into the types / its own enum variant, rather than implicitly reusing a random instruction as the last store, and thereby removing incidental complexity to more-directly model the inherent complexity.
- The distinction between
MemoryVersionandLastStore, and conversion between them;Again, I believe this is inherent complexity, and is best dealt with by being honest about that and modeling it in the types:
- In one case, we cannot have multiple "bottom" elements (the lattice case)
- In another case, we must have multiple "bottom" elements for correctness (the
MemoryLockey for known memory values case)Trying to have one thing satisfy two conflicting requirements simultaneously is going to lead to bugs in practice (as we have already seen, by realizing that the worklist traversal order is observable).
- The pre-pass to find "merge blocks", and the subtle semantics around the lattice value that refers to this (external) table. In particular I'm having trouble thinking about "merge token"-version values that flow across other merge blocks (e.g. within a subregion) -- why is this correct?
I'm not sure I follow the question. Merge blocks do not flow across other merge points. A block's merge block is the first predecessor (or itself) that is either (a) the function entry or (b) has
len(predecessors) != 1. If there was another merge point between a block and its merge block that is not the merge block, then that is a violation of the premise.
- The observe-pass thing with
Option<HashMap>and confusing control flow.We have some options here, and I don't particularly love any of them. We could do any of:
- take an
Optionand only callobservewhen it isSome(what we do now, with some.is_some()checks to avoid iterating over collections when all we do while iterating over them is call otherobserve_*methods), or- make functions compile-time generic over whether they are calling
observeor not, or- always call
observe_*methods even whenobserved_stores.is_none()to avoid the special-casing from (1) and its additional control flow, or- duplicate the definitions of all these functions to have one version that takes
Observationsand callsobserveand one that doesn't takeObservationsand doesn't call `observe.We are balancing code duplication, complexity, and whether or not we skip unnecessary work. I don't know how we can optimize all at once, though.
I wonder if we could benchmark the alternative where we fix the lattice the other way -- no
loc-keyed new version that occurs at a meet-point, just send that to bottom instead.As mentioned above, that would be unsound, because it means that we could have:
- block A, which is a control-flow join point, and therefore has bottom as its last store
- a load in block A
- block B, which is also a control-flow join point, and therefore also has bottom as its last store
- a load of the same address/type/etc... in block B
If we process block A first, then we will insert
MemoryLoc { last_store: None, // or whatever bottom is represented as address, offset, ty, extending_opcode, endianness, }as the key for the known value that we just loaded. Then, when we process block B, it will look up the exact same
MemoryLoc, and, finding an entry, will forward the known memory value here and replace the redundant load. But that is invalid! We are just RLE'ing values from unrelated control-flow join points to each other! (And things are identical if we do block B and then A).Zooming out, the only other way to avoid the merge blocks, without salting our
MemoryLockeys with some kind of token that similarly represents our scope, is to remove entries fromAliasAnalysis::mem_valueswhen they become invalid, which means switching to a scoped hash map of some sort (no big deal) but also changing the interface toAliasAnalysis. That interface change is a bit more annoying: it adds correctness requirements to block visitation order, rather than just enabling better optimization if done in dom tree pre-order, and requires additional push/poppubmethods that need to be called at the exact right times. That is all doable, but the changes leak out from just an internal implementation detail ofalias_analysis.rsand into new invariants to uphold and API calls to make for code that usesAliasAnalysis, and this doesn't really seem any simpler (especially since it becomes non-local) to me than what this PR proposes...
cfallin commented on PR #14230:
It is _not_ that we want to support _better_ merged versions than "unknown", it is that for correctness we _need_ to distinguish between these different control-flow join points so that we get different
MemoryLocs for different control-flow join points and do not accidentally forward loads from one join point to loads in another join point. FWIW, this machinery to distinguish different join points already existed before I ever started touching alias analysis: this is the whole "if we don't agree on the last store, choose the first instruction in the block as a tombstone marker" logic inLastStores::meetthat has been there since alias analysis first landed. That is, this is not new complexity, and, if anything, I'd argue it is getting _less_ complex by explicitly putting this case into the types / its own enum variant, rather than implicitly reusing a random instruction as the last store, and thereby removing incidental complexity to more-directly model the inherent complexity.Yes, agreed that it was a pre-existing issue.
There is something I still don't understand though. It seems that the above is assuming that we would still associate known values with locs that have "bottom" locations (i.e., merged locations). I was assuming (and I think had implied?) that we would simply not associate values with such locations.
That's far simpler and avoids all of this machinery, no?
fitzgen commented on PR #14230:
There is something I still don't understand though. It seems that the above is assuming that we would still associate known values with locs that have "bottom" locations (i.e., merged locations). I was assuming (and I think had implied?) that we would simply _not_ associate values with such locations.
Ah, I see what you're suggesting now, I had misunderstood your comment.
I prototyped this, not adding entries to
AliasAnalysis::mem_valueswhen theMemoryLockey is the bottom element, and we get the following filetest failures:<details>
FAIL /Users/n.fitzgerald/scratch/wasmtime-alias-analysis-lattice/cranelift/filetests/filetests/egraph/alias_analysis.clif: optimize Caused by: filecheck failed for function on line 5: #0 check: v3 = load.i64 region0 v0 #1 check: store v0, v3 #2 check: v7 = load.i64 v0 #3 check: return v7 > function %f(i64) -> i64 fast { > region0 = 0 "heap" > > block0(v0: i64): > v3 = load.i64 region0 v0 ^~~~~~~~~~~~~~~~~~~~~~~~ Matched #0: \bv3 = load\.i64 region0 v0\b Missed #1: \bstore v0, v3\b > v4 = load.i64 region0 v0 > v5 = band v3, v4 > store v0, v5 > v6 = load.i64 v3 > v7 = load.i64 v6 > return v7 > } FAIL /Users/n.fitzgerald/scratch/wasmtime-alias-analysis-lattice/cranelift/filetests/filetests/alias/dead-store-then-idempotent-store.clif: optimize Caused by: compilation of function on line 8 does not match the text expectation --- expected +++ actual @@ -3,5 +3,6 @@ block0(v0: i64): v1 = load.i32 notrap aligned region0 v0 + store notrap aligned region0 v1, v0 return } This test assertion can be automatically updated by setting the CRANELIFT_TEST_BLESS=1 environment variable when running this test. FAIL /Users/n.fitzgerald/scratch/wasmtime-alias-analysis-lattice/cranelift/filetests/filetests/alias/issue-13508.clif: alias-analysis Caused by: filecheck failed for function on line 9: #0 check: v5 = load.i32 region0 v0 #1 check: v6 -> v5 #2 check: v8 = load.i32 region0 v0 > function %f(i64, i64, i64) -> i32, i32, i32 apple_aarch64 { > region0 = 2 "vmctx" > > block0(v0: i64, v1: i64, v2: i64): > v3 = iconst.i64 -1 > store notrap v3, v0 ; v3 = -1 > v4 = atomic_cas v2, v0, v0 > v5 = load.i32 region0 v0 ^~~~~~~~~~~~~~~~~~~~~~~~ Matched #0: \bv5 = load\.i32 region0 v0\b Missed #1: \bv6 \-> v5\b > v6 = load.i32 region0 v0 > v7 = iconst.i32 42 > store notrap v7, v1 ; v7 = 42 > v8 = load.i32 region0 v0 > return v5, v6, v8 > } FAIL /Users/n.fitzgerald/scratch/wasmtime-alias-analysis-lattice/cranelift/filetests/filetests/alias/idempotent-store.clif: optimize Caused by: compilation of function on line 6 does not match the text expectation --- expected +++ actual @@ -3,5 +3,6 @@ block0(v0: i64): v1 = load.i32 region0 v0+8 + store region0 v1, v0+8 return } This test assertion can be automatically updated by setting the CRANELIFT_TEST_BLESS=1 environment variable when running this test. FAIL /Users/n.fitzgerald/scratch/wasmtime-alias-analysis-lattice/cranelift/filetests/filetests/alias/multiple-blocks.clif: alias-analysis Caused by: filecheck failed for function on line 7: #0 check: v4 -> v3 > function %f0(i64 vmctx, i32) -> i32 fast { Missed #0: \bv4 \-> v3\b > gv0 = vmctx > gv1 = load.i64 notrap aligned readonly gv0+8 > > block0(v0: i64, v1: i32): > v2 = load.i64 notrap aligned readonly v0+8 > v3 = load.i32 v2+8 > brif v2, block2, block1 > > block1: > v4 = load.i32 v2+8 > jump block3(v4) > > block2: > jump block3(v3) > > block3(v5: i32): > return v5 > } FAIL /Users/n.fitzgerald/scratch/wasmtime-alias-analysis-lattice/cranelift/filetests/filetests/alias/endianness.clif: optimize Caused by: compilation of function on line 33 does not match the text expectation --- expected +++ actual @@ -3,5 +3,6 @@ block0(v0: i64): v1 = load.i16 big region0 v0 - return v1, v1 + v2 = load.i16 big region0 v0 + return v1, v2 } This test assertion can be automatically updated by setting the CRANELIFT_TEST_BLESS=1 environment variable when running this test. FAIL /Users/n.fitzgerald/scratch/wasmtime-alias-analysis-lattice/cranelift/filetests/filetests/alias/simple-alias.clif: alias-analysis Caused by: filecheck failed for function on line 9: #0 check: v5 -> v3 #1 check: v7 -> v6 > function %f0(i64 vmctx, i32) -> i32, i32, i32, i32 fast { Missed #0: \bv5 \-> v3\b > gv0 = vmctx > gv1 = load.i64 notrap aligned readonly gv0+8 > sig0 = (i64 vmctx) fast > fn0 = %g sig0 > > block0(v0: i64, v1: i32): > v2 = load.i64 notrap aligned readonly v0+8 > v3 = load.i32 v2+8 > v5 = load.i32 v2+8 > call fn0(v0) > v6 = load.i32 v2+8 > v7 -> v6 > return v3, v5, v6, v7 > } FAIL /Users/n.fitzgerald/scratch/wasmtime-alias-analysis-lattice/cranelift/filetests/filetests/alias/join-order-independence.clif: optimize Caused by: compilation of function on line 9 does not match the text expectation --- expected +++ actual @@ -14,5 +14,6 @@ jump block4 block4: - return v3, v3 + v4 = load.i32 notrap aligned region0 v0 + return v3, v4 } This test assertion can be automatically updated by setting the CRANELIFT_TEST_BLESS=1 environment variable when running this test. FAIL /Users/n.fitzgerald/scratch/wasmtime-alias-analysis-lattice/cranelift/filetests/filetests/alias/join-single-predecessor.clif: optimize Caused by: compilation of function on line 5 does not match the text expectation --- expected +++ actual @@ -16,9 +16,10 @@ jump block4 block4: + v11 = load.i32 notrap aligned region0 v0 store.i32 notrap aligned region0 v3, v0 brif.i32 v1, block1, block5 block5: - return v10, v10 + return v10, v11 } This test assertion can be automatically updated by setting the CRANELIFT_TEST_BLESS=1 environment variable when running this test. FAIL /Users/n.fitzgerald/scratch/wasmtime-alias-analysis-lattice/cranelift/filetests/filetests/alias/fence-fallback-across-join.clif: optimize Caused by: compilation of function on line 5 does not match the text expectation --- expected +++ actual @@ -12,5 +12,6 @@ jump block3 block3: - return v3, v3 + v4 = load.i32 notrap aligned region0 v0 + return v3, v4 } This test assertion can be automatically updated by setting the CRANELIFT_TEST_BLESS=1 environment variable when running this test. FAIL /Users/n.fitzgerald/scratch/wasmtime-alias-analysis-lattice/cranelift/filetests/filetests/alias/check-unset-reset-flag.clif: optimize Caused by: compilation of function on line 5 does not match the text expectation --- expected +++ actual @@ -4,6 +4,7 @@ block0(v0: i64, v1: i32): v2 = load.i64 notrap aligned region0 v0 trapz v2, user42 + store notrap aligned region0 v2, v0 v4 = iadd v1, v1 return v4 } This test assertion can be automatically updated by setting the CRANELIFT_TEST_BLESS=1 environment variable when running this test. FAIL /Users/n.fitzgerald/scratch/wasmtime-alias-analysis-lattice/cranelift/filetests/filetests/alias/forward-across-trap.clif: optimize Caused by: compilation of function on line 7 does not match the text expectation --- expected +++ actual @@ -4,5 +4,6 @@ block0(v0: i64, v1: i32): v2 = load.i64 notrap aligned region0 v0 trapz v1, user42 - return v2, v2 + v3 = load.i64 notrap aligned region0 v0 + return v2, v3 } This test assertion can be automatically updated by setting the CRANELIFT_TEST_BLESS=1 environment variable when running this test. 1289 tests Error: 12 failures</details>
A lot of those look like regressions to pretty simple cases that I'd really expect to be handled.
We also get the following
disastest failures:<details>
failures: ---- ./tests/disas/dynamic-memory-yes-spectre-access-same-index-different-offsets.wat ---- failed to run tests "./tests/disas/dynamic-memory-yes-spectre-access-same-index-different-offsets.wat" Caused by: Did not get the expected CLIF translation: --- expected +++ actual @@ -18,15 +18,22 @@ @0047 v7 = iadd v6, v3 @0047 v9 = select_spectre_guard v5, v8, v7 ; v8 = 0 @0047 v10 = load.i32 little region4 v9 +@004c v12 = load.i64 notrap aligned region3 v0+64 +@004c [message truncated]
fitzgen commented on PR #14230:
The more I think about it, the more the scoped hash map I mentioned above becomes more appealing, since "scope" and where it is valid to reuse a known memory value is really what we are trying to capture here...
fitzgen updated PR #14230.
fitzgen commented on PR #14230:
Just pushed a commit with the scope API version of the fix discussed above. I like this much better: we can incrementally compute extents (merge tokens) in a very simple, straightforward way and we can get rid of the post-pass to compute each block's extent and we can also get rid of the
LastStore/MemoryVersionsplit.
:thumbs_up: cfallin submitted PR review:
This looks good for me -- thanks for the patience and rethinking here; I like where we ended up as well!
:thumbs_up: cfallin submitted PR review:
This looks good to me -- thanks for the patience and rethinking here; I like where we ended up as well!
cfallin added PR #14230 Make LastStores a proper lattice to the merge queue.
:check: cfallin merged PR #14230.
cfallin removed PR #14230 Make LastStores a proper lattice from the merge queue.
Last updated: Sep 20 2026 at 19:05 UTC