alexcrichton opened issue #14047:
For Wasmtime's original 46.0.0 release it originally contained a crop of changes where bulk operations (e.g.
memory.copy) were updated to chunk the operation internally when fuel/epochs were enabled with preemption checks in the middle. The goal of this translation was to ensure that a very largememory.copy, such as moving 2GiB of data, wouldn't represent an uninterrputible chunk of work the guest could perform. This was later discovered to be the source of a number of possible security vulnerabilities with epoch callbacks, and this behavior was reverted in https://github.com/bytecodealliance/wasmtime/pull/14045 to only check for epochs/fuel at the start of an operation instead of during an operation. This results in a doc block on epochs/fuel, however, that large bulk operations represent uninterruptible chunks of work where the only way to limit them is to limit the total size of a linear memory itself.This isn't a great situation and we should probably somehow figure out how to add preemption checks in the middle. This'll need to be done relatively carefully to avoid reintroducing the problems that were fixed in #14045.
fitzgen added the wasmtime:api label to Issue #14047.
adamrk commented on issue #14047:
What about this rough idea?:
We break up the bulk operation into a sequence of smaller bulk operations so that the result of each smaller operation leaves the store in a valid state and then only inject preemption points between the smaller operations. The size of the smaller operations are heuristically determined depending on the preemption method.
- For fuel, we store the last value
fuelthat was passed toset_fueland assume the same value will be passed in the future, so we pick the smaller bulk operations to cost roughlyfuel. This could additionally be adjusted in the middle of the loop if we see that different values were passed toset_fuelat preemption points.- For epochs, we store the previous duration between preemption points and use that for our heuristic. I guess we'd need to have a config for how to translate time into bulk operation size for this. I'm also not sure what we'd do if the epoch hasn't been implemented yet - maybe that would need to be an additional config. This one could also be adjusted in the middle of the loop if we have some way to check how much we've overshot an epoch increment.
alexcrichton commented on issue #14047:
My inclination is that the hardest part here is going to be leaving the store in a valid state after each operation. For the exact fuel/epoch check I think it's fine to kind of do "whatever" in a sense. Previously byte-based operations were chunked by 128MiB, and per-element operations (e.g.
array.copy) were checking fuel/epochs at the start of each loop iteration. Keeping a fixed-chunk size for byte operations and perhaps using a fixed-chunk size for per-element operations I think is fine w.r.t. checks/fuel/etc.For keeping the store valid, there's a few concerns to handle. In theory most of these issues are derivative of https://github.com/bytecodealliance/wasmtime/issues/14048 where the callback is too powerful, but I'm inclined to say that were possible we should ignore that issue in codegen and assume that modifications can happen to be a bit more defensive. If we assume that, then handling each operation is one of two buckets:
- Operations where even in the middle everything is valid -- this is something like
array.copyormemory.init. For these I think we mostly just need to be careful to not preserve pointers across epoch checks and instead only preserve indices to re-derive pointers/etc on each chunk that's done.- Operations where in the middle the store is invalid -- this is
table.growfor non-nullable tables, initializing tables on instantiation for non-nullable tables, and array constructors likearray.{new,new_default,new_elem,new_data}where uninitialized GC elements may be present.The first bucket can be codegen'd reasonable assuming store modifications in the middle, but for the second bucket it's not so easy. We've considered the idea historically of having a len/capacity separatation for tables in the past (currently they just have length mostly) which could solve this, where a
table.growis sort of split up into smaller growths. Such a scheme would require something different for arrays though because we'd have to adjust in-heap GC representations to handle pre-initialized state.I haven't thought too too hard about this latter bucket and preventing store modifications in the middle of these operations. It might just end up being that assuming store modifications can happen is totally unreasonable. If that's the case though then we'd want t oaddress #14048 first and probably add some sort of defensive checks in the codegen to double-check that preemption doesn't change anything.
adamrk commented on issue #14047:
For the chunking, I was imagining that we perform a full bulk operation (with a smaller size) so that the store is left in a valid state even for the instructions in the second bucket. Almost as if the original Wasm was rewritten to perform a bunch of smaller
table.grows in a loop. I guess the main difficulty there would be rolling back the earlier operations if a later one fails? And of course the cost of memory allocation wouldn't be amortized because we do many smaller increases.
alexcrichton commented on issue #14047:
Yeah rolling back
table.growisn't possible, and additionally GC arrays can't grow at all, so we can't easily chunk those up in a similar manner :(
alexcrichton commented on issue #14047:
One other issue we're running into today with the current implementation which is related to this issue itself but not entirely the same: fuel is consumed up-front for
table.growbefore the actual operation happens. That's an operation that's allowed to gracefully fail (return -1) but running out of fuel traps. This means that(table.grow (i32.const -1))executes normally without fuel but traps with fuel. Basically the ahead-of-time fuel consumption doesn't model the case where fuel only gets charged if the growth is successful.This is currently leading to differential fuzz-failures on OSS-Fuzz right now where, for example, Wasmtime with "infinite" fuel traps and Wasmtime without fuel succeeds.
alexcrichton edited a comment on issue #14047:
One other issue we're running into today with the current implementation which is related to this issue itself but not entirely the same: fuel is consumed up-front for
table.growbefore the actual operation happens. That's an operation that's allowed to gracefully fail (return -1) but running out of fuel traps. This means that(table.grow (i32.const -1))executes normally without fuel but traps with fuel. Basically the ahead-of-time fuel consumption doesn't model the case where fuel should only get charged if the growth is successful.This is currently leading to differential fuzz-failures on OSS-Fuzz right now where, for example, Wasmtime with "infinite" fuel traps and Wasmtime without fuel succeeds.
alexcrichton added the wasmtime:fuel label to Issue #14047.
Last updated: Aug 30 2026 at 09:07 UTC