Stream: git-wasmtime

Topic: wasmtime / issue #12199 Enhance `OutOfMemory` error with ...

Wasmtime GitHub notifications bot (Dec 22 2025 at 19:17):

alexcrichton opened issue #12199:

There's some more discussion here, but I personally think we should strive to extend this error with the contextual number of bytes that were attempted to be allocated. Implementation-wise this will be tricky, however, so it'll require deliberate design.

Wasmtime GitHub notifications bot (Dec 22 2025 at 19:17):

alexcrichton added the wasmtime:api label to Issue #12199.

Wasmtime GitHub notifications bot (Jan 13 2026 at 18:29):

fitzgen commented on issue #12199:

How does this sound?

• Bitpack the size and layout into the &OutOfMemory pointer.
• Add a pub fn OutOfMemory::layout(&self) -> Layout method that reads the bitpacked data.
• Do _not_ have a public ctor for OutOfMemory (and I guess also remove the clone/copy derives?)
• Instead have pub fn Error::out_of_memory(layout: Layout) -> Error, which does the bitpacking and wrapping up into an Error.

So with this setup, you are generally only ever dealing with &OutOfMemory or an Error wrapping an OutOfMemory, but not an owned OutOfMemory that isn't inside an Error. So you can still ask things like error.is<OutOfMemory>() and do error.downcast_ref::<OutOfMemory>().unwrap().layout() and all that, which is / will be (probably?) the most common uses of OutOfMemory.

However, the one tricky thing is that error.downcast::<OutOfMemory>().unwrap() will give you an owned OutOfMemory, but because it isn't a pointer into an Error, you will lose the bitpacked layout info, and so OutOfMemory::layout will need to return an option or a zero-sized layout or something. Perhaps even worse is that we have no guarantee that taking a reference to an owned OutOfMemory will produce a pointer that won't "accidentally" have non-zero data in our bitpacked areas, so we could just return arbitrary (and incorrect) layouts in this case.

FWIW, we could also avoid this one last edge case by moving the layout method to Error, so you can only get the layout info from an Error that wraps an OutOfMemory (presumably; we'd still have an option for the non-OOM Error case, but we wouldn't have to worry about arbitrary/incorrect layout data). This is probably the way to go, despite the API being a bit funkier...

Wasmtime GitHub notifications bot (Jan 13 2026 at 18:36):

fitzgen commented on issue #12199:

However, the one tricky thing is that error.downcast::<OutOfMemory>().unwrap() will give you an owned OutOfMemory

We could also just make error.downcast::<OutOfMemory>() always fail too. This might actually be the best option, since that way we can use the bitpacked layout information inside the display and debug impls for OutOfMemory, which having the layout accessors on Error wouldn't allow.

Wasmtime GitHub notifications bot (Jan 13 2026 at 18:39):

alexcrichton commented on issue #12199:

Wouldn't we want a public constructor of OutOfMemory for collection-based APIs which'd return Result<T, OutOfMemory>?

Wasmtime GitHub notifications bot (Jan 13 2026 at 18:41):

fitzgen commented on issue #12199:

Wouldn't we want a public constructor of OutOfMemory for collection-based APIs which'd return Result<T, OutOfMemory>?

Yeah I just ran into this as I started investigating further.

New idea: make OutOfMemory a newtype wrapper around Error where the wrapped Error is guaranteed to be bitpacked with layout info and have a discriminant appropriately set to mark that it is not a boxed dyn error (as it already does today).

Wasmtime GitHub notifications bot (Jan 13 2026 at 18:46):

alexcrichton commented on issue #12199:

I may not quite be following something, or I think I'm missing something? I would expect OutOfMemory to contain NonZero<usize> and then Error's internal NonNull<T> which it contains would be transmuted to NonZero<usize> for getting &OutOfMemory or something like that. I also don't think we can deal with a full Layout (size + align) and can only deal with the size for now, I don't know how to bitpack the alignment into the payload.

Wasmtime GitHub notifications bot (Jan 13 2026 at 18:58):

fitzgen commented on issue #12199:

I would expect OutOfMemory to contain NonZero<usize> and then Error's internal NonNull<T> which it contains would be transmuted to NonZero<usize> for getting &OutOfMemory or something like that.

Yeah, this is basically equivalent to my newtype idea.

I also don't think we can deal with a full Layout (size + align) and can only deal with the size for now, I don't know how to bitpack the alignment into the payload.

I think you are correct. I was thinking that, because the alloc size is limited to isize::MAX we had half the bits for the layout, but we have half the usize range which is only one bit. Mixing up domains.

Wasmtime GitHub notifications bot (Jan 13 2026 at 19:02):

alexcrichton commented on issue #12199:

On second thought -- I don't think that &NonNull<T> can be safely coerced to &NonZero<usize>. What with provenance and CHERI I suspect that's an explicitly unsupported operation, and that might sink this entire endeavour

Wasmtime GitHub notifications bot (Jan 13 2026 at 19:04):

fitzgen commented on issue #12199:

We can have OutOfMemory wrap a NonNull<()> or whatever and bit pack in the address

Wasmtime GitHub notifications bot (Jan 13 2026 at 19:04):

fitzgen commented on issue #12199:

(Or be a newtype of Error, as originally suggested)

Wasmtime GitHub notifications bot (Jan 13 2026 at 19:08):

alexcrichton commented on issue #12199:

Good point, and ok yeah that seems reasonable (struct OutOfMemory(Error)) with some extra guarantees perhaps or something like that

Wasmtime GitHub notifications bot (Jan 14 2026 at 20:27):

fitzgen closed issue #12199:

There's some more discussion here, but I personally think we should strive to extend this error with the contextual number of bytes that were attempted to be allocated. Implementation-wise this will be tricky, however, so it'll require deliberate design.

Last updated: Feb 24 2026 at 05:28 UTC