Victor Adossi (Jun 11 2026 at 15:27): Victor Adossi (Jun 11 2026 at 15:27):Hey all,
Some friends building a no_std wasmtime embedding ran into an issue with the global_code() check that is used from EngineCode::new. Surprisingly this is the only place right now they're running into an issue (they just don't have a std for the thread-support required to use OnceLock)
Is there anything we can do about this in no_std environments? I've thought about it a little bit and it seems like there are at least a couple options:
That said would like to take a step back and ask if this has come up before/if anyone has thought about this more deeply.
Chris Fallin (Jun 11 2026 at 15:53):Hi @Victor Adossi -- we do have a nostd implementation of OnceLock here and it should just work. For example in the Wasmtime deployment I'm responsible for, we run in no-std mode no problem
(Note also that it's not just a "check" -- the map is used to look up modules from places where we have no other global context to reach them from. In a no-std build without signals this is still reachable e.g. from Pulley)
Victor Adossi (Jun 11 2026 at 16:01):Ah thanks Chris -- that's really helpful, didn't know that was there. And yeah to be clear, everything goes just fine for them until they hit that check.
(Note also that it's just just a "check" -- the map is used to look up modules from places where we have no other global context to reach them from. In a no-std build without signals this is still reachable e.g. from Pulley)
I assume here you mean it's not just a check -- yeah I see that it's guarding the registry, I'm just surprised how other no_std builds could have have not run into this.
I don't think they're running with the std feature enabled but maybe its' actually just that simple
Chris Fallin (Jun 11 2026 at 16:04):Sorry, yes, typo -- not just a check. When you say "everything goes fine for them until they hit that [module map registration not check]", what happens? Is there a panic of some sort? And yes, my next thing to verify would be that they're building with the appropriate feature set, e.g. not enabling std and using a stub impl that always panics or something
Victor Adossi (Jun 11 2026 at 16:08):So evidently they get a failure which (I think is a panic) during engine teardown -- not during start or while running components, which was also quite surprising to me.
But yeah I've reached out again, it will be hard to get an easy reproducer (I haven't seen the logs of their failure just yet!), but hopefully it is something simple. Really sounded like they just weren't getting the no_std version of OnceLock for some reason, but I definitely need to get some more detail.
Thanks again for the sanity check here
Chris Fallin (Jun 11 2026 at 16:28):no problem!
Chris Fallin (Jun 11 2026 at 16:29):if they have a specific panic message, we're happy to help diagnose further
Victor Adossi (Jun 12 2026 at 13:06):Ah so here's the panic message:
PANIC on core 6! .../wasmtime/src/runtime/module/registry.rs:405:5:
assertion failed: prev.is_none()
So it turns out they were running a much older version of wasmtime (they're updated now!) -- and std was not enabled as one would expect (their no_std build is just fine).
What they're finding is that multiple Engines seem to get the same location for CodeMemorys and the second (nth?) EngineCode::new creation fails when calling register_code.
I'm asking now whether it could be a problem in their mmap implementation and should hear back soon.
One thing that IMO is worth asking ahead of that result there is whether register_code needs to be done at all if signals_based_traps is set to false.
If register_code is required specifically for signals_based_traps handlers (am I reading that wrong?), then maybe the check actually shouldn't happen in the first place when signals_based_traps is disabled.
Some environments however may not have access to signal handlers. For example embedded scenarios may not support virtual memory. Other environments where Wasmtime is embedded within the surrounding environment may require that new signal handlers aren’t registered due to the global nature of signal handlers. This option exists to disable the signal handler registration when required for these scenarios.
Also asked whether they have has_virtual_memory turned on/off to see which path they're taking through the mmap machinery
Alex Crichton (Jun 12 2026 at 14:19):I think we had an issue on this historically in the issue tracker as well, or rather we still do. IIRC that was registering the same image in different engines and it trips an internal assert, but basically our data structures aren't prepared to handle this if that's the case. Can you confirm if that's what's happening?
Victor Adossi (Jun 13 2026 at 06:13):Yeah so they mentioned this issue initially (a while ago now):
https://github.com/bytecodealliance/wasmtime/issues/12511
I think that's the one you're alluding to, right @Alex Crichton ?
A quote:
Updated (v36 -> v47), but still need our patched gate register_code / unregister_code on signals_based_traps.
GLOBAL_CODE is one process-global BTreeMap and our allocator is per-CPU, so a node inserted on one core and freed on another at teardown corrupts that core's free list. It only feeds the signal-handler PC->trap lookup, which we never hit (signals_based_traps(false)) so skipping it when signals are off is what we need.
I asked them to check if maybe their mmap impl was producing the same location, but are you saying that's the current expected behavior with the same image?
Victor Adossi (Jun 13 2026 at 06:13):Really trying to get them to just get in here and chat about their very interesting use case :)
Lex S (Jun 13 2026 at 06:36):Can confirm we have multiple "Engine::new"s each being fed the exact same component image. Is adding a random number to the images a possible workaround?
bjorn3 (Jun 13 2026 at 09:18):Why do you have multiple Engines? Do you actually need different configs? And just to confirm are you sharing a wasmtime::component::Component across engines or do you mean that the bytes of the component are identical, but you are still creating distinct wasmtime::component::Component between the engines? The latter should work.
bjorn3 (Jun 13 2026 at 09:22):Generally you only need a single Engine across your entire application. It is meant to be shared between all your Stores. It only contains configuration options and a global type registry.
Lex S (Jun 13 2026 at 09:24):Our program is running in a no_std multi-core multi-domain environment. One engine per-core right now. Distinct wasmtime::component::Component per engine, instantiated from the exact same cwasm (bytes of the component are identical).
Lex S (Jun 13 2026 at 09:25):Really appreciate the community support and the wasmtime experience has been incredible.
Alex Crichton (Jun 16 2026 at 01:57):@Lex S could you explain why you're using multiple engines though? The architecture you're using makes sense, but it should also work and be reasonable with a single Engine and Components shared across threads. For example is this intended to reduce locking? Have different settings per-engine, or something else?
Lex S (Jun 16 2026 at 07:18):Our system has different CCDs in separate NUMA domains with partitioned memory allocators. The Engine instantiated on thread A would not be accessible from thread N. Same settings per-engine right now. The resulting reduction in locking is absolutely great, but not currently a bottleneck as far as I am aware.
We have things working well with a local patch to disable GLOBAL_CODE.
Alex Crichton (Jun 16 2026 at 13:39):Ah ok that makes sense yeah. If you're up for it I think this is something we could handle upstream, refcounting the entries in the GLOBAL_CODE map, and I'd be up for reviewing a PR to do that. The last time this was discussed on the issue it got bogged down in other concerns/issues that I think were unrelated, but just refcounting in the global map seems fine by me.
Chris Fallin (Jun 16 2026 at 15:27):Just to clarify: you have partitioned memory allocators (no address in common between cores), but you're still hitting this issue?
Are you using the from-raw-slice API and passing in a component image that is baked into shared .rodata or something similar? Otherwise I'm surprised, because the issue requires a component image at the same address on different cores...
Lex S (Jun 17 2026 at 00:31):Not sure the best way to provide a GLOBAL_CODE refcount PR, but I'd be happy to give it a shot.
We're currently calling Component::deserialize with a static CWASM once per Engine.
Chris Fallin (Jun 17 2026 at 03:41):Cool, yeah, given that (shared rodata but completely segregated allocators/heaps) it seems like refcounting is the way to go.
Do note that if the intent of segregating your heaps is that you have zero coherency traffic between cores, you'll violate that by nature of the shared map. However the painful part is on updates (loading new modules); subsequently the cache lines should be all read-only and shareable across cores. Just something to watch out for if your update (module-load) rate is non-negligible...
Last updated: Jul 29 2026 at 05:03 UTC