Stream: git-wasmtime

Topic: wasmtime / issue #7054 Threading in a Scripting Environment

Wasmtime GitHub notifications bot (Sep 18 2023 at 01:50):

WireWhiz opened issue #7054:

The title basically says it, I've been attempting to block out a scripting system for a game engine the past few weeks and the current feature I've been attempting to figure out is multi-threading. And I can't seem to find any way to run two threads at once, that have access to the same data. This is the only requirement I'm trying to fulfill by any method.

The engine runs on ECS, so you have big shared arrays of data, operated upon by a bunch of scripts containing functions that interact with those arrays.

Ideally I'd like to accomplish two things, have systems/jobs called from separate threads be able to act on the shared data at the same time. And possibly multi thread access to data arrays within those systems/jobs.

My current setup is multiple wasm modules (one for each job/system, though I'm undecided if I need to segment them that much), with an imported shared array buffer so they're able to access each other's memory through pointers. Ideally built to raw wasm, not wasi, since I want to keep my experimental dependencies down, but for this threading exploration I switched to building wasi targets without entry points.

Initially I tried figuring out a way to multi-thread calls into the wasm runtime, but with the &mut store requirement that's not possible unless I want to risk some unsafe skirt around the borrow checker stuff.

So then I decided to try to build the job scheduling/thread pooling system inside of wasm, If I can't use threads outside I might as well use them inside right? After researching this, upgrading to wasi since I found out you need too, I looked into the wasmtime_wasi_threading crate and it seems intended (as all things wasi) for the entire application to be embedded in a wasm module. This doesn't really vibe with my "host runtime calling scripts" end goal. So that path of exploration has kind of ended as well. It also seems that an entirely different store is created for each thread (unverified through testing, but the source looks that way), and my understanding is that stores can't interact with each other, and that's my basic requirement.

I'm fully aware you can instantiate clones of a module in different stores, but again, the memory cant interact. I also know you can copy data but I just cringe instantly at that because with a game engine you really need that performance.

My question/issue is how do I access shared data from multiple threads. Does the wasi threading runtime do some black magic behind the scenes to share data and I just need to write my job scheduling system as a program that lives on another thread? Even then how would I interact with it while it's running?

My thoughts are that since SharedMemoryBuffer is already supposed to be thread-safe, threaded function calls modifying that should be fine, even for calls that would cause a memory allocation behind the scenes, so the &mut store lock is only really needed for loading or unloading new modules. I can totally see a system where I have the job scheduling system create multiple (unsafe) references to the store to be able to call functions from multiple threads, then periodically pause so that modules may be loaded or unloaded before continuing to make sure that no functions bug out while stuff is changing.

Is there an intended way to do this? Or is my use case unexplored territory? I'm also more than happy to get involved with the project and try to get something working if it's not possible currently.

Wasmtime GitHub notifications bot (Sep 18 2023 at 14:31):

alexcrichton commented on issue #7054:

This blog post might be interesting for you, but I can try to help fill in some other gaps here:

• As you've discovered the only way to have multithreading with wasm today and is through SharedMemory as the linear memory for wasm. This is the WebAssembly threads proposal (which actually mostly just adds atomic-related instructions).
• Multithreading via the WebAssembly threads proposal is modeled as an instance-per-thread. This means that your module should likely import a shared memory which will then be instantiated once per thread.
• Wasmtime's design requires a Store-per-thread, which means for your application you'll have a Store per thread, each with a single Instance, all of which instantiate the same module (Module can be shared across threads).

This should all work today if you enable the right features and flags in both LLVM and Wasmtime, but it's unlikely to be easy. The developer experience story here is pretty underbaked and you're very much on your own unfortunately. There's spec work for threads however which is intended to provide a better story here in the future, but that's far out.

That I hope is at least some basic information to help clear up a few things. Whether or not you use small modules or one large module is up to you and your embedding and I won't have much guidance on that myself. Or I should also ask, does that help clear things up? Are there other Wasmtime questions I can help with?

Wasmtime GitHub notifications bot (Sep 18 2023 at 15:58):

WireWhiz commented on issue #7054:

It clears up the intended set up for threads, so that's more clear. My last question would be, if I were to create a separate store per thread, and separate model instances, would I be able to share a single SharedMemoryBuffer between them? My understanding currently is that memory allocated in one store cannot be used by a module in a different store.

Wasmtime GitHub notifications bot (Sep 18 2023 at 16:17):

bjorn3 commented on issue #7054:

Memory::new requires a reference to a Store as argument. SharedMemory::new merely requires a references to an Engine which is not bound to a single thread and generally shared between all Stores in the process.

Wasmtime GitHub notifications bot (Sep 18 2023 at 16:34):

WireWhiz commented on issue #7054:

OH I don't know how I missed that, I've just been creating shared memory buffer by passing in the type to the Memory constructor: Memory::new(&mut store, MemoryType::shared(32, 32768)).unwrap()

I'll look into this and see if I can get it working, this probably provides a route for me.

Wasmtime GitHub notifications bot (Sep 21 2023 at 05:44):

WireWhiz commented on issue #7054:

Ok, after some testing I've run into a super weird bug. Before everything was fine, I could load several different wasm modules into the same shared memory and they coexisted fine. But changing to SharedMemory::new with no other significant changes from what I was doing before causes the first memory allocation within wasm to throw an error.

I suspected it might be different modules not coexisting well so I removed all but one and that fixed it.

So it looks like for some reason memory allocation gets messed up but the presence of differing modules.

Could this be some variables to do with malloc not being synced between them? Do they not sync in the first place? If that's the case would I need to look into maybe writing a custom allocator that can handle memory allocation for multiple modules together?

Wasmtime GitHub notifications bot (Sep 21 2023 at 18:10):

WireWhiz commented on issue #7054:

I attempted switching from wasm32-unknown-unknown to using the wasm32-wasi-preview1-threads build target, though and appears to be throwing the same error. At the moment the two modules I have are spawned on the same thread (different code in each), and the error is triggered before any threads and thus module instances on other threads are spawned. Would shadow stacks still be something worth looking into? Or do those only apply for multi-threaded scenarios?

Wasmtime GitHub notifications bot (Sep 21 2023 at 18:16):

cfallin commented on issue #7054:

If on a single thread, shadow-stack interference wouldn't be a factor, no.

Another thing that could be going wrong: perhaps the memory is just missing the initial heap image entirely? How are you providing the shared-memory -- did you change the exported memory 0 to an imported memory instead? Are there still data segments in the module to initialize it?

Wasmtime GitHub notifications bot (Sep 21 2023 at 18:37):

WireWhiz commented on issue #7054:

Here's the shell script I currently use to build, to give an idea of the flags/options I'm using

export RUSTFLAGS="--cfg=web_sys_unstable_apis -C target-feature=+atomics,+bulk-memory,+mutable-globals -C link-arg=--no-entry -C link-arg=--shared-memory -C link-arg=--import-memory -C link-arg=--max-memory=2147483648"
echo "Compiling scripts with flags:" $RUSTFLAGS
cargo +nightly build --target wasm32-wasi-preview1-threads -Z build-std=std,panic_abort

running wasm2wat it seems that memory is both imported and exported, I'm unsure if this is by design. My intent is to allocate the shared memory in the runtime and then provide it to modules for them to import.

...
  (import "env" "memory" (memory (;0;) 17 32768 shared))
...
  (export "memory" (memory 0))
...

If I print out imports and exports after loading the module with wasmtime it appears to both export and import the same shared memory (excuse my command line formatting):

Imports
Memory(MemoryType { ty: Memory { minimum: 17, maximum: Some(32768), shared: true, memory64: false } }): env.memory
Exports
Memory(MemoryType { ty: Memory { minimum: 17, maximum: Some(32768), shared: true, memory64: false } }): env.memory

I'm unsure where to look for data initialization but I'd guess it'd have something to do with this portion of the file?

Wasmtime GitHub notifications bot (Sep 21 2023 at 18:43):

WireWhiz commented on issue #7054:

Here's that init memory function, I assume that's useful as well. I should note I never call this, and I'm directly calling functions from a wasmtime::Linker so I don't know if it would be called automatically.

(func $__wasm_init_memory (type 9)
    block  ;; label = @1
      block  ;; label = @2
        block  ;; label = @3
          i32.const 1097564
          i32.const 0
          i32.const 1
          i32.atomic.rmw.cmpxchg
          br_table 0 (;@3;) 1 (;@2;) 2 (;@1;)
        end
        i32.const 1048576
        i32.const 1048576
        global.set $__tls_base
        i32.const 0
        i32.const 148
        memory.init $.tdata
        i32.const 1048736
        i32.const 0
        i32.const 48153
        memory.init $.rodata
        i32.const 1096892
        i32.const 0
        i32.const 12
        memory.init $.data
        i32.const 1096904
        i32.const 0
        i32.const 660
        memory.fill
        i32.const 1097564
        i32.const 2
        i32.atomic.store
        i32.const 1097564
        i32.const -1
        memory.atomic.notify
        drop
        br 1 (;@1;)
      end
      i32.const 1097564
      i32.const 1
      i64.const -1
      memory.atomic.wait32
      drop
    end
    data.drop $.rodata
    data.drop $.data)

Wasmtime GitHub notifications bot (Sep 21 2023 at 18:54):

bjorn3 commented on issue #7054:

__wasm_init_memory shouldn't be in the start section. That will cause every spawned thread to overwrite the shared memory. Instead it should be called once on the main thread and then the function for initializing TLS should be called once per thread.

Wasmtime GitHub notifications bot (Sep 21 2023 at 19:08):

WireWhiz commented on issue #7054:

Is there a link option I can use to set that? Or should I write a script to modify it? Also does linker.module auto call any functions if wasi has been added to it? Or am I in charge of calling those functions. I'm not currently using WasiThreadsCtx either. I'm also not seeing an explicit init tls function, but I am seeing a wasi_thread_start function, does that serve a similar purpose?

Wasmtime GitHub notifications bot (Sep 21 2023 at 19:11):

bjorn3 commented on issue #7054:

I'm also not seeing an explicit init tls function, but I am seeing a wasi_thread_start function, does that serve a similar purpose?

I believe that is one of the things wasi_thread_start does.

Is there a link option I can use to set that?

The wasm32-wasi-preview1-threads target uses --import-memory --export-memory --shared-memory, but I don't think the extra --export-memory argument would help.

Wasmtime GitHub notifications bot (Sep 21 2023 at 22:56):

abrown commented on issue #7054:

cc: @g0djan, who worked on the Rust wasm32-wasi-preview1-threads target you're using. That target uses the wasi-libc implemetation IIRC, which boils down to wasi_thread_start and __wasi_thread_start_C. It could very well be that there is a bug here somewhere; remember this target is still a work in progress.

Wasmtime GitHub notifications bot (Sep 22 2023 at 07:52):

WireWhiz commented on issue #7054:

But thread start shouldn't need to be called for single threaded instantiation of multiple modules, maybe I should create a new issue over in the rust or wasm repos for the memory issue and then come back to this once that's fixed since it is technically a separate issue that most likely isn't in wasmtime.

Wasmtime GitHub notifications bot (Sep 22 2023 at 13:47):

g0djan commented on issue #7054:

running wasm2wat it seems that memory is both imported and exported, I'm unsure if this is by design.
Yes it was by design but idk remember the exact reasoning for that as it was done just the same way as it's done in C by wasi-sdk. And also wasmtime didn't allow to run it until it's compiled with --export-memory .

Wasmtime GitHub notifications bot (Sep 22 2023 at 13:47):

g0djan edited a comment on issue #7054:

running wasm2wat it seems that memory is both imported and exported, I'm unsure if this is by design.

Yes it was by design but idk remember the exact reasoning for that as it was done just the same way as it's done in C by wasi-sdk. And also wasmtime didn't allow to run it until it's compiled with --export-memory .

Wasmtime GitHub notifications bot (Sep 22 2023 at 13:47):

g0djan edited a comment on issue #7054:

running wasm2wat it seems that memory is both imported and exported, I'm unsure if this is by design.

Yes it was by design but idk remember the exact reasoning for that as it was done just the same way as it's done in C by wasi-sdk. And also wasmtime didn't allow to run it until it's compiled with --export-memory .

I'll take a look at your example next week.

Wasmtime GitHub notifications bot (Sep 22 2023 at 16:14):

bjorn3 commented on issue #7054:

Wasi requires exporting the memory for the wasi runtime to know where to read and write, but multi threading requires importing the memory to aboid each module getting it's own distinct memory.

Last updated: Nov 22 2024 at 17:03 UTC