Stream: rust-toolchain

Topic: shared memory

Alexandru Ene (Aug 01 2022 at 17:45):

Is there a way to build rust with shared memory support in wasm32-wasi?

the only flags I could find are:

Features supported by rustc for this target:
    simd128             - Enable 128-bit SIMD.
    atomics             - Enable Atomics.
    nontrapping-fptoint - Enable non-trapping float-to-int conversion operators.
    crt-static          - Enables C Run-time Libraries to be statically linked.

Code-generation features supported by LLVM for this target:
    bulk-memory         - Enable bulk memory operations.
    exception-handling  - Enable Wasm exception handling.
    multivalue          - Enable multivalue blocks, instructions, and functions.
    mutable-globals     - Enable mutable globals.
    reference-types     - Enable reference types.
    relaxed-simd        - Enable relaxed-simd instructions.
    sign-ext            - Enable sign extension operators.
    tail-call           - Enable tail call instructions.

Alexandru Ene (Aug 01 2022 at 17:50):

Essentially I'm looking to re-create this in rust, but it seems that the main memory has to be shared in order for the example to work, otherwise each thread in wamr will get it's own memory. https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/samples/multi-thread/wasm-apps/main.c

Alexandru Ene (Aug 02 2022 at 10:54):

Shouldn't atomics also enable shared_memory?

Marcin Kolny (Aug 02 2022 at 15:14):

I'm not sure if shared memory is even implemented by the rust toolchain. AFAIR last time I checked, it wasn't.

Alexandru Ene (Aug 02 2022 at 15:34):

I thought that's more a thing that the code generator does, so it should be ok to just pass a setting on to LLVM? Not familiar with what part of the toolchain does what bits

Alexandru Ene (Aug 02 2022 at 15:35):

I'd be up for doing some work to enable this, but it's not clear to me at this point what the gaps are to enable this so the wamr binary memory has the shared flag on it

Alexandru Ene (Aug 04 2022 at 06:25):

Now I'm a bit more confused, it seems like -shared-memory is forwarded from here: https://github.com/rust-lang/rust/blob/master/compiler/rustc_codegen_ssa/src/back/linker.rs#L1210

However, my binary doesn't have the shared memory set when adding +atomics

Alexandru Ene (Aug 04 2022 at 07:00):

Ok so it's clear now what's happening. If the program depends on rust's stdlib, then it won't compile with shared memory as that's not supported.

I've discovered this by having .cargo/config do:

[build]
rustflags = ["-C", "target-cpu=native"]

[target.wasm32-wasi]
rustflags = ["-Ctarget-feature=+atomics,+bulk-memory", "-Clink_args=--shared-memory"]

And then it fails with a linker error. In order to fix it I've had to compile with #![nostd] and add a dummy panic handler.

More specifically the error is:

  = note: rust-lld: error: --shared-memory is disallowed by std-f45c87023ba188f4.std.4d6b15f2-cgu.0.rcgu.o because it was not compiled with 'atomics' or 'bulk-memory' features.

bjorn3 (Aug 04 2022 at 12:13):

You have to recompile the standard library of you enable shared memory. By default it is compiled with the assumption that wasm is single threaded. You can use -Zbuild-std=std as cargo argument for this. I don't think you need -Clink-args=--shared-memory in that case.

Alexandru Ene (Aug 04 2022 at 12:52):

Oh yes, I have tried but wasm32-wasi doesn't compile with shared memory on:

s_common/thread_parker/futex.rs:4:17
  |
4 | use crate::sys::futex::{futex_wait, futex_wake};
  |                 ^^^^^ could not find `futex` in `sys`

and

sys/wasi/os.rs:149:22
    |
149 |         let _guard = env_lock();
    |                      ^^^^^^^^ not found in this scope

I'll have to dig a bit deeper to see what's missing here. Maybe some impl needs to be stolen from wasm32-unknown-unknown target.

env_lock is now defined as:

#[cfg(not(target_feature = "atomics"))]
pub unsafe fn env_lock() -> impl Any {
    // No need for a lock if we're single-threaded, but this function will need
    // to get implemented for multi-threaded scenarios
}

Marcin Kolny (Aug 04 2022 at 13:47):

Not sure if wasm32-unknown-unknown should even have access to env (only WASI should)

Alexandru Ene (Aug 04 2022 at 13:48):

Oh yes, I mean for futex stuff. I'll look into it when I get some more time

Marcin Kolny (Aug 04 2022 at 19:12):

ah yeah, that one probably can be copied from wasm-unknown-unknown from what I see

Last updated: Dec 23 2024 at 14:03 UTC