Stream: general

Topic: Memory Not Shared Between Modules

Dennis Zhang (Jan 20 2025 at 16:06):

I'm working on a WebAssembly project with two modules (main.wasm and env.wasm) that use shared memory. env.wasm exports the memory and a function, while main.wasm imports the memory and calls the function. However, when I try to run the modules using Wasmtime, I encounter the following error:

Error: failed to run main module `main.wasm`

Caused by:
    0: failed to instantiate "main.wasm"
    1: unknown import: `env::memory` has not been defined

Here are my compilation commands:
For env.wasm:

  clang \
  --target=wasm32-wasi-threads \
  --sysroot /wasi-libc/sysroot \
  -pthread \
  env.c \
  -o env.wasm \
  -Wl,--no-entry \
  -Wl,--export=process_string \
  -Wl,--export-memory \
  -Wl,--shared-memory \
  -Wl,--initial-memory=131072 \
  -Wl,--max-memory=131072

For main.wasm:

  clang \
  -Wno-implicit-function-declaration \
  --target=wasm32-wasi-threads \
  --sysroot /wasi-libc/sysroot \
  -pthread \
  main.c \
  -o main.wasm \
  -Wl,--export=_start \
  -Wl,--allow-undefined \
  -Wl,--shared-memory \
  -Wl,--import-memory \
  -Wl,--initial-memory=131072 \
  -Wl,--max-memory=131072

I'm running the modules in Wasmtime with the following command:

wasmtime --preload env=env.wasm \
         main.wasm \
         --invoke main \
         --wasm-features=threads \
         --wasi-modules=default

What I've Checked:

1. Memory Export in env.wasm: Using wasm-objdump -x, I confirmed that memory is exported with (memory (;0;) 2 2 shared).
2. Memory Import in main.wasm: Also confirmed via wasm-objdump -x that main.wasm is attempting to import env::memory with (memory (;0;) 2 2 shared).
3. Namespace Matching: Both modules seem to use env as the namespace for the memory.

Question:

What could be causing Wasmtime to fail to resolve env::memory? Are there additional flags or configurations I might have missed? Any help would be greatly appreciated!

Alex Crichton (Jan 20 2025 at 16:53):

Would you be able to share the two modules in question?

Dennis Zhang (Jan 20 2025 at 17:17):

Hi Alex! Of course, here are the two modules:

main.c

#include <stdio.h>

int main(int argc, char** argv) {

    const char* input_string = "Hello from main.wasm";

    int response_int = process_string(input_string);

    printf("Received integer from env.wasm: %d\n", response_int);

    return 0;
}

env.c

#include <stdio.h>
#include <string.h>

static int my_return_integer = 66;

// Function that prints the input string and returns a int
int process_string(const char* input) {

    printf("lib.wasm received string: %s\n", input);

    return my_return_integer;
}

Let me know if you need further clarification!

Alex Crichton (Jan 20 2025 at 18:19):

ah looks like the problem is the _start export of env.wasm which causes it to be classified as a "command" and only functions can be imported from commands, not globals/memories (as command-imported functions get a fresh new instance each time)

Dennis Zhang (Jan 20 2025 at 18:42):

Good point! Thanks, Alex. I used -Wl,--no-entry and -nostartfiles to remove the _start function in env.wasm. I have double-checked this using wasm-objdump, and _start is no longer present in the exports.

However, when I run the following command:

wasmtime --preload env=env.wasm \
         main.wasm \
         --invoke main \
         --wasm-features=threads \
         --wasi-modules=default

The program seems to hang indefinitely. Any help on resolving this issue would be greatly appreciated!

Alex Crichton (Jan 21 2025 at 15:17):

I suspect what's happening is these two modules are corrupting each other. You're loading them into the same shared memory but they're not compiled knowing the other is in the same shared memory, so they're probably stomping all over each other's stacks/data structures/rodata/data/etc

Joel Dice (Jan 21 2025 at 16:28):

In case it's useful this is the convention for sharing memory and table(s) in an orderly way.

Dennis Zhang (Jan 22 2025 at 18:47):

Hi Alex and Joel,

Thanks for the help! I’m now able to move further. However, when I run the command:

wasmtime --preload env=env.wasm \
         main.wasm \
         --invoke main \
         --wasm-features=threads \
         --wasi-modules=default

I get some output along with the following error message:

Error: failed to run main module `main.wasm`

Caused by:
    0: failed to invoke command default
    1: error while executing at wasm backtrace:
           0: 0x2be2 - vfprintf
                           at /wasi-libc/libc-top-half/musl/src/stdio/vfprintf.c:726:3
           1:  0x2db - printf
                           at /wasi-libc/libc-top-half/musl/src/stdio/printf.c:9:8
           2:  0x265 - <unknown>!process_string
           3: 0xb965 - <unknown>!process_string.command_export
           4:  0x44d - <unknown>!main
           5: 0xcd49 - <unknown>!__main_void
           6:  0x39c - _start
                           at /wasi-libc/libc-bottom-half/crt/crt1-command.c:43:13
    2: wasm trap: undefined element: out of bounds table access

I think both module are loaded, but seems like something is going wrong with function pointers (or function table) in a multi-module setup. Can you help me understand what might be causing this error?

Dennis Zhang (Jan 27 2025 at 21:02):

Joel Dice said:

In case it's useful this is the convention for sharing memory and table(s) in an orderly way.

Hi Joel @Joel Dice, thanks for sharing this document! Could you let me know the current status of the implementation?

Joel Dice (Jan 27 2025 at 21:08):

LLVM/Clang/wasm-ld have fully supported that convention for several years. You can build modules which adhere to it using --shared.

On the host side, Emscripten supports loading and linking such modules. For WASI, the only way to use it AFAIK is via wasm-tools component link, which links the input modules together and produces a single Wasm component as the output.

Joel Dice (Jan 27 2025 at 21:12):

If you're interested in using wasm-tools component link to produce a component from a set of modules which share memory, this article might be helpful.

Dennis Zhang (Feb 03 2025 at 19:01):

Hi Joel @Joel Dice . Thanks for your help! We are now trying to compile the program with the -fPIC option, but it seems that the output object file is no different from the one compiled without -fPIC. The -fPIC version should contain a "dylink" section, right? According to the document you shared (Dynamic Linking in WebAssembly), it should be present.

Joel Dice (Feb 04 2025 at 13:15):

Did you specify the -shared flag in your clang command? That is what tells it to generate a dynamic library with a dylink.0 section

Dennis Zhang (Feb 05 2025 at 18:45):

Hi Joel! Thanks for your help. We upgrade our clang version from 16 to 18, and now we have the "- name: "dylink.0"" section in our module!

Dennis Zhang (Feb 05 2025 at 18:57):

@Joel Dice Hi Joel. I have another question: Does Wasmtime support running WebAssembly modules that enable dynamically loaded Wasm binaries, such as when using the --preload option? Or only Emscripten support that?

Joel Dice (Feb 06 2025 at 14:02):

AFAIK, Wasmtime does not support such binaries except when embedded in a component using wasm-tools component link. Only Emscripten knows how to load them outside of a component.

Last updated: Feb 28 2025 at 02:27 UTC