Merge two wasm files. · wasm · Zulip Chat Archive

I just discovered wasm-tools. Wow :) I have a certain task to do. I would like to merge two wasm files into one with a single module. Which tool should I use? I would like to merge functions, exports and imports and even resolve an import from one wasm with exported fn from another one.

Maciej Zieliński (Feb 28 2023 at 08:17):

I played a bit more. Why does the wasm_encoder and wasmparser have their own type systems? After parsing wasm it needs to be manualy translated to wasm_encoder types. Is it possible to parse *.wasm file to a wasm_encoder::Module in one step?

Robin Brown (Feb 28 2023 at 15:33):

wasm_encoder's types aren't an AST for wasm. They're more of a builder that appends bytes to a buffer as you go. If you had wasmparser create a wasm_encoder module, you'd be back to the same exact vector of bytes.

Robin Brown (Feb 28 2023 at 15:36):

What's the use case for merging to modules together? There are some other things I might suggest depending on what you're trying to achieve.

Maciej Zieliński (Feb 28 2023 at 17:36):

I want to have a Rust library that can be run on two different WASM VMs. Each VM has its own ffi set.

Let me give you SQL based example. There are two WASM VMs that can run SQL query SQLBackend1 and SQLBackend2. Both have different ffis that might look like that:

// SQLBackend1
extern {
    pub fn run_sql(query: &str);
}

// SQLBackend2
extern {
    pub fn query_db(query: &str, mode: u32);
}

Now my goal is allowing developers to code just once, and be able to pick the backend later.
To do that I'd write two "backend" wasms with the same interface:

// sql_backend_1.wasm
extern {
    pub fn run_sql(query: &str);
}

#[no_mangle]
fn sql(query: &str) {
    unsafe {
        run_sql(query);
    }
}

// sql_backend_2.wasm
extern {
    pub fn query_db(query: &str, mode: u32);
}

#[no_mangle]
fn sql(query: &str) {
    unsafe {
        query_db(query, 0);
    }
}

sql_backend_1.wasm
    - import:
        - run_sql
    - export:
        - sql

sql_backend_2.wasm
    - import:
        - query_db
    - export:
        - sql

// caller.wasm

extern {
    pub fn sql(query: &str);
}

#[no_mangle]
fn main() {
    unsafe {
        sql("INSERT INTO Customers (CustomerName) VALUES ('Cardinal');");
    }
}

caller.wasm
    - import:
        - sql
    - export:
        - main

backend_1_caller.wasm
    - import:
        - run_sql
    - export:
        - main

sql_backend_2.wasm
    - import:
        - query_db
    - export:
        - main

Components should solve this "plugable" backend issue. The problem is that those two WASM VMs I have only accept WASMs with a single module in it.

What I thought of doing is using wasmparser on each wasm resolve exports and imports manualy and put together a single module using wasm_encoder.

Robin Brown (Mar 01 2023 at 14:02):

Components are definitely the "right" way to do this, but it should be possible as long as A) the runtime you're using supports multi-memory, B) only one .wasm needs a memory, or C) they both use memory in a way that won't clobber each other if they share.

Robin Brown (Mar 01 2023 at 14:03):

There is no tool for doing this that I know of, but it should be entirely possible. It's just a matter of reading both, figuring out how the indexes change, and re-encoding everything.

Robin Brown (Mar 01 2023 at 14:04):

Say goodbye to your debug info if you've got any, because this won't preserve it.

David Lloyd (Mar 01 2023 at 14:10):

I don't know a lot about this, but emscripten seems to support doing this, by recording ELF-like relocations into custom sections (I think) and treating them similarly to ELF object files

Stream: wasm

Topic: Merge two wasm files.

Maciej Zieliński (Feb 27 2023 at 15:21):