wasmtime / issue #10359 Question about Wasmtime thread model · git-wasmtime

Stream: git-wasmtime

Topic: wasmtime / issue #10359 Question about Wasmtime thread model

Wasmtime GitHub notifications bot (Mar 08 2025 at 18:28):

Quetion

Does wasmtime always run Wasm guest code in main thread?

Background

I plan to use Wasm as a plugin system for rust service, and call multiple host functions within the guest code, I found that the Wasm itself became the bottleneck, after doing some investigation, I found that whateven method I used, despite of the multiple threads Wasmtime Engine/Module/Instance I have in host code, the same wasm file will always runs in the same thread (thread ID 1, the main thread)

There is the log I print in console, hopefully it's self explainable
main: ThreadId(1)
host call guest.fun at: ThreadId(16)
host call guest.fun at: ThreadId(15)
host call guest.fun at: ThreadId(17)
host call guest.fun at: ThreadId(14)
guest: call host.fun at: ThreadId(1)
guest: call host.fun at: ThreadId(1)
guest: call host.fun at: ThreadId(1)
host: called from guest at: ThreadId(16)
guest: return from host.fun at: ThreadId(1)
host: called from guest at: ThreadId(15)
guest: return from host.fun at: ThreadId(1)
host: called from guest at: ThreadId(17)
guest: call host.fun at: ThreadId(1)
host: called from guest at: ThreadId(14)
guest: return from host.fun at: ThreadId(1)
guest: return from host.fun at: ThreadId(1)
res: guest result: 42
res: guest result: 42
res: guest result: 42
res: guest result: 42
![Image](https://github.com/user-attachments/assets/8923a9f0-63dc-4480-a7a5-b3da012eed3c)

the issue

I'd like to improve the throughput of this service, but if the guest function alwasys runs in the same single thread, the overall throughput will depen on the time spending in guest code, let's say if it takes 10ms, the overall QPS will be limited to 100, even if I add more threads to support more wasmtime instances.

Is this by design, or I did I miss some critical configuration?

the requirement

I'd like to eliminate the bottleneck in single guest thread mode and improve service performance, which part should I change?

Wasmtime GitHub notifications bot (Mar 08 2025 at 18:30):

xdlin edited issue #10359:

Quetion

Does wasmtime always run Wasm guest code in main thread?

Background

I plan to use Wasm as a plugin system for rust service, and call multiple host functions within the guest code, I found that the Wasm itself became the bottleneck, after doing some investigation, I found that whatever method I used, despite of the multiple threads Wasmtime Engine/Module/Instance I have in host code, __the same wasm file__ will always runs in the same thread (thread ID 1, the main thread)

There is the log I print in console, hopefully it's self explainable
main: ThreadId(1)
host call guest.fun at: ThreadId(16)
host call guest.fun at: ThreadId(15)
host call guest.fun at: ThreadId(17)
host call guest.fun at: ThreadId(14)
guest: call host.fun at: ThreadId(1)
guest: call host.fun at: ThreadId(1)
guest: call host.fun at: ThreadId(1)
host: called from guest at: ThreadId(16)
guest: return from host.fun at: ThreadId(1)
host: called from guest at: ThreadId(15)
guest: return from host.fun at: ThreadId(1)
host: called from guest at: ThreadId(17)
guest: call host.fun at: ThreadId(1)
host: called from guest at: ThreadId(14)
guest: return from host.fun at: ThreadId(1)
guest: return from host.fun at: ThreadId(1)
res: guest result: 42
res: guest result: 42
res: guest result: 42
res: guest result: 42
![Image](https://github.com/user-attachments/assets/8923a9f0-63dc-4480-a7a5-b3da012eed3c)

the issue

I'd like to improve the throughput of this service, but if the guest function alwasys runs in the same single thread, the overall throughput will depen on the time spending in guest code, let's say if it takes 10ms, the overall QPS will be limited to 100, even if I add more threads to support more wasmtime instances.

Is this by design, or I did I miss some critical configuration?

the requirement

I'd like to eliminate the bottleneck in single guest thread mode and improve service performance, which part should I change?

Wasmtime GitHub notifications bot (Mar 08 2025 at 21:28):

cfallin commented on issue #10359:

Does wasmtime always run Wasm guest code in main thread?

Short answer: no; it will run on the thread where you call it.

Longer answer: Wasmtime has two invocation models, sync and async.

The sync model is very simple: calling a Wasm function will execute that function on the stack it was called on; Wasmtime has no knowledge of threads; it will just run the code like a normal function call.

The async model fits into the async approach in Rust. "Calling" a Wasm function will return a future and will not actually execute the code. It is then up to the async runtime to poll that future. You cause that to happen by doing .await in Rust code. The threading model is up to the async runtime, which is a separate piece of software that is not part of Wasmtime. Tokio in its usual configuration will use a thread-pool, so code will run on multiple threads.

You haven't described how you produced the log above; how does the "guest" know what thread it is on? In any case, there is nothing in Wasmtime that would cause execution to migrate to only one thread, so I suspect the issue is elsewhere in your application architecture.

Wasmtime GitHub notifications bot (Mar 08 2025 at 21:58):

xdlin commented on issue #10359:

Actually I tried both:

sync mode
async mode

I thought it's a foundmantal question related with invocation mode, so I didn't paste the code, let's me post the minimal reproducible example here

the WIT file（`wit/server.wit`)

package my:service;

interface native {
    predict: func(input: string) -> u64;
}

interface guest {
    factor-get: func(id: string) -> string;
}

world factor-server {
    import native;
    export guest;
}

host code (`host/src/main.rs`)

use cargo_metadata::MetadataCommand;
use std::path::PathBuf;
use {
    anyhow::Result,
    wasmtime::{
        component::{Component, Linker, ResourceTable},
        Config, Engine, Store,
    },
    wasmtime_wasi::{IoView, WasiCtx, WasiCtxBuilder, WasiView},
};

mod handler {
    wasmtime::component::bindgen!({
        path: "../wit",
        world: "factor-server",
        async: false,
        concurrent_imports: false,
        concurrent_exports: false,
        trappable_imports: false,
    });
}

struct WasmStates {
    wasi: WasiCtx,
    table: ResourceTable,
}

impl WasmStates {
    pub fn new() -> Self {
        let table = ResourceTable::new();
        let wasi = WasiCtxBuilder::new().inherit_stdio().inherit_args().build();

        Self { table, wasi }
    }
}

impl IoView for WasmStates {
    fn table(&mut self) -> &mut ResourceTable {
        &mut self.table
    }
}
impl WasiView for WasmStates {
    fn ctx(&mut self) -> &mut WasiCtx {
        &mut self.wasi
    }
}

impl handler::my::service::native::Host for WasmStates {
    fn predict(&mut self, _input: String) -> u64 {
        println!(
            "host: called from guest at: {:?}",
            std::thread::current().id()
        );
        42
    }
}

fn get_workspace_root() -> PathBuf {
    let metadata = MetadataCommand::new().exec().unwrap();
    metadata.workspace_root.into()
}

#[tokio::main(flavor = "multi_thread")]
async fn main() -> Result<()> {
    let root = get_workspace_root();
    let wasm_path = root.join("wasm_modules/guest.wasm");

    let mut config = Config::new();
    config.wasm_component_model(true);
    config.async_support(false);
    config.debug_info(true);
    config.cranelift_debug_verifier(true);
    config.wasm_threads(true);

    println!("main: {:?}", std::thread::current().id());
    let mut handles = vec![];

    for _ in 0..4 {
        let handle = {
            let wasm_path = wasm_path.clone();
            let config = config.clone();
            std::thread::spawn(move || {
                let engine = Engine::new(&config).unwrap();
                let mut linker: Linker<WasmStates> = Linker::new(&engine);
                wasmtime_wasi::add_to_linker_sync(&mut linker).unwrap();
                handler::FactorServer::add_to_linker(&mut linker, |ctx: &mut WasmStates| ctx)
                    .unwrap();
                let component = Component::from_file(&engine, wasm_path).unwrap();
                let wasm_states = WasmStates::new();
                let mut store = Store::new(&engine, wasm_states);
                let instance = linker.instantiate(&mut store, &component).unwrap();

                let instance = handler::FactorServer::new(&mut store, &instance).unwrap();
                println!("host call guest.fun at: {:?}", std::thread::current().id());
                let res = instance
                    .my_service_guest()
                    .call_factor_get(&mut store, "foo")
                    .unwrap();
                res
            })
        };
        handles.push(handle);
    }

    for handle in handles {
        let res = handle.join().unwrap();
        println!("res: {}", res);
    }

    Ok(())
}

the guest code (`guest/src/lib.rs`)

wit_demo.tgz

mod bindings {
    wit_bindgen::generate!({
        path: "../wit",
        world: "factor-server",
    });

    use super::Component;
    export!(Component);
}

use bindings::{exports::my::service::guest::Guest, my::service::native};

struct Component;

impl Guest for Component {
    fn factor_get(id: String) -> String {
        println!("guest: call host.fun at: {:?}", wasm_thread::current().id());
        let res = native::predict(&id);
        println!(
            "guest: return from host.fun at: {:?}",
            wasm_thread::current().id()
        );
        format!("guest result: {}", res)
    }
}

Cargo.toml

workspace Cargo.toml

[workspace]
resolver = "2"
members = [
    "host",
    "guest",
]

host/Cargo.toml:

[package]
name = "host"
version = "0.1.0"
edition = "2021"

[dependencies]
anyhow = { version = "1.0.81", features = ["backtrace"] }
tokio = { version = "1.36.0", features = ["fs", "macros", "rt-multi-thread", "time"] }
wasmtime = { version = "*", features = ["component-model"]}
wasmtime-wasi = { version = "*"}
cargo_metadata = "0.18"
axum = "0.8.1"
rand = "0.9.0"

guest/Cargo.toml

[package]
name = "guest"
version = "0.1.0"
edition = "2021"

[dependencies]
futures = "0.3.30"
once_cell = "1.19.0"
wasm_thread = "0.3.3"
wit-bindgen = "0.40"
wit-bindgen-rt = "0.40"

[lib]
crate-type = ["cdylib"]

Makefile

In order to use component mode, here is the additional steps in Makefile

SHELL := /bin/bash
release:
    pushd guest && cargo build --release && popd && echo "done"
    mkdir -p wasm_modules && wasm-tools component new --skip-validation   ./target/wasm32-wasip1/release/guest.wasm  --adapt ./wasi_snapshot_preview1.reactor.wasm -o wasm_modules/guest.wasm
    pushd host && cargo run --release

debug:
    pushd guest && cargo build && popd && echo "done"
    mkdir -p wasm_modules && wasm-tools component new --skip-validation   ./target/wasm32-wasip1/debug/guest.wasm  --adapt ./wasi_snapshot_preview1.reactor.wasm -o wasm_modules/guest.wasm
    pushd host && cargo run

the additional `wasi_snapshot_preview1.reactor.wasm`

I got it from wasmtime download page, and I put a tgz file including the whole project in case it's needed.

This is the sync version, I tried the async version before, it had the same Thread 1 from guest code, so I simplified the code and make it sync as shown above. If necessary, I could provide the async version as well

Wasmtime GitHub notifications bot (Mar 08 2025 at 21:59):

xdlin edited a comment on issue #10359:

Actually I tried both:

sync mode
async mode

I thought it's a foundmantal question related with invocation mode, so I didn't paste the code, let's me post the minimal reproducible example here

the WIT file（`wit/server.wit`)

package my:service;

interface native {
    predict: func(input: string) -> u64;
}

interface guest {
    factor-get: func(id: string) -> string;
}

world factor-server {
    import native;
    export guest;
}

host code (`host/src/main.rs`)

use cargo_metadata::MetadataCommand;
use std::path::PathBuf;
use {
    anyhow::Result,
    wasmtime::{
        component::{Component, Linker, ResourceTable},
        Config, Engine, Store,
    },
    wasmtime_wasi::{IoView, WasiCtx, WasiCtxBuilder, WasiView},
};

mod handler {
    wasmtime::component::bindgen!({
        path: "../wit",
        world: "factor-server",
        async: false,
        concurrent_imports: false,
        concurrent_exports: false,
        trappable_imports: false,
    });
}

struct WasmStates {
    wasi: WasiCtx,
    table: ResourceTable,
}

impl WasmStates {
    pub fn new() -> Self {
        let table = ResourceTable::new();
        let wasi = WasiCtxBuilder::new().inherit_stdio().inherit_args().build();

        Self { table, wasi }
    }
}

impl IoView for WasmStates {
    fn table(&mut self) -> &mut ResourceTable {
        &mut self.table
    }
}
impl WasiView for WasmStates {
    fn ctx(&mut self) -> &mut WasiCtx {
        &mut self.wasi
    }
}

impl handler::my::service::native::Host for WasmStates {
    fn predict(&mut self, _input: String) -> u64 {
        println!(
            "host: called from guest at: {:?}",
            std::thread::current().id()
        );
        42
    }
}

fn get_workspace_root() -> PathBuf {
    let metadata = MetadataCommand::new().exec().unwrap();
    metadata.workspace_root.into()
}

#[tokio::main(flavor = "multi_thread")]
async fn main() -> Result<()> {
    let root = get_workspace_root();
    let wasm_path = root.join("wasm_modules/guest.wasm");

    let mut config = Config::new();
    config.wasm_component_model(true);
    config.async_support(false);
    config.debug_info(true);
    config.cranelift_debug_verifier(true);
    config.wasm_threads(true);

    println!("main: {:?}", std::thread::current().id());
    let mut handles = vec![];

    for _ in 0..4 {
        let handle = {
            let wasm_path = wasm_path.clone();
            let config = config.clone();
            std::thread::spawn(move || {
                let engine = Engine::new(&config).unwrap();
                let mut linker: Linker<WasmStates> = Linker::new(&engine);
                wasmtime_wasi::add_to_linker_sync(&mut linker).unwrap();
                handler::FactorServer::add_to_linker(&mut linker, |ctx: &mut WasmStates| ctx)
                    .unwrap();
                let component = Component::from_file(&engine, wasm_path).unwrap();
                let wasm_states = WasmStates::new();
                let mut store = Store::new(&engine, wasm_states);
                let instance = linker.instantiate(&mut store, &component).unwrap();

                let instance = handler::FactorServer::new(&mut store, &instance).unwrap();
                println!("host call guest.fun at: {:?}", std::thread::current().id());
                let res = instance
                    .my_service_guest()
                    .call_factor_get(&mut store, "foo")
                    .unwrap();
                res
            })
        };
        handles.push(handle);
    }

    for handle in handles {
        let res = handle.join().unwrap();
        println!("res: {}", res);
    }

    Ok(())
}

the guest code (`guest/src/lib.rs`)

mod bindings {
    wit_bindgen::generate!({
        path: "../wit",
        world: "factor-server",
    });

    use super::Component;
    export!(Component);
}

use bindings::{exports::my::service::guest::Guest, my::service::native};

struct Component;

impl Guest for Component {
    fn factor_get(id: String) -> String {
        println!("guest: call host.fun at: {:?}", wasm_thread::current().id());
        let res = native::predict(&id);
        println!(
            "guest: return from host.fun at: {:?}",
            wasm_thread::current().id()
        );
        format!("guest result: {}", res)
    }
}

Cargo.toml

workspace Cargo.toml

[workspace]
resolver = "2"
members = [
    "host",
    "guest",
]

host/Cargo.toml:

[package]
name = "host"
version = "0.1.0"
edition = "2021"

[dependencies]
anyhow = { version = "1.0.81", features = ["backtrace"] }
tokio = { version = "1.36.0", features = ["fs", "macros", "rt-multi-thread", "time"] }
wasmtime = { version = "*", features = ["component-model"]}
wasmtime-wasi = { version = "*"}
cargo_metadata = "0.18"
axum = "0.8.1"
rand = "0.9.0"

guest/Cargo.toml

[package]
name = "guest"
version = "0.1.0"
edition = "2021"

[dependencies]
futures = "0.3.30"
once_cell = "1.19.0"
wasm_thread = "0.3.3"
wit-bindgen = "0.40"
wit-bindgen-rt = "0.40"

[lib]
crate-type = ["cdylib"]

Makefile

In order to use component mode, here is the additional steps in Makefile

SHELL := /bin/bash
release:
    pushd guest && cargo build --release && popd && echo "done"
    mkdir -p wasm_modules && wasm-tools component new --skip-validation   ./target/wasm32-wasip1/release/guest.wasm  --adapt ./wasi_snapshot_preview1.reactor.wasm -o wasm_modules/guest.wasm
    pushd host && cargo run --release

debug:
    pushd guest && cargo build && popd && echo "done"
    mkdir -p wasm_modules && wasm-tools component new --skip-validation   ./target/wasm32-wasip1/debug/guest.wasm  --adapt ./wasi_snapshot_preview1.reactor.wasm -o wasm_modules/guest.wasm
    pushd host && cargo run

the additional `wasi_snapshot_preview1.reactor.wasm`

I got it from wasmtime download page, and I put a tgz file (wit_demo.tgz) including the whole project in case it's needed.

Wasmtime GitHub notifications bot (Mar 08 2025 at 22:31):

cfallin commented on issue #10359:

println!("guest: call host.fun at: {:?}", wasm_thread::current().id());

This is printing the guest's view of the current thread. You can think of each separate Wasm store, with Wasm instances, as a small virtual machine: the first thread will be "thread 1". That doesn't mean it's running on the host's thread 1.

Wasmtime GitHub notifications bot (Mar 08 2025 at 22:44):

xdlin commented on issue #10359:

Wow, nice catch, thanks for pointing it out, this do resolved my question.
Then I will try to focus on my own code optimization.

Thank you and have a sweet weekend~

Chris Fallin @.***>于2025年3月8日周六14:32写道：
   println!("guest: call host.fun at: {:?}", wasm_thread::current().id());
This is printing the guest's view of the current thread. You can think
of each separate Wasm store, with Wasm instances, as a small virtual
machine: the first thread will be "thread 1". That doesn't mean it's
running on the host's thread 1.

—
Reply to this email directly, view it on GitHub
<https://github.com/bytecodealliance/wasmtime/issues/10359#issuecomment-2708513991>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAGXHFEKUXXJJYA2VPNVCV32TNVWHAVCNFSM6AAAAABYTJK3RGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMBYGUYTGOJZGE>
.
You are receiving this because you authored the thread.Message ID:
@.***>
[image: cfallin]cfallin left a comment (bytecodealliance/wasmtime#10359)
<https://github.com/bytecodealliance/wasmtime/issues/10359#issuecomment-2708513991>
   println!("guest: call host.fun at: {:?}", wasm_thread::current().id());
This is printing the guest's view of the current thread. You can think
of each separate Wasm store, with Wasm instances, as a small virtual
machine: the first thread will be "thread 1". That doesn't mean it's
running on the host's thread 1.

—
Reply to this email directly, view it on GitHub
<https://github.com/bytecodealliance/wasmtime/issues/10359#issuecomment-2708513991>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAGXHFEKUXXJJYA2VPNVCV32TNVWHAVCNFSM6AAAAABYTJK3RGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMBYGUYTGOJZGE>
.
You are receiving this because you authored the thread.Message ID:
@.***>

Wasmtime GitHub notifications bot (Mar 08 2025 at 22:49):

cfallin closed issue #10359:

Quetion

Does wasmtime always run Wasm guest code in main thread?

Background

I plan to use Wasm as a plugin system for rust service, and call multiple host functions within the guest code, I found that the Wasm itself became the bottleneck, after doing some investigation, I found that whatever method I used, despite of the multiple threads Wasmtime Engine/Module/Instance I have in host code, __the same wasm file__ will always runs in the same thread (thread ID 1, the main thread)

There is the log I print in console, hopefully it's self explainable
main: ThreadId(1)
host call guest.fun at: ThreadId(16)
host call guest.fun at: ThreadId(15)
host call guest.fun at: ThreadId(17)
host call guest.fun at: ThreadId(14)
guest: call host.fun at: ThreadId(1)
guest: call host.fun at: ThreadId(1)
guest: call host.fun at: ThreadId(1)
host: called from guest at: ThreadId(16)
guest: return from host.fun at: ThreadId(1)
host: called from guest at: ThreadId(15)
guest: return from host.fun at: ThreadId(1)
host: called from guest at: ThreadId(17)
guest: call host.fun at: ThreadId(1)
host: called from guest at: ThreadId(14)
guest: return from host.fun at: ThreadId(1)
guest: return from host.fun at: ThreadId(1)
res: guest result: 42
res: guest result: 42
res: guest result: 42
res: guest result: 42
![Image](https://github.com/user-attachments/assets/8923a9f0-63dc-4480-a7a5-b3da012eed3c)

the issue

I'd like to improve the throughput of this service, but if the guest function alwasys runs in the same single thread, the overall throughput will depen on the time spending in guest code, let's say if it takes 10ms, the overall QPS will be limited to 100, even if I add more threads to support more wasmtime instances.

Is this by design, or I did I miss some critical configuration?

the requirement

I'd like to eliminate the bottleneck in single guest thread mode and improve service performance, which part should I change?

Last updated: Apr 17 2025 at 01:31 UTC

Stream: git-wasmtime

Topic: wasmtime / issue #10359 Question about Wasmtime thread model

Wasmtime GitHub notifications bot (Mar 08 2025 at 18:28):

Quetion

Background

the issue

the requirement

Wasmtime GitHub notifications bot (Mar 08 2025 at 18:30):

Quetion

Background

the issue

the requirement

Wasmtime GitHub notifications bot (Mar 08 2025 at 21:28):

Wasmtime GitHub notifications bot (Mar 08 2025 at 21:58):

the WIT file（wit/server.wit)

host code (host/src/main.rs)

the guest code (guest/src/lib.rs)

Cargo.toml

Makefile

the additional wasi_snapshot_preview1.reactor.wasm

Wasmtime GitHub notifications bot (Mar 08 2025 at 21:59):

the WIT file（wit/server.wit)

host code (host/src/main.rs)

the guest code (guest/src/lib.rs)

Cargo.toml

Makefile

the additional wasi_snapshot_preview1.reactor.wasm

Wasmtime GitHub notifications bot (Mar 08 2025 at 22:31):

Wasmtime GitHub notifications bot (Mar 08 2025 at 22:44):

Wasmtime GitHub notifications bot (Mar 08 2025 at 22:49):

Quetion

Background

the issue

the requirement

the WIT file（`wit/server.wit`)

host code (`host/src/main.rs`)

the guest code (`guest/src/lib.rs`)

the additional `wasi_snapshot_preview1.reactor.wasm`

the WIT file（`wit/server.wit`)

host code (`host/src/main.rs`)

the guest code (`guest/src/lib.rs`)

the additional `wasi_snapshot_preview1.reactor.wasm`