Stream: git-wasmtime

Topic: wasmtime / issue #8428 Degradation in real-time performance

Wasmtime GitHub notifications bot (Apr 22 2024 at 09:08):

daichifukui added the bug label to Issue #8428.

Wasmtime GitHub notifications bot (Apr 22 2024 at 09:08):

daichifukui opened issue #8428:

Describe the bug

We are experiencing degradation in real-time performance with the latest version of Wasmtime runtime.

Conditions

• wasmtime-cli 21.0.0 (dac3bdb07 2024-04-14)
• cargo-wasi 0.1.28 (e811d4889b 2023-06-12)
• rustc 1.77.2 (25ef9e3d8 2024-04-09)

Steps to reproduce

$ cat src/main.rs
use std::{thread, time};
use std::time::{Instant};

fn main() {
    let duration = time::Duration::from_millis(1*1000);

    for _ in 0u32.. 10 {
        let start = Instant::now();
        thread::sleep(duration);
        let elapsed_time = start.elapsed();
        println!("Time elapsed is: {:?}", elapsed_time);
    }
}


$ cargo wasi build --release

$ /path/to/wasmtime/target/release/wasmtime --version
wasmtime-cli 21.0.0 (dac3bdb07 2024-04-14)

$ /path/to/wasmtime/target/release/wasmtime run target/wasm32-wasi/release/rust_clock_nanosleep.wasm
Time elapsed is: 1.00209518s
Time elapsed is: 1.001975393s
Time elapsed is: 1.002069423s
Time elapsed is: 1.001098972s
Time elapsed is: 1.001260921s
Time elapsed is: 1.002105879s
Time elapsed is: 1.002093792s
Time elapsed is: 1.002016636s
Time elapsed is: 1.001522486s
Time elapsed is: 1.001914764s

Expected behaviour

The latency is expected to vary in microseconds.
FYI, using an older version of wasmtime such as v8.0.1, the performance is as follows.

$ /path/to/wasmtime-v8.0.1-x86_64-linux/wasmtime target/wasm32-wasi/release/rust_clock_nanosleep.wasm
Time elapsed is: 1.000509875s
Time elapsed is: 1.000589676s
Time elapsed is: 1.000437833s
Time elapsed is: 1.000510245s
Time elapsed is: 1.000517657s
Time elapsed is: 1.000513208s
Time elapsed is: 1.000521429s
Time elapsed is: 1.000516253s
Time elapsed is: 1.000525438s
Time elapsed is: 1.000400342s

In addition, if we disable preview2 through the -S option, with the latest version of wasmtime, the performance gets better than enabling preview2:

$ /path/to/wasmtime/target/release/wasmtime run -Spreview2=n target/wasm32-wasi/release/rust_clock_nanosleep.wasm
Time elapsed is: 1.000515945s
Time elapsed is: 1.000522065s
Time elapsed is: 1.000437704s
Time elapsed is: 1.000522172s
Time elapsed is: 1.000532635s
Time elapsed is: 1.000460092s
Time elapsed is: 1.000514949s
Time elapsed is: 1.000519259s
Time elapsed is: 1.000514321s
Time elapsed is: 1.00051906s

Actual behaviour

The latency varies in about milliseconds.

$ /path/to/wasmtime/target/release/wasmtime run target/wasm32-wasi/release/rust_clock_nanosleep.wasm
Time elapsed is: 1.00209518s
Time elapsed is: 1.001975393s
Time elapsed is: 1.002069423s
Time elapsed is: 1.001098972s
Time elapsed is: 1.001260921s
Time elapsed is: 1.002105879s
Time elapsed is: 1.002093792s
Time elapsed is: 1.002016636s
Time elapsed is: 1.001522486s
Time elapsed is: 1.001914764s

As we can see expected behaviour when disabling preview2, it is considered that the performance degradation is related to the difference between the implimentations of preview1, meaning that the difference between wasmtime-wasi and wasi-common has something to do with this issue.

Additional context

In the background, we are planning to use Wasmtime on top of a control system.
We believe that using a WASM runtime would make applications for control systems more portable and scalable.
This is because WASM uses a CPU-agnostic binary format and consumes less computing resource than a container runtime.
As timing is critical in control systems, we would like to have better real-time performance with WASM runtimes.
WAMR is designed as a lightweight standalone WASM runtime and can be used for a control system, but it primarily implements an interpreter, so this could potentially lead to slower execution times compared to a JIT compiler like Wasmtime.

Wasmtime GitHub notifications bot (Apr 25 2024 at 16:25):

alexcrichton closed issue #8428:

Describe the bug

We are experiencing degradation in real-time performance with the latest version of Wasmtime runtime.

Conditions

• wasmtime-cli 21.0.0 (dac3bdb07 2024-04-14)
• cargo-wasi 0.1.28 (e811d4889b 2023-06-12)
• rustc 1.77.2 (25ef9e3d8 2024-04-09)

Steps to reproduce

$ cat src/main.rs
use std::{thread, time};
use std::time::{Instant};

fn main() {
    let duration = time::Duration::from_millis(1*1000);

    for _ in 0u32.. 10 {
        let start = Instant::now();
        thread::sleep(duration);
        let elapsed_time = start.elapsed();
        println!("Time elapsed is: {:?}", elapsed_time);
    }
}


$ cargo wasi build --release

$ /path/to/wasmtime/target/release/wasmtime --version
wasmtime-cli 21.0.0 (dac3bdb07 2024-04-14)

$ /path/to/wasmtime/target/release/wasmtime run target/wasm32-wasi/release/rust_clock_nanosleep.wasm
Time elapsed is: 1.00209518s
Time elapsed is: 1.001975393s
Time elapsed is: 1.002069423s
Time elapsed is: 1.001098972s
Time elapsed is: 1.001260921s
Time elapsed is: 1.002105879s
Time elapsed is: 1.002093792s
Time elapsed is: 1.002016636s
Time elapsed is: 1.001522486s
Time elapsed is: 1.001914764s

Expected behaviour

The latency is expected to vary in microseconds.
FYI, using an older version of wasmtime such as v8.0.1, the performance is as follows.

$ /path/to/wasmtime-v8.0.1-x86_64-linux/wasmtime target/wasm32-wasi/release/rust_clock_nanosleep.wasm
Time elapsed is: 1.000509875s
Time elapsed is: 1.000589676s
Time elapsed is: 1.000437833s
Time elapsed is: 1.000510245s
Time elapsed is: 1.000517657s
Time elapsed is: 1.000513208s
Time elapsed is: 1.000521429s
Time elapsed is: 1.000516253s
Time elapsed is: 1.000525438s
Time elapsed is: 1.000400342s

In addition, if we disable preview2 through the -S option, with the latest version of wasmtime, the performance gets better than enabling preview2:

$ /path/to/wasmtime/target/release/wasmtime run -Spreview2=n target/wasm32-wasi/release/rust_clock_nanosleep.wasm
Time elapsed is: 1.000515945s
Time elapsed is: 1.000522065s
Time elapsed is: 1.000437704s
Time elapsed is: 1.000522172s
Time elapsed is: 1.000532635s
Time elapsed is: 1.000460092s
Time elapsed is: 1.000514949s
Time elapsed is: 1.000519259s
Time elapsed is: 1.000514321s
Time elapsed is: 1.00051906s

Actual behaviour

The latency varies in about milliseconds.

$ /path/to/wasmtime/target/release/wasmtime run target/wasm32-wasi/release/rust_clock_nanosleep.wasm
Time elapsed is: 1.00209518s
Time elapsed is: 1.001975393s
Time elapsed is: 1.002069423s
Time elapsed is: 1.001098972s
Time elapsed is: 1.001260921s
Time elapsed is: 1.002105879s
Time elapsed is: 1.002093792s
Time elapsed is: 1.002016636s
Time elapsed is: 1.001522486s
Time elapsed is: 1.001914764s

As we can see expected behaviour when disabling preview2, it is considered that the performance degradation is related to the difference between the implimentations of preview1, meaning that the difference between wasmtime-wasi and wasi-common has something to do with this issue.

Additional context

In the background, we are planning to use Wasmtime on top of a control system.
We believe that using a WASM runtime would make applications for control systems more portable and scalable.
This is because WASM uses a CPU-agnostic binary format and consumes less computing resource than a container runtime.
As timing is critical in control systems, we would like to have better real-time performance with WASM runtimes.
WAMR is designed as a lightweight standalone WASM runtime and can be used for a control system, but it primarily implements an interpreter, so this could potentially lead to slower execution times compared to a JIT compiler like Wasmtime.

Last updated: Dec 23 2024 at 12:05 UTC