Stream: git-wasmtime

Topic: wasmtime / PR #3691 Add copy-on-write based instance reus...

Wasmtime GitHub notifications bot (Jan 14 2022 at 11:12):

koute opened PR #3691 from main_cow_instance_reuse to main:

This PR adds a new copy-on-write based instance reuse mechanism on Linux.

Usage

The general idea is - you instantiate your instance once, and then you can reset its state back to how it was when it was initially instantiated.

    // Initialize a new instance. This is done only once.
    let mut store = Store::new(&engine, ());
    let instance_pre = linker.instantiate_pre(&mut store, &module)?;
    let instance = instance_pre.instantiate_reusable(&mut store)?;

    loop {
        // Call methods, modify memory, modify the table, etc.
        use_instance(instance, &mut store)?;

        // Reset the instance to its initial state.
        instance.reset(&mut store)?;
    }

How does it work?

After the instance is instantiated a snapshot of its state is taken. Tables and globals are simply cloned into a spare Vec, while the memory is copied into an memfd and remapped inplace in a copy-on-write fashion. Then when reset is called the tables and the globals are restored by simply copying them back over, and the memory is reset using madvise(MADV_DONTNEED) which either clears the memory (for those pages which are not mapped to the memfd) or restores the original contents (for those pages which are mapped to the memfd).

Benchmarks

In our benchmarks this is currently the fastest way to call into a WASM module assuming you rarely need to instantiate it from scratch.



Legend:
- instance_pooling_with_uffd: create a fresh instance with InstanceAllocationStrategy::Pooling strategy with uffd turned on
- instance_pooling_without_uffd: create a fresh instance with InstanceAllocationStrategy::Pooling strategy without uffd turned on
- recreate_instance: create a fresh instance with InstanceAllocationStrategy::OnDemand strategy
- native_instance_reuse: this PR
- interpreted: just for our own reference; an instance created through the wasmi crate
- legacy_instance_reuse: just for our own reference; this is what we're currently using - it is an instance spawned with InstanceAllocationStrategy::OnDemand strategy and then reused after manually clearing its memory and restoring its data segments and globals

The two of the benchmarks shown here are:
- call_empty_function: an empty function is called in a loop, resetting (or recreating) the instance after each call
- dirty_1mb_of_memory: a function which dirties 1MB of memory and then returns is called in a loop, resetting (or recreating) the instance after each call

The measurements are only for the main thread; thread count on the bottom signifies how many other threads were running in the background doing exactly the same thing as the main thread, e.g. for 4 threads there was 1 thread (the main thread) being benchmarked while other 3 threads were running in the background.

For your reference the benchmarks used to generate these graphs can be found here:

https://github.com/koute/substrate/tree/master_wasmtime_benchmarks

Which can be run like this after cloning the repository:

$ cd substrate/client/executor/benches

# `instance_pooling_with_uffd`
$ FORCE_WASMTIME_INSTANCE_POOLING=1 rustup run nightly-2021-11-01-x86_64-unknown-linux-gnu cargo bench --features wasmtime,sc-executor-wasmtime/uffd --bench bench -- with_recreate_instance

# `instance_pooling_without_uffd`
$ FORCE_WASMTIME_INSTANCE_POOLING=1 rustup run nightly-2021-11-01-x86_64-unknown-linux-gnu cargo bench --features wasmtime --bench bench -- with_recreate_instance

# `recreate_instance`
FORCE_WASMTIME_INSTANCE_POOLING=0 rustup run nightly-2021-11-01-x86_64-unknown-linux-gnu cargo bench --features wasmtime --bench bench -- with_recreate_instance

# `native_instance_reuse`
rustup run nightly-2021-11-01-x86_64-unknown-linux-gnu cargo bench --features wasmtime --bench bench -- native_instance_reuse

(The rustc version is just for reference as to what I used. Also please forgive the hacky way the benchmarks have to be launched for instance pooling; we don't intend to keep this codepath, so I quckly hacked it in only for the benchmarks.)

The benchmarks were run on the following machine:

AMD Threadripper 3970x (32-core)
64GB of RAM
Linux 5.14.16

Possible future work (missing features)

• Add support for other OSes (this is Linux-only for now)
• Add support for imported memories (this only supports modules which do not import memory)
• Add support for taking snapshots at an arbitrary points in time (it always takes a snapshot right after instantiation)
• Add the ability to customize what is restored on reset (it always resets everything)
• Add address-space pooling to make the initial initialization faster (for cases where new modules are frequently instantiated from scratch)
  

Wasmtime GitHub notifications bot (Jan 14 2022 at 11:20):

koute updated PR #3691 from main_cow_instance_reuse to main.

Wasmtime GitHub notifications bot (Jan 14 2022 at 11:27):

koute submitted PR review.

Wasmtime GitHub notifications bot (Jan 14 2022 at 11:27):

koute created PR review comment:

I'm pretty sure this is necessary for correctness, but when I comment it out nothing fails. Any idea how I could write a test which would verify that this is here?

Wasmtime GitHub notifications bot (Jan 14 2022 at 11:27):

bjorn3 submitted PR review.

Wasmtime GitHub notifications bot (Jan 14 2022 at 11:27):

bjorn3 created PR review comment:

This doesn't work if /proc isn't mounted. In addition it doesn't check /proc is actually a mounted procfs. Rustix has code to check if the /proc is sane. It doesn't seem like it is exported though.

Wasmtime GitHub notifications bot (Jan 14 2022 at 15:55):

programmerjake created PR review comment:

why not just pass by value?

Wasmtime GitHub notifications bot (Jan 14 2022 at 15:55):

programmerjake submitted PR review.

Wasmtime GitHub notifications bot (Jan 17 2022 at 09:14):

koute updated PR #3691 from main_cow_instance_reuse to main.

Wasmtime GitHub notifications bot (Jan 17 2022 at 09:15):

koute submitted PR review.

Wasmtime GitHub notifications bot (Jan 17 2022 at 09:15):

koute created PR review comment:

A remnant from before when I had a different implementation where MemorySource was heavier so it wasn't Copy.

Fixed!

Wasmtime GitHub notifications bot (Jan 17 2022 at 09:36):

koute updated PR #3691 from main_cow_instance_reuse to main.

Wasmtime GitHub notifications bot (Jan 17 2022 at 09:37):

koute submitted PR review.

Wasmtime GitHub notifications bot (Jan 17 2022 at 09:37):

koute created PR review comment:

Good point.

I've made a PR to rustix here to export it: https://github.com/bytecodealliance/rustix/pull/174

Wasmtime GitHub notifications bot (Jan 17 2022 at 10:49):

koute updated PR #3691 from main_cow_instance_reuse to main.

Wasmtime GitHub notifications bot (Jan 17 2022 at 11:29):

koute updated PR #3691 from main_cow_instance_reuse to main.

Wasmtime GitHub notifications bot (Jan 17 2022 at 11:58):

koute updated PR #3691 from main_cow_instance_reuse to main.

Wasmtime GitHub notifications bot (Jan 18 2022 at 11:53):

koute updated PR #3691 from main_cow_instance_reuse to main.

Wasmtime GitHub notifications bot (Feb 23 2022 at 10:00):

koute closed without merge PR #3691.

Last updated: Dec 23 2024 at 12:05 UTC