Questions about wasm sandbox interruption and resume · general

Hi guys, I try to call wasmtime C API to create multiple wasm instances and round robin to run them based on specified duration and then resume them. I think epoch based interruption can do this, but I am not sure. Does the C API now support what I want to do? I want to interrupt some instance and let another instance run, later on, I need to resume the previous interrupted instance, is this doable? Thank you very much.

Alex Crichton (Aug 06 2025 at 21:10):

lyuxiaosu (Aug 06 2025 at 21:25):

@Alex Crichton Thank you very much for the reply. I saw most examples just terminate the interrupted wasm sandbox, not resume them, is there any example file showing how to resume the interrupted sandbox?

Alex Crichton (Aug 06 2025 at 21:26):

The examples of async may be a bit light on details, but it should be possible to resume a suspended async function

lyuxiaosu (Aug 06 2025 at 21:28):

Thank you very much to confirm this, I will see async APIs and async example file more carefully to figure this out :grinning_face_with_smiling_eyes:

lyuxiaosu (Aug 13 2025 at 17:30):

Hi @Alex Crichton , I wrote a test program with C API. I create an instance with ASYN and epoch interruption support. I initially call wasmtime_context_epoch_deadline_async_yield_and_update(context, 1); to set the deadline to current_epoch + 1. Another thread will call wasmtime_engine_increment_epoch to increase the current epoch every 10 seconds. I loop call wasmtime_call_future_poll, then it returns false every 10 seconds, indicating the deadline setting and increasing current epoch works. However, within the loop calling wasmtime_call_future_poll at some point, like the counter equals to a specified number, I want to interrupt the instance immediately, I call wasmtime_context_epoch_deadline_async_yield_and_update(context, 0);, I thought this will trigger the instance yield immediately, however, it seems it doesn't work, it still triggers the yield every 10 seconds, not the one that I reset the deadline, is something I understood wrong or I missed? Thank you for your help

Alex Crichton (Aug 13 2025 at 17:42):

ah yes that's expected, the only way to immediately interrupt an instance is to change the epoch, you can't change the epoch/yield counters while the instance is running (that's UB to do that from the C API)

lyuxiaosu (Aug 13 2025 at 18:39):

Thank you for your reply @Alex Crichton . Then how I can let an instance yield immediately? When should I change the epoch? You said "you can't change the epoch/yield counters while the instance is running", then after the wasmtime_call_future_poll return false, the instance is alive but pending, is it a "running" instance that you mean? Does that mean there is no way to let an running instance to yield immediately? Thank you for your help

Alex Crichton (Aug 13 2025 at 18:54):

Ah, to clarify, once you've created the future computation you can't edit the store/instance at all. The future "owns" the store/instance at that time so you can't update the store-local yield counter (e.g. wasmtime_context_* methods). What you can do is increment the epoch counter in the engine from another thread, that's always safe to do.

To force an instance to yield immediately while it's currently executing you'd have to previously arrange, before the execution, for the instance to yield at some epoch deadline (e.g. the async_yield_and_update method). Then to yield immediately you'd force the epoch to reach that point (e.g. increment it once or however many times necessary). That'll force the thread to yield and the poll will return "not done"

lyuxiaosu (Aug 13 2025 at 18:59):

Oh, thank you for the explanation. I want to do some real-time scheduling, that means we cannot plan in advance sometime, if a more urgent task comes in, I need to let some instance yield immediately and then run the new task. It seems the current C API cannot support this. Do I understand correctly?

Alex Crichton (Aug 13 2025 at 19:01):

I'm not sure I fully understand, because if the store is configured to yield at every epoch then you could set it up to yield periodically and if something urgent comes in you do an early update of the epoch. On the yield you'd then determine whether to resume if nothing else needed priority or whether to resume.

Basically I wouldn't say that the C API can't support this use case necessarily, but it may require subtle changes in how you model the problem

lyuxiaosu (Aug 13 2025 at 19:06):

"and if something urgent comes in you do an early update of the epoch", I used wasmtime_context_epoch_deadline_async_yield_and_update(context, 0); to update the deadline when I want to let an running instance yield in my test code , however, it doesn't work. The code is like this:

// And call it!
  printf("Entering infinite loop...\n");
  int count = 0;
  while (!wasmtime_call_future_poll(call_future.get())) {
    count++;
    if (count == 2) {
      wasmtime_engine_increment_epoch(engine.get());
      wasmtime_context_epoch_deadline_async_yield_and_update(context, 0);
      printf("let yield immediately\n");
    }
    std::cout << "Yield, pending..." << std::endl;
  }
  printf("Excecution finished\n");
  return 0;

did I use wrong, it shouldn't be update the deadline within this loop calling wasmtime_call_future_poll?

Alex Crichton (Aug 13 2025 at 19:18):

You might be able to fix this by swapping the increment/epoch_deadline lines. In general though while a future is running you can't call wasmtime_context_* functions as that's UB. If the epoch deadline is already 1 then incrementing the epoch should be all you need to make the future exit

lyuxiaosu (Aug 13 2025 at 19:40):

I tried to swap increment/epoch_deadline lines, or even removed epoch deadline update but called wasmtime_engine_increment_epoch multiple times, but it the same thing, not yield immediately but periodically yield at the previous setting interval. I paste my test code:

template <typename T, void (*fn)(T *)> struct deleter {
  void operator()(T *ptr) { fn(ptr); }
};
template <typename T, void (*fn)(T *)>
using handle = std::unique_ptr<T, deleter<T, fn>>;

static void *helper(void *_engine) {
  wasm_engine_t *engine = (wasm_engine_t*)_engine;
  while(true) {
    struct timespec sleep_dur;
    sleep_dur.tv_sec = 5;
    sleep_dur.tv_nsec = 0;
    printf("sleep for 5 seconds\n");
    nanosleep(&sleep_dur, NULL);
    printf("Sending an interrupt\n");
    wasmtime_engine_increment_epoch(engine);
  }
  return 0;
}

static void spawn_interrupt(wasm_engine_t *engine) {
  pthread_t child;
  int rc = pthread_create(&child, NULL, helper, engine);
  assert(rc == 0);
}
#endif

static void exit_with_error(const char *message, wasmtime_error_t *error,
                            wasm_trap_t *trap);

handle<wasm_engine_t, wasm_engine_delete> create_engine(wasm_config_t *config) {
  assert(config != nullptr);
  wasmtime_config_async_support_set(config, true);
  wasmtime_config_epoch_interruption_set(config, true);
  //wasmtime_config_consume_fuel_set(config, true);
  handle<wasm_engine_t, wasm_engine_delete> engine;
  // this takes ownership of config
  engine.reset(wasm_engine_new_with_config(config));
  assert(engine);
  return engine;
}

handle<wasmtime_store_t, wasmtime_store_delete>
create_store(wasm_engine_t *engine) {
  handle<wasmtime_store_t, wasmtime_store_delete> store;
  store.reset(wasmtime_store_new(engine, nullptr, nullptr));
  assert(store);
  return store;
}

handle<wasmtime_linker_t, wasmtime_linker_delete>
create_linker(wasm_engine_t *engine) {
  handle<wasmtime_linker_t, wasmtime_linker_delete> linker;
  linker.reset(wasmtime_linker_new(engine));
  assert(linker);
  return linker;
}

handle<wasmtime_module_t, wasmtime_module_delete>
compile_wat_module_from_file(wasm_engine_t *engine,
                             const std::string &filename) {
  std::ifstream t(filename);
  std::stringstream buffer;
  buffer << t.rdbuf();
  if (t.bad()) {
    std::cerr << "error reading file: " << filename << std::endl;
    std::exit(1);
  }
  const std::string &content = buffer.str();
  wasm_byte_vec_t wasm_bytes;
  handle<wasmtime_error_t, wasmtime_error_delete> error{
      wasmtime_wat2wasm(content.data(), content.size(), &wasm_bytes)};
  if (error) {
    exit_with_error("failed to parse wat", error.get(), nullptr);
  }
  wasmtime_module_t *mod_ptr = nullptr;
  error.reset(wasmtime_module_new(engine,
                                  reinterpret_cast<uint8_t *>(wasm_bytes.data),
                                  wasm_bytes.size, &mod_ptr));
  wasm_byte_vec_delete(&wasm_bytes);
  handle<wasmtime_module_t, wasmtime_module_delete> mod{mod_ptr};
  if (!mod) {
    exit_with_error("failed to compile module", error.get(), nullptr);
  }
  return mod;
}

int main() {
  // Create a `wasm_store_t` with interrupts enabled
  wasm_config_t *config = wasm_config_new();
  assert(config != NULL);
  handle<wasmtime_error_t, wasmtime_error_delete> error;

  auto engine = create_engine(config);
  assert(engine != NULL);
  auto store = create_store(engine.get());
  assert(store != NULL);
  wasmtime_context_t *context = wasmtime_store_context(store.get());

  // Configure the epoch deadline after which WebAssembly code will yield.
  wasmtime_context_epoch_deadline_async_yield_and_update(context, 1);

  // Read our input file, which in this case is a wasm text file.
  auto compiled_module =
      compile_wat_module_from_file(engine.get(), "examples/my_interrupt.wat");
  auto linker = create_linker(engine.get());

  // Now instantiate our module using the linker.
  handle<wasmtime_call_future_t, wasmtime_call_future_delete> call_future;
  wasm_trap_t *trap_ptr = nullptr;
  wasmtime_error_t *error_ptr = nullptr;
  wasmtime_instance_t instance;
  call_future.reset(wasmtime_linker_instantiate_async(
      linker.get(), context, compiled_module.get(), &instance, &trap_ptr,
      &error_ptr));
  while (!wasmtime_call_future_poll(call_future.get())) {
    std::cout << "yielding instantiation!" << std::endl;
  }
  error.reset(error_ptr);
  handle<wasm_trap_t, wasm_trap_delete> trap{trap_ptr};
  if (error || trap) {
    exit_with_error("failed to instantiate module", error.get(), trap.get());
  }

  call_future = nullptr;
  linker = nullptr;
  // Lookup our `run` export function
  wasmtime_extern_t run;
  bool ok = wasmtime_instance_export_get(context, &instance, "run", 3, &run);
  assert(ok);
  assert(run.kind == WASMTIME_EXTERN_FUNC);

  //get future
  std::array<wasmtime_val_t, 0> results;
  call_future.reset(wasmtime_func_call_async(
      context, &run.of.func, NULL, 0,
      results.data(), results.size(), &trap_ptr, &error_ptr));

  if (error_ptr) {
    error.reset(error_ptr);
    exit_with_error("error during async call", error.get(), nullptr);
  }

  if (trap_ptr) {
    handle<wasm_trap_t, wasm_trap_delete> trap{trap_ptr};
    exit_with_error("trap during async call", nullptr, trap.get());
  }
  // Spawn a thread to send us an interrupt after a period of time.
  spawn_interrupt(engine.get());

  // And call it!
  printf("Entering infinite loop...\n");
  int count = 0;
  while (!wasmtime_call_future_poll(call_future.get())) {
    count++;
    if (count == 2) {
      wasmtime_engine_increment_epoch(engine.get());
      wasmtime_engine_increment_epoch(engine.get());
      wasmtime_engine_increment_epoch(engine.get());
      //wasmtime_context_epoch_deadline_async_yield_and_update(context, 0);
      printf("let yield immediately\n");
    }
    std::cout << "Yield, pending..." << std::endl;
  }
  printf("Excecution finished\n");
  return 0;
}

static void exit_with_error(const char *message, wasmtime_error_t *error,
                            wasm_trap_t *trap) {
  fprintf(stderr, "error: %s\n", message);
  wasm_byte_vec_t error_message;
  if (error != NULL) {
    wasmtime_error_message(error, &error_message);
    wasmtime_error_delete(error);
  } else {
    wasm_trap_message(trap, &error_message);
    wasm_trap_delete(trap);
  }
  fprintf(stderr, "%.*s\n", (int)error_message.size, error_message.data);
  wasm_byte_vec_delete(&error_message);
  exit(1);
}

Entering infinite loop...
sleep for 5 seconds
Yield, pending...

(after 5 seconds)
Sending an interrupt
sleep for 5 seconds
let yield immediately
Yield, pending...

(after 5 seconds)
Sending an interrupt
sleep for 5 seconds
Yield, pending...

......repeat

Alex Crichton (Aug 13 2025 at 19:47):

Ah ok I expected that to work but I see why it's not working now. The problem is that in the implementation of the "new epoch point" it yields before calculating the new epoch. That means that your increments during the yield don't end up doing anything because the next epoch point is calculated relative away from the current epoch on resumption.

lyuxiaosu (Aug 13 2025 at 20:52):

OK, thank you very much to find out the root reason though I don't fully understand the reason and the rust code, but at least I know immediate yield cannot do at this point.

Miles R-L (Aug 15 2025 at 05:20):

Hi @Alex Crichton following up on this -- is it possible to do similar with the Rust async APIs (I need to drive the futures by hand if I want to have control over when to resume?)

And, can I do this using the fuel approach or only epochs (I'm fine either way but trying to plan ahead a bit).
For context I really, really loved the call_resumable in wasmi but I really, really love the components / canonical ABI in wasmtime -- would love to have my cake and eat it too lol

Miles R-L (Aug 15 2025 at 05:44):

It seems like typically I would drive this with an actual async runtime, but I don't have to. The thing I'm not clear on is, is it really as simple as setting up periodic yields by fuel / epoch, and then polling whenever I'm okay with it doing another batch of work? Do I need a waker or other fancy stuff?

Lann Martin (Aug 15 2025 at 12:25):

If your guest needs to call back into the host (like with wasi) then those calls will also need to be async and if those calls use existing async libraries then you will probably need a fully functional async executor.

Alex Crichton (Aug 15 2025 at 14:45):

Wasmtime doesn't have a perfect equivalent to call_resumable in wasmi but the async support should get you what you want. It looks like in wasmi a function can be "interrupted" either for running out of fuel or manually being raised in the host. In Wasmtime a function is "interrupted" when it async-yields which could happen for the same reasons. If in Wasmtime you want a host function to manually trigger an async point you could drain the remaining fuel before the next yield point with the store in the host function call.

Both fuel and epochs can be used for async yield points. I think you can build everything necessary with the APIs in Wasmtime today, but if you find anything is missing please feel free to open an issue.

I'll note that this isn't necessary actually, when async_support is enabled in Wasmtime it's required that all entrypoints into wasm are async (e.g. call_async) but host functions can be synchronous, so there's no need to use an async runtime, you'll just have to call poll on the call_async future yourself

Lann Martin (Aug 15 2025 at 15:05):

Miles R-L (Aug 16 2025 at 04:52):

I am really loving it, and I'm going to start talking people's ears off to see what I can unlock with this.

Tbh for several years I've been looking for an embeddable programming environment that can be fully sandboxed, stepped in fixed quantities and accept full, production programming languages and I'm pretty sure this is it :)

Stream: general

Topic: Questions about wasm sandbox interruption and resume

lyuxiaosu (Aug 06 2025 at 18:58):