Stream: wit-bindgen

Topic: Invoke guest module in a thread

Mossaka (Joe) (Jun 21 2022 at 23:39):

I used wit-bindgen to implement a callback mechanism - the guest will invoke a host-exported function exec(), which invoke a guest-exported function event_handler(). Here is what the guest module looks like in Rust

wit_bindgen_rust::import!("../exec.wit");
wit_bindgen_rust::export!("../event-handler.wit");

fn main() {
    exec::exec();
}

pub struct EventHandler {}

impl event_handler::EventHandler for EventHandler {
    fn event_handler(event: String) -> String {
        event
    }
}

Then I implemented the exec() function in the host to start a new thread and then invoke the event_hanlder() function.

let t = thread::spawn(move || {
  let handler = handler.lock().unwrap();
  handler.event_handler(store, "event-a").unwrap();
});
t.join().unwrap();

This compiles, but then when I ran the program, I got a StackOverflow error at the unwrapping of event_handler() function call.

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value:
Trap { reason: InstructionTrap(StackOverflow), wasm_trace: [FrameInfo { module_name: None, func_index: 283, func_name: Some("canonical_abi_realloc.command_export"), func_start: FilePos(62777), instr: Some(FilePos(4294967295)), symbols: [] }], native_trace:    0: <unknown>

If I removed the thread::spawn() part, then the program runs without any runtime errors.

You can find the complete source code at here

Could I get some insights on why this happens? Thanks!

Alex Crichton (Jun 21 2022 at 23:55):

I believe the unsafe code you've written is causing memory unsafety, which is the problem here.

Mossaka (Joe) (Jun 22 2022 at 00:04):

@Alex Crichton thanks for your response! The unsafe block is to circumvent Rust ownership checker, because both handler.event_handler(store, "event-a") and instance.call(store) need mutable references to wasmtime::Store<T>. The main branch of the repo does not run into Stack Overflow runtime errors with the same unsafe blocks. It only occurs when the guest function was invokved in a thread. This is why I am confused.

Alex Crichton (Jun 22 2022 at 00:10):

Oh sure that part I got, but that's the unsafe part, you don't have access to that. If you specifically subvert wasmtime's API then bad things like this happen

Alex Crichton (Jun 22 2022 at 00:11):

the wit-bindgen-generated bindings don't give you access to the store during the duration of host calls, but that's by design today to enable other optimizations. For now I'd recommend you use the Wasmtime API directly which does enable this and in the future we can perhaps alter wit-bindgen to better accommodate your use case

Mossaka (Joe) (Jun 22 2022 at 00:14):

Okay it makes sense. Which wasmtime API were you hinting that can give me more flexible access to the store?

Mossaka (Joe) (Jun 22 2022 at 00:20):

As a side note, I've heard that callbackor higher-order functions won't be a thing in wasm component model proposal. Do you think defining callback as a separate function is a reasonable thing to do if the application needs reactive-style programming? @Alex Crichton

Alex Crichton (Jun 22 2022 at 00:41):

With Wasmtime directly host functions get access to a Caller<T> which is a proxy for StoreContextMut<T>. For reactive-style programming I'm not so sure, but no the component model isn't planning on getting first-class functions or similiar right now

Mossaka (Joe) (Jun 22 2022 at 17:56):

Thanks @Alex Crichton , I have a follow up question. It seems like the Caller<T> parameter won't save me from using unsafe in this case. My host is invoking main function exposed by the guest, and the main function has a callback to a handler function in the guest. So the Store<T> will be mutably borrowed by the host in invoking main, and thus Store can't be borrowed again for invoking the handler function. Do you have any thoughts on how to solve this owernship problem?

Alex Crichton (Jun 22 2022 at 17:59):

The Caller<T> gives access to StoreContextMut<T> which is effectively a mutable store, so although you give up a mutable store as part of the first call you're given back the mutable store when the host gets called

Mossaka (Joe) (Jun 22 2022 at 19:11):

Thanks Alex!! I was able to implement the same callback semantics without unsafe block now, given that I didn't use wit_bindgen_wasmtime::export!() and used wasmtime API directly. If you are interested, here is the code

Mossaka (Joe) (Jun 22 2022 at 19:33):

oops, link is broken. Here is the new one.

Last updated: Nov 22 2024 at 16:03 UTC