Stream: general

Topic: Linking Example

Tim Park (Nov 04 2021 at 20:15):

This is potentially a newbie question - but I'm experimenting with the linker.rs wasmtime example. That example links together two .wat modules into a coherent program, which has fascinating possibilities of course.

My questions is: Is there an example somewhere of how you could build write something in Rust that would result in the two .wat files linking1.wat and linking2.wat?

Alex Crichton (Nov 04 2021 at 20:28):

AFAIK right now this isn't easily possible, it's the realm of "dynamic linking" for wasm which while it isn't really dynamic in the sense that you can have a static module graph it's dynamic in the sense that each module may be loaded at a different position in linear memory. I believe Emscripten has done a lot of work in this regard to get things and at least a basic story working, but I'm not aware of anyone implementing the story in Rust and/or demoing how it's all supposed to look

Tim Park (Nov 04 2021 at 20:34):

Ok - thanks @Alex Crichton

Tim Park (Nov 05 2021 at 21:41):

Following on to this - I experimented with witx-bindgen this morning to build the imports and exports bindings for Rust versions of linking1.wat and linking2.wat and what I'm seeing (i think) is that in the log function that it is referencing memory in that module and not the calling module like is intended. Is that what you meant with your comment above @Alex Crichton?

ala - Vec::from_raw_parts below from witx-bindgen will reference arg0, len0 in this module's memory and not the callers:

mod log {
    #[export_name = "log"]
    unsafe extern "C" fn __witx_bindgen_log(arg0: i32, arg1: i32) {
        let len0 = arg1 as usize;

        println!("ptr: {}", arg0);
        println!("len0: {}", len0);

        <super::DefaultLog as Log>::log(
            String::from_utf8(Vec::from_raw_parts(arg0 as *mut _, len0, len0)).unwrap(),
        );
    }
    pub trait Log {
        fn log(msg: String);
    }
}

use crate::log::Log;

struct DefaultLog {}

impl Log for DefaultLog {
    fn log(msg: String) {
        println!("{}", msg);
    }
}

Tim Park (Nov 06 2021 at 17:44):

I updated this utilizing the witx-bindgen-rust crate I found into these two modules:

The "app" (aka what would generate linking1.wat ):

witx_bindgen_rust::import!("../log/witx/log.witx");

#[no_mangle]
pub fn run() {
    log::log("Hello World");
}

and the "log" module (aka what would generate linking2.wat from the linking example):

witx_bindgen_rust::export!("witx/log.witx");

struct Log;

impl log::Log for Log {
    fn log(msg: String) {
        println!("this string is from this module and will be printed as msg because it is using this module's memory");
        println!("{}", msg);
    }
}

When I run this with this linking.rs example I get the following output

this string is from this module and will be printed as msg because it is using this module's memory
this string

instead of Hello World because it seems to be using log's memory and not the app that called it. Is there a way to fix this currently or is that the implementation ga?

bjorn3 (Nov 06 2021 at 20:02):

I think you need to import log from the log impl module in the code using wasmtime and then export it again to the app. witx-bindgen requires both sides of the interface to share the same linear memory. Otherwise it obviously can't copy from the one to the other side. Having two wasm modules share the same linear memory would require dynamic linking which LLVM doesn't support, so the only option I think is to have the wasmtime using code be an intermediary between the two wasm modules.

Alex Crichton (Nov 08 2021 at 15:41):

@Tim Park witx-bindgen and interface types in general

Alex Crichton (Nov 08 2021 at 15:42):

er, sry, early send, but in general it's intended for "components" which are isolated from each other with memory, which means that the linking example in the repository don't work since witx-generated things rely on not actually sharing memory. Something in the middle wiring up needs to copy between memories (this is what interface types does)

Alex Crichton (Nov 08 2021 at 15:42):

so it's expected that this example doesn't work

Tim Park (Nov 08 2021 at 21:00):

Thank you @Alex Crichton - ok got it - so to summarize: witx is for single components right now and there isn't any way right now to express (or accomplish) that copy between memories without dropping into WAT - but it is on the "roadmap" to make this possible via Interface Types (or otherwise) depending on where that conversation ends up.

Alex Crichton (Nov 08 2021 at 21:01):

indeed!

Peter Huene (Nov 08 2021 at 21:57):

@Tim Park right now one may use wasmlink to link "components" together into another wasm file, where the new wasm file contains generated code to marshal data between the individual component memories based on the interfaces. it's a prototype and one that might not get much further development, but feel free to kick the tires

Tim Park (Nov 17 2021 at 02:53):

Thanks for the pointer to this @Peter Huene - finally had a chance to try this - and it works great. Very exciting stuff!

Last updated: Nov 22 2024 at 16:03 UTC