wasmtime / Issue #2578 How to define extern func with ref... · git-wasmtime

Stream: git-wasmtime

Topic: wasmtime / Issue #2578 How to define extern func with ref...

Wasmtime GitHub notifications bot (Jan 13 2021 at 14:17):

clearloop opened Issue #2578:

Desc

Here I'm trying to define the HostFuncType<T> below as extern functions in wasmtime, but I don't know how to handle the &mut T
pub type HostFuncType<T> = fn(&mut T, &[Value]) -> Result<ReturnValue, HostError>;
It might be possible using ExternRef<Arc<RefCell<T>>> instead to do something like this, but if there is any solution can pass the &mut T without get/set memory directly?

Wasmtime GitHub notifications bot (Jan 13 2021 at 14:18):

clearloop edited Issue #2578:

Desc

Here I'm trying to define the HostFuncType<T> below as extern functions in wasmtime, but I don't know how to handle the &mut T
pub type HostFuncType<T> = fn(&mut T, &[Value]) -> Result<ReturnValue, HostError>;
It might be possible using ExternRef<Arc<RefCell<T>>> instead to do something like this, but if there is any solution can pass the &mut T without get/set memory directly?

Wasmtime GitHub notifications bot (Jan 13 2021 at 14:19):

clearloop edited Issue #2578:

Desc

Here I'm trying to define the HostFuncType<T> below as extern functions in wasmtime, but I don't know how to handle the &mut T
pub type HostFuncType<T> = fn(&mut T, &[Value]) -> Result<ReturnValue, HostError>;
It might be possible using ExternRef<Arc<RefCell<T>>> instead to do something like this, but if there is any solution can pass the &mut T without breaking the arguments of the HostFuncType<T> or get/set memory directly?

Wasmtime GitHub notifications bot (Jan 13 2021 at 14:19):

clearloop edited Issue #2578:

Desc

Here I'm trying to define the HostFuncType<T> below as extern functions in wasmtime, but I don't know how to handle the &mut T
pub type HostFuncType<T> = fn(&mut T, &[Value]) -> Result<ReturnValue, HostError>;
It might be possible using ExternRef<Arc<RefCell<T>>> instead of doing something like this, but if there is any solution can pass the &mut T without breaking the arguments of the HostFuncType<T> or get/set memory directly?

Wasmtime GitHub notifications bot (Jan 13 2021 at 14:32):

clearloop edited Issue #2578:

Desc

Here I'm trying to define the HostFuncType<T> below as extern functions in wasmtime, but I don't know how to handle the &mut T
pub type HostFuncType<T> = fn(&mut T, &[Value]) -> Result<ReturnValue, HostError>;
It might be possible using ExternRef<Arc<RefCell<T>>> instead of doing something like this, but if there is any solution can pass the &mut T without breaking the arguments of the HostFuncType<T>?

Wasmtime GitHub notifications bot (Jan 13 2021 at 15:18):

alexcrichton commented on Issue #2578:

Thanks for the report! Can you clarify what you're trying to do, however? Host functions are generally defined with the Func type, which should have a number of examples on it as well. Are you unsure on how to create a Func, though?

Wasmtime GitHub notifications bot (Jan 13 2021 at 15:27):

clearloop commented on Issue #2578:

Thanks for your relying @alexcrichton ~

Thanks for the report! Can you clarify what you're trying to do, however? Host functions are generally defined with the Func type, which should have a number of examples on it as well. Are you unsure on how to create a Func, though?

Yep, I'm trying to implement a Func like:
fn my_host_func<T>(state: &mut T, args: &[val]) {}

fn main() -> Result<()> {
    // ...

    let my_extern_host_func = Func::wrap(&store, |state: &mut T, args: &[Val]| {
        my_host_func(state, args);
    });

    let instance = Instance::new(&store, &module, &[my_extern_host_func.into()])?;

    //...

    Ok(())
}

Wasmtime GitHub notifications bot (Jan 13 2021 at 15:31):

alexcrichton commented on Issue #2578:

When using &[Val] you'll want to use Func::new, and state: &mut T can't be given to you from wasm, so it can't be a parameter in the closure passed to Func::wrap. What you can do, however, is close over the state in the closure you provide to Func::wrap. If you need mutable access you'll need to use something like RefCell to account for the possibility that your closure is invoked recursively.

Wasmtime GitHub notifications bot (Jan 13 2021 at 15:31):

clearloop edited a comment on Issue #2578:

Thanks for your relying @alexcrichton ~

Thanks for the report! Can you clarify what you're trying to do, however? Host functions are generally defined with the Func type, which should have a number of examples on it as well. Are you unsure on how to create a Func, though?

Yep, I'm trying to implement a Func like:

fn my_host_func<T>(state: &mut T, args: &[val]) {}

fn main() -> Result<()> {
    // ...

    let my_extern_host_func = Func::wrap(&store, |state: &mut T, args: &[Val]| {
        my_host_func(state, args);
    });

    let instance = Instance::new(&store, &module, &[my_extern_host_func.into()])?;

    //...
    Ok(())
}

I can handle this in wasmi easily because the invoke methods of functions in wasmi always comes with external: &mut T:

    pub fn invoke<T: Externals>(
        func: &FuncRef,
        args: &[RuntimeValue],
        externals: &mut T,
    ) -> Result<Option<RuntimeValue>, Trap> {
        check_function_args(func.signature(), &args)?;
        match *func.as_internal() {
            FuncInstanceInternal::Internal { .. } => {
                let mut interpreter = Interpreter::new(func, args, None)?;
                interpreter.start_execution(externals)
            }
            FuncInstanceInternal::Host {
                ref host_func_index,
                ..
            } => externals.invoke_index(*host_func_index, args.into()),
        }
    }

Wasmtime GitHub notifications bot (Jan 13 2021 at 15:43):

clearloop commented on Issue #2578:

When using &[Val] you'll want to use Func::new, and state: &mut T can't be given to you from wasm, so it can't be a parameter in the closure passed to Func::wrap. What you can do, however, is close over the state in the closure you provide to Func::wrap. If you need mutable access you'll need to use something like RefCell to account for the possibility that your closure is invoked recursively.

Thanks!

At least I'm certain about I can't pass state as a parameter now, I'll try to move state into the closure, afraid of not using state as a parameter, functions implemented in wasm could not recognize the signature of the host functions.

I'll do more practice and close this issue as I implement it.

Ref to https://github.com/bytecodealliance/wasmtime/issues/1678

Wasmtime GitHub notifications bot (Jan 14 2021 at 03:39):

clearloop commented on Issue #2578:

Is there any example of making a function with &[Val] as parameters into a Func?

Wasmtime GitHub notifications bot (Jan 14 2021 at 04:11):

clearloop edited a comment on Issue #2578:

~~Is there any example of making a function with &[Val] as parameters into a Func?~~

Wasmtime GitHub notifications bot (Jan 14 2021 at 04:50):

clearloop edited a comment on Issue #2578:

Is there any example of making a function with &[Val] as parameters into a Func?

Wasmtime GitHub notifications bot (Jan 14 2021 at 04:50):

clearloop edited a comment on Issue #2578:

Is there any example of converting a function with &[Val] as parameters into a Func?

Wasmtime GitHub notifications bot (Jan 14 2021 at 09:02):

clearloop edited a comment on Issue #2578:

Is there any example of converting a function with &[Val] as parameters into a Func without using Func::new?

Wasmtime GitHub notifications bot (Jan 14 2021 at 09:21):

clearloop edited a comment on Issue #2578:

Is there any example of converting a function with &[Val] as parameters into a Func without using Func::new? Just find that Func::new requires Fn which can not use variables from the environment.

Wasmtime GitHub notifications bot (Jan 14 2021 at 09:23):

clearloop edited a comment on Issue #2578:

Is there any example of converting a function with &[Val] as parameters into a Func? Just find that both Func::new and Func::wrap require Fn which can not use variables from the environment. How could I move the RefCell<T> into closure?

Wasmtime GitHub notifications bot (Jan 14 2021 at 09:27):

clearloop edited a comment on Issue #2578:

Is there any example of converting a function with &[Val] as parameters into a Func? Just find that both Func::new and Func::wrap require 'static which can not move variables from the environment easily.

Wasmtime GitHub notifications bot (Jan 14 2021 at 09:33):

clearloop edited a comment on Issue #2578:

Is there any example of converting a function with &[Val] as parameters into a Func? Just find that both Func::new and Func::wrap require 'static which can not move variables from the environment easily.

Could Store offer a generic type filed for holding the state of host functions?

Wasmtime GitHub notifications bot (Jan 14 2021 at 09:33):

clearloop edited a comment on Issue #2578:

Is there any example of converting a function with &[Val] as parameters into a Func? Just find that both Func::new and Func::wrap require 'static which can not move variables from the environment easily.

Could Store offer a generic type field for holding the state of host functions?

Wasmtime GitHub notifications bot (Jan 14 2021 at 09:45):

clearloop edited a comment on Issue #2578:

Is there any example of converting a function with &[Val] as parameters into a Func? Just find that both Func::new and Func::wrap require 'static which can not move variables from the environment into them easily.

Could Store offer a generic type field for holding the state of host functions?

Wasmtime GitHub notifications bot (Jan 14 2021 at 09:50):

clearloop edited a comment on Issue #2578:

Is there any example of converting a function with &[Val] as parameters into a Func? Just find that both Func::new and Func::wrap require 'static which can not move variables from the environment into them easily.

error[E0525]: expected a closure that implements the `Fn` trait, but this closure only implements `FnOnce`
  --> primitives/sandbox/src/imp/wasmtime/util.rs:67:13
   |
67 |     let func = move |_: Caller<'_>, args: &[Val], results: &mut [Val]| {
   |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this closure implements `FnOnce`, not `Fn`
...
77 |         match unsafe { f(state, &inner_args) } {
   |                          ----- closure is `FnOnce` because it moves the variable `state` out of its environment
...
89 |     Func::new(store, wasmtime_sig(sig), func)
   |     --------- the requirement to implement `Fn` derives from here

Could Store offer a generic type field for holding the state of host functions?

Wasmtime GitHub notifications bot (Jan 14 2021 at 09:51):

clearloop edited a comment on Issue #2578:

pub type HostFuncTypeA<T> = fn(RefCell<T>, &[Value]) -> Result<ReturnValue, crate::HostError>;

pub fn wrap_fn_test<T>(
    store: &Store,
    state: RefCell<T>,
    f: HostFuncTypeA<T>,
    sig: FunctionType,
) -> Func {
    // let state_mut = state as *mut T;
    let func = move |_: Caller<'_>, args: &[Val], results: &mut [Val]| {
        let mut inner_args = vec![];
        for arg in args {
            if let Some(arg) = from_val(arg.clone()) {
                inner_args.push(arg);
            } else {
                return Err(Trap::new("Could not wrap host function"));
            }
        }

        match unsafe { f(state, &inner_args) } {
            Ok(ret) => {
                if let Some(ret) = from_ret_val(ret) {
                    results = &mut [ret];
                }
                Ok(())
            }
            Err(_) => Err(Trap::new("Could not wrap host function")),
        }

        // return Err(Trap::new("Could not wrap host function"));
    };
    Func::new(store, wasmtime_sig(sig), func)
}

error[E0525]: expected a closure that implements the `Fn` trait, but this closure only implements `FnOnce`
  --> primitives/sandbox/src/imp/wasmtime/util.rs:67:13
   |
67 |     let func = move |_: Caller<'_>, args: &[Val], results: &mut [Val]| {
   |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this closure implements `FnOnce`, not `Fn`
...
77 |         match unsafe { f(state, &inner_args) } {
   |                          ----- closure is `FnOnce` because it moves the variable `state` out of its environment
...
89 |     Func::new(store, wasmtime_sig(sig), func)
   |     --------- the requirement to implement `Fn` derives from here

Could Store offer a generic type field for holding the state of host functions?

Wasmtime GitHub notifications bot (Jan 14 2021 at 10:29):

clearloop edited a comment on Issue #2578:

pub type HostFuncTypeA<T> = fn(RefCell<T>, &[Value]) -> Result<ReturnValue, crate::HostError>;

pub fn wrap_fn_test<T>(
    store: &Store,
    state: RefCell<T>,
    f: HostFuncTypeA<T>,
    sig: FunctionType,
) -> Func {
    // let state_mut = state as *mut T;
    let func = move |_: Caller<'_>, args: &[Val], results: &mut [Val]| {
        let mut inner_args = vec![];
        for arg in args {
            if let Some(arg) = from_val(arg.clone()) {
                inner_args.push(arg);
            } else {
                return Err(Trap::new("Could not wrap host function"));
            }
        }

        match unsafe { f(state, &inner_args) } {
            Ok(ret) => {
                if let Some(ret) = from_ret_val(ret) {
                    results = &mut [ret];
                }
                Ok(())
            }
            Err(_) => Err(Trap::new("Could not wrap host function")),
        }

        // return Err(Trap::new("Could not wrap host function"));
    };
    Func::new(store, wasmtime_sig(sig), func)
}

error[E0525]: expected a closure that implements the `Fn` trait, but this closure only implements `FnOnce`
  --> primitives/sandbox/src/imp/wasmtime/util.rs:67:13
   |
67 |     let func = move |_: Caller<'_>, args: &[Val], results: &mut [Val]| {
   |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this closure implements `FnOnce`, not `Fn`
...
77 |         match unsafe { f(state, &inner_args) } {
   |                          ----- closure is `FnOnce` because it moves the variable `state` out of its environment
...
89 |     Func::new(store, wasmtime_sig(sig), func)
   |     --------- the requirement to implement `Fn` derives from here

or some errors like

error[E0310]: the parameter type `T` may not live l
[message truncated]

Wasmtime GitHub notifications bot (Jan 14 2021 at 15:37):

alexcrichton commented on Issue #2578:

In tthis case HostFuncTypeA will need to take &RefCell<T>, not a by-value version.

Wasmtime GitHub notifications bot (Jan 14 2021 at 15:44):

clearloop commented on Issue #2578:

In tthis case HostFuncTypeA will need to take &RefCell<T>, not a by-value version.

error[E0621]: explicit lifetime required in the type of `state`
  --> primitives/sandbox/src/imp/wasmtime/util.rs:93:2
   |
66 |     state: &RefCell<T>,
   |            ----------- help: add explicit lifetime `'static` to the type of `state`: `&'static RefCell<T>`
...
93 |     Func::new(store, wasmtime_sig(sig), func)
   |     ^^^^^^^^^ lifetime `'static` required

error: aborting due to previous error

If use &RefCell, it will require 'static, I'm not sure if this situation is possible, not sure about if I have to add 'static to T.

Wasmtime GitHub notifications bot (Jan 14 2021 at 15:48):

clearloop edited a comment on Issue #2578:

In this case HostFuncTypeA will need to take &RefCell<T>, not a by-value version.
error[E0621]: explicit lifetime required in the type of `state`
  --> primitives/sandbox/src/imp/wasmtime/util.rs:93:2
   |
66 |     state: &RefCell<T>,
   |            ----------- help: add explicit lifetime `'static` to the type of `state`: `&'static RefCell<T>`
...
93 |     Func::new(store, wasmtime_sig(sig), func)
   |     ^^^^^^^^^ lifetime `'static` required

error: aborting due to previous error
If using &RefCell, it will require 'static, I'm not sure if this situation is possible, not sure about if I have to add 'static to T.

Thanks for your reply @alexcrichton~ I know there might be some problems in my implementation, I just can't figure out how to make it work, I have stuck at this point for 1 week then : (

Wasmtime GitHub notifications bot (Jan 14 2021 at 15:53):

clearloop edited a comment on Issue #2578:

In this case HostFuncTypeA will need to take &RefCell<T>, not a by-value version.

error[E0621]: explicit lifetime required in the type of `state`
  --> primitives/sandbox/src/imp/wasmtime/util.rs:93:2
   |
66 |     state: &RefCell<T>,
   |            ----------- help: add explicit lifetime `'static` to the type of `state`: `&'static RefCell<T>`
...
93 |     Func::new(store, wasmtime_sig(sig), func)
   |     ^^^^^^^^^ lifetime `'static` required

error: aborting due to previous error

If using &RefCell, it will require 'static, I'm not sure if this situation is possible, not sure about if I have to add 'static to T.

Thanks for your reply @alexcrichton~ I know there might be some problems in my implementation, I just can't figure out how to make it work, I have stuck at this point for 1 week then : (

pub type HostFuncTypeA<T> = fn(&RefCell<T>, &[Value]) -> Result<ReturnValue, crate::HostError>;

pub fn wrap_fn_test<T>(
    store: &Store,
    state: &'static RefCell<T>,
    f: HostFuncTypeA<T>,
    sig: FunctionType,
) -> Func {
    // let state_mut = state as *mut T;
    let func = move |_: Caller<'_>, args: &[Val], results: &mut [Val]| {
        let mut inner_args = vec![];
        for arg in args {
            if let Some(arg) = from_val(arg.clone()) {
                inner_args.push(arg);
            } else {
                return Err(Trap::new("Could not wrap host function"));
            }
        }

        match f(state, &inner_args) {
            Ok(ret) => {
                if let Some(ret) = from_ret_val(ret) {
                    results[0] = ret;
                }
                Ok(())
            }
            Err(_) => Err(Trap::new("Could not wrap host function")),
        }
    };
    Func::new(store, wasmtime_sig(sig), func)
}

It could be compiled like this, but if I have to use 'static?

Wasmtime GitHub notifications bot (Jan 14 2021 at 16:22):

clearloop edited a comment on Issue #2578:

When using &[Val] you'll want to use Func::new, and state: &mut T can't be given to you from wasm, so it can't be a parameter in the closure passed to Func::wrap. What you can do, however, is close over the state in the closure you provide to Func::wrap. If you need mutable access you'll need to use something like RefCell to account for the possibility that your closure is invoked recursively.

Thanks!

At least I'm certain about I can't pass state as a parameter now, I'll try to move state into the closure, afraid of not using state as a parameter, functions implemented in wasm could not recognize the signature of the host functions.

I'll do more practice and close this issue as I implement it.

Ref to https://github.com/bytecodealliance/wasmtime/issues/1678, https://github.com/bytecodealliance/wasmtime/issues/2491

Wasmtime GitHub notifications bot (Jan 14 2021 at 16:50):

clearloop commented on Issue #2578:

Just got an example following #2491, https://github.com/chifflier/suricata/blob/295091c2e6598e84519d92b87dbe24d31c5fd070/rust/src/wasm/runtime_util.rs, but using Func::wrap()

Wasmtime GitHub notifications bot (Jan 15 2021 at 03:31):

clearloop commented on Issue #2578:

Closing this because I'm certain about the problem in my implementation is not a wasmtime problem now.

Wasmtime GitHub notifications bot (Jan 15 2021 at 03:31):

clearloop closed Issue #2578:

Desc

Here I'm trying to define the HostFuncType<T> below as extern functions in wasmtime, but I don't know how to handle the &mut T
pub type HostFuncType<T> = fn(&mut T, &[Value]) -> Result<ReturnValue, HostError>;
It might be possible using ExternRef<Arc<RefCell<T>>> instead of doing something like this, but if there is any solution can pass the &mut T without breaking the arguments of the HostFuncType<T>?

Wasmtime GitHub notifications bot (Jan 15 2021 at 06:26):

clearloop edited a comment on Issue #2578:

When using &[Val] you'll want to use Func::new, and state: &mut T can't be given to you from wasm, so it can't be a parameter in the closure passed to Func::wrap. What you can do, however, is close over the state in the closure you provide to Func::wrap. If you need mutable access you'll need to use something like RefCell to account for the possibility that your closure is invoked recursively.

Thanks!

At least I'm certain about I can't pass state as a parameter now, I'll try to move state into the closure, afraid of not using state as a parameter, functions implemented in wasm could not recognize the signature of the host functions.

I'll do more practice and close this issue as I implement it.

Ref to https://github.com/bytecodealliance/wasmtime/issues/1678, https://github.com/bytecodealliance/wasmtime/issues/2491, https://github.com/bytecodealliance/wasmtime/issues/2159

Last updated: Dec 23 2024 at 12:05 UTC