wasmtime / issue #9848 Error linking to function that tak... · git-wasmtime

Stream: git-wasmtime

Topic: wasmtime / issue #9848 Error linking to function that tak...

Wasmtime GitHub notifications bot (Dec 18 2024 at 01:08):

viridia added the bug label to Issue #9848.

Wasmtime GitHub notifications bot (Dec 18 2024 at 01:08):

I'm getting this error:

types incompatible: expected type `(func (param (ref array (engine 5))))`, found type `(func (param (ref array)))`

The wasm looks like this:

(type $String (;3;) (array i8))
(type $debug.type (;4;) (func (param (ref $String))))
(import "host" "debug" (func $debug (;0;) (type $debug.type)))

And I'm registering the function like this:

let engine = wasmtime::Engine::new(&config).unwrap();
let store = wasmtime::Store::<StoreData>::new(&engine, ());
let mut linker = wasmtime::Linker::new(&engine);
linker
    .func_wrap(
        "host",
        "debug",
        |caller: Caller<'_, StoreData>, param: Rooted<ArrayRef>| {
            // ...
        },
    )
    .unwrap();

Test Case

(module $hello.saga
  (type $hello.type (;0;) (func (result i32)))
  (type $add.type (;1;) (func (param i32) (result i32)))
  (type $name.type (;2;) (func))
  (type $String (;3;) (array (mut i8)))
  (type $debug.type (;4;) (func (param (ref $String))))
  (import "host" "debug" (func $debug (;0;) (type $debug.type)))
  (export "hello" (func $hello))
  (func $hello (;1;) (type $hello.type) (result i32)
    i32.const 1
    call $add
    i32.const 2
    i32.add
    f32.const 0x1.8p+1 (;=3;)
    i32.trunc_f32_s
    i32.add
    return
  )
  (func $add (;2;) (type $add.type) (param i32) (result i32)
    (local i32)
    i32.const 1
    local.set 1
    local.get 0
    local.get 1
    i32.add
    return
  )
  (func $name (;3;) (type $name.type)
    i32.const 0
    i32.const 15
    array.new_data $String 0
    call $debug
  )
  (data (;0;) "Hello from SAGA")
)

Steps to Reproduce

Create an Engine and Linker.
Bind a function which accepts an ArrayRef as a parameter.
Attempt to load the script.

Expected Results

Script should load without error (it loads if I delete the code that calls the host func).

Actual Results

Error when I attempt to instantiate the script.

Versions and Environment

Wasmtime version or commit: { version = "27.0.0", features = ["gc", "gc-drc"] }

Operating system: OS X

Architecture: M2

Wasmtime GitHub notifications bot (Dec 18 2024 at 15:54):

alexcrichton assigned fitzgen to issue #9848.

Wasmtime GitHub notifications bot (Dec 18 2024 at 17:16):

fitzgen commented on issue #9848:

Thanks for filing an issue. I won't have time to look into this for a while, unfortunately, but I appreciate getting it on the record for posterity.

Wasmtime GitHub notifications bot (Dec 19 2024 at 17:51):

fitzgen added the wasmtime:ref-types label to Issue #9848.

Wasmtime GitHub notifications bot (Dec 19 2024 at 17:51):

fitzgen added the wasm-proposal:gc label to Issue #9848.

Wasmtime GitHub notifications bot (Dec 19 2024 at 17:52):

github-actions[bot] commented on issue #9848:

Subscribe to Label Action

cc @fitzgen

<details>
This issue or pull request has been labeled: "wasmtime:ref-types"

Thus the following users have been cc'd because of the following labels:

fitzgen: wasmtime:ref-types

To subscribe or unsubscribe from this label, edit the <code>.github/subscribe-to-label.json</code> configuration file.

Learn more.
</details>

Wasmtime GitHub notifications bot (Dec 19 2024 at 17:53):

fitzgen commented on issue #9848:

FWIW, this is probably related to the WasmTy implementation for ArrayRef

Wasmtime GitHub notifications bot (Jan 23 2025 at 01:04):

fitzgen-f5 commented on issue #9848:

Hm so I converted this into a test in tests/all/arrays.rs:

#[test]
fn instantiate_with_func_that_takes_array() -> Result<()> {
    let mut store = gc_store()?;

    let module = Module::new(
        store.engine(),
        r#"
            (module
                (type $String (;3;) (array i8))
                (type $debug.type (;4;) (func (param (ref $String))))
                (import "host" "debug" (func $debug (;0;) (type $debug.type)))
            )
        "#,
    )?;

    let mut linker = wasmtime::Linker::new(store.engine());
    linker.func_wrap("host", "debug", |_: Caller<'_, ()>, _: Rooted<ArrayRef>| {})?;

    let _instance = linker.instantiate(&mut store, &module)?;
    Ok(())
}

And I am indeed reproducing the error:

Error: incompatible import type for `host::debug`

Caused by:
    types incompatible: expected type `(func (param (ref array (engine 0))))`, found type `(func (param (ref array)))`

However, I'm not convinced that this is actually incorrect behavior. The type of the wrapped function is (func (param (ref array))) which indeed is not a subtype of (func (param (ref array $String))) and so the MatchCx used to typecheck instance linking is correctly yielding a type error.

If we make $debug.type non-final (which it implicitly was originally) and switch from using linker.func_wrap to Func::new and explicitly provide a function whose type is a subtype of $debug.type, then the test does indeed pass:

#[test]
fn instantiate_with_func_that_takes_array() -> Result<()> {
    let mut store = gc_store()?;

    let module = Module::new(
        store.engine(),
        r#"
            (module
                (type $String (;3;) (array i8))
                (type $debug.type (;4;) (sub (func (param (ref $String)))))
                (import "host" "debug" (func $debug (;0;) (type $debug.type)))
            )
        "#,
    )?;

    let mut linker = wasmtime::Linker::new(store.engine());

    let string_ty = ArrayType::new(
        store.engine(),
        FieldType::new(Mutability::Const, StorageType::I8),
    );
    let sup_ty = FuncType::with_finality_and_supertype(
        store.engine(),
        Finality::NonFinal,
        None,
        [ValType::Ref(RefType::new(
            false,
            HeapType::ConcreteArray(string_ty),
        ))],
        None,
    )?;
    let sub_ty = FuncType::with_finality_and_supertype(
        store.engine(),
        Finality::NonFinal,
        Some(&sup_ty),
        [ValType::Ref(RefType::new(false, HeapType::Array))],
        None,
    )?;
    linker.func_new("host", "debug", sub_ty, |_caller, _args, _rets| Ok(()))?;

    let _instance = linker.instantiate(&mut store, &module)?;
    Ok(())
}

test arrays::instantiate_with_func_that_takes_array ... ok

That is all technically correct, but I'm not sure if it is what we ultimately want. It seems very reasonable to want to write code like in the OP, even if it technically doesn't match subtyping rules (but would if the types involved were non-final and declared to be subtypes; basically those that structurally match but are not actually declared subtypes).

One extra twist is that I think we actually used to allow this kind of thing, but I think I removed it when getting the GC wast tests passing, since (IIRC) there were some assert_unlinkable pragmas exercising this stuff. Makes sense to disallow in that context, but maybe not in the context of a wasmtime::Linker in the host? Not totally convinced either way.

Wasmtime GitHub notifications bot (Jan 23 2025 at 17:58):

alexcrichton commented on issue #9848:

I always forget the directionality of subtyping but I'm confused why this doesn't pass the subtyping check. If I ask for (ref array) and you give me (ref $specific_array) isn't that ok? Shouldn't the functions be subtypes here? (or is the "final" bit coming into play, I have less of a grasp on that)

Otherwise though I'd agree that we want something Func::wrap-related to work here. That's much more efficient, works better with bindings generators, and is generally more ergonomic. How best to thread that needle though I'm not sure either.

Wasmtime GitHub notifications bot (Jan 23 2025 at 18:10):

fitzgen commented on issue #9848:

The issue is twofold:

First, that the wrapped function's type, when fully desugared, is
(sub final (func (param (ref array))))
which does not declare itself to be a subtype of $debug.type and therefore is not a subtype of $debug.type even if the underlying parameter and result types match.

And second, that
(type $debug.type (func (param (ref $String)))
desugars to a final type
(type $debug.type (sub final (param (ref $String))))
so no other function type can be a subtype of $debug.type as written, even if it tried.

This all makes sense inside the world of pure Wasm, but from a host API and developer niceties point of view, it becomes pretty frustrating and leads to bad developer experience :-/

Wasmtime GitHub notifications bot (Jan 23 2025 at 18:11):

fitzgen edited a comment on issue #9848:

The issue is twofold:

First, that the wrapped function's type, when fully desugared, is
(sub final (func (param (ref array))))
which does not declare itself to be a subtype of $debug.type and therefore is not a subtype of $debug.type even if the underlying parameter and result types match.

And second, that
(type $debug.type (func (param (ref $String)))
desugars to a final type
(type $debug.type (sub final (func (param (ref $String)))))
so no other function type can be a subtype of $debug.type as written, even if it tried.

This all makes sense inside the world of pure Wasm, but from a host API and developer niceties point of view, it becomes pretty frustrating and leads to bad developer experience :-/

Wasmtime GitHub notifications bot (Jan 23 2025 at 20:22):

alexcrichton commented on issue #9848:

Ah I see, I was mistakenly thinking about the (sub ...) in the wrong place!

I thought we had something though which was testing that as long as the top type was the same then the host-level type-check passed? And then we optionally included a runtime type-check if the actual type was different from the top type? I'm probably misremembering here...

Another option perhaps is to do something different for wasm gc than we do for normal scalars. Some sort of argument passed to Func::wrap_new_name which is acquired via constructing the type or parsing it out somehow? Maybe that's just a slightly more ergonomic Func::new in that case ...

I suppose put another way one thing we could do is to change Func::new to take any closure, not just the signature today. The function type is then used to perform a runtime type-check against the closure's static type and for today's signature it'd be a noop but you could pass a more specialized signature like Func::wrap. That specialized signature would have a slightly different type-check than the subtyping checks we have today?

In any case I'm just musing here...

Wasmtime GitHub notifications bot (Jan 24 2025 at 23:14):

fitzgen commented on issue #9848:

I thought we had something though which was testing that as long as the top type was the same then the host-level type-check passed? And then we optionally included a runtime type-check if the actual type was different from the top type? I'm probably misremembering here...

We do that for casting an untyped function to a typed function at the Rust/host level, and for matching the underlying Wasm types against the given Rust types, but not when comparing two Wasm function types, which is what linking bottoms out in.

I suppose put another way one thing we could do is to change Func::new to take any closure, not just the signature today. The function type is then used to perform a runtime type-check against the closure's static type and for today's signature it'd be a noop but you could pass a more specialized signature like Func::wrap. That specialized signature would have a slightly different type-check than the subtyping checks we have today?

I was also thinking along these lines. Something like Func::wrap_with_func_type that would take the desired function type as well as the closure, and then do the fuzzy-matching stuff to make sure that the closure is valid for the given function type.

The other other idea I was thinking was adding a flag to MatchCx that is like whether to do strict subtype checking or fuzzy, shallow-structural matching. For the component model, and statically linking two Wasm modules together, we would not set that flag to fuzzy mode. For dynamically linking to host functions, we would set the flag to fuzzy mode. But this also gets weird if you just dynamically link two Wasm modules together, which would "accidentally" set the fuzzy mode flag and allow linking two Wasm modules together that it technically shouldn't. So I guess in that case we could instead check if the function is a host function under the covers or not, but this is all getting into the weeds...

Wasmtime GitHub notifications bot (Jan 27 2025 at 17:57):

alexcrichton commented on issue #9848:

Ok that makes sense to me too yeah. I'm also hesitant to add too many flags to the linker type-matching since that's pretty critical for correctness and such, so maybe wrap_with_func_type is the way to go for now?

Wasmtime GitHub notifications bot (Jan 28 2025 at 19:39):

fitzgen commented on issue #9848:

so maybe wrap_with_func_type is the way to go for now?

Sounds good to me :+1:

Last updated: Feb 28 2025 at 03:10 UTC