Stream: wasmtime

Topic: ✔ How do I configure `bindgen!` to work with dependencies?


view this post on Zulip Samuel Mwangi (Jul 24 2024 at 21:48):

I have with files in a wit folder as follows:

wit/
├── base-pkg/
   └── base-pkg.wit
├── super-pkg.wit

I have configured the bindgen! macro for super-pkg.wit as follows:

bindgen!({
    path: "wit",
    with: {
        "pkg:super-pkg": crate::mod_name::pkg::base_pkg,
    },
    trappable_imports: true,
});

and its (super-pkg.wit) contains the following statements:

package pkg:super-pkg;

use pkg:base-pkg/types as base-pkg-types;

...

world cloud-pattern-world {
    import pkg:base-pkg/types;
    import node-info;

    export evaluator;
}

I get the error:

error: failed to resolve directory while parsing WIT for path [/wit]

       Caused by:
           package not found
                --> /wit/super-pkg.wit:3:5
                 |
               3 | use pkg:base-pkg/types as base-pkg-types;
                 |     ^-----------------

What am I missing?

view this post on Zulip Alex Crichton (Jul 24 2024 at 22:03):

I think you'll need to put base-pkg in a deps folder, so wit/deps/base-pkg

view this post on Zulip Samuel Mwangi (Jul 24 2024 at 22:23):

Thank you, @Alex Crichton.

Putting base-pkg in a deps folder has solved the above bindings generation issue.

However, I am now getting the error:

error[E0405]: cannot find trait `Host` in module `__with_name1::types`
  --> /target/debug/build/wasmtime-component-macro-b0355b6157b884bd/out/super-pkg-world0.rs:99:37
   |
99 |             U: __with_name1::types::Host + pkg::super_pkg::base_pkg::Host,
   |                                     ^^^^ not found in `__with_name1::types`
   |
help: consider importing one of these items
  --> common/wasm-evaluators/src/super_pkg/mod.rs:6:1
   |
6  + use crate::super_pkg::pkg::super_pkg::base_pkg::Host;
  --> common/wasm-evaluators/src/super_pkg/mod.rs:6:1
   |
6  + use crate::base_pkg::pkg::base_pkg::db::Host;
  --> common/wasm-evaluators/src/super_pkg/mod.rs:6:1
   |
6  + use wasmtime_wasi::bindings::cli::environment::Host;
  --> common/wasm-evaluators/src/super_pkg/mod.rs:6:1
   |
6  + use wasmtime_wasi::bindings::cli::exit::Host;
   |
     and 34 other candidates
help: if you import `Host`, refer to it directly
   |
99 -             U: __with_name1::types::Host + pkg::super_pkg::base_pkg::Host,
99 +             U: Host + pkg::super_pkg::base_pkg::Host,
   |

error[E0425]: cannot find function `add_to_linker` in module `__with_name1::types`
   --> /target/debug/build/wasmtime-component-macro-b0355b6157b884bd/out/super-pkg-world0.rs:101:34
    |
101 |             __with_name1::types::add_to_linker(linker, get)?;
    |                                  ^^^^^^^^^^^^^ not found in `__with_name1::types`
    |
help: consider importing one of these items
   --> common/wasm-evaluators/src/super_pkg/mod.rs:6:1
    |
6   + use crate::super_pkg::pkg::super_pkg::base_pkg::add_to_linker;
   --> |common/wasm-evaluators/src/super_pkg/mod.rs:6:1
    |
6   + use crate::base_pkg::pkg::base_pkg::db::add_to_linker;
   --> |common/wasm-evaluators/src/super_pkg/mod.rs:6:1
    |
6   + use wasmtime_wasi::bindings::cli::environment::add_to_linker;
   --> |common/wasm-evaluators/src/super_pkg/mod.rs:6:1
    |
6   + use wasmtime_wasi::bindings::cli::exit::add_to_linker;
    |
      and 36 other candidates
help: if you import `add_to_linker`, refer to it directly
    |
101 -             __with_name1::types::add_to_linker(linker, get)?;
101 +             add_to_linker(linker, get)?;
    |

Some errors have detailed explanations: E0405, E0425.
For more information about an error, try `rustc --explain E0405`.
error: could not compile `common-wasm-evaluators` (lib) due to 2 previous errors

With the bindgen! invocation:

use wasmtime::component::bindgen;

bindgen!({
    with: {
        "pkg:base-pkg": base_pkg,
    },
    trappable_imports: true,
});

pub use crate::base_pkg_module::exports::pkg::base_pkg;

Are there any additional configuration options that I am expected to pass to the bindgen! macro to get the use to work right?

view this post on Zulip Samuel Mwangi (Jul 24 2024 at 22:48):

@Alex Crichton The examples provided here on the use of with:

with: {
        "wasi:random/random": wasmtime_wasi::bindings::random::random,
        "wasi:cli": wasmtime_wasi::bindings::cli,
        "wasi": wasmtime_wasi::bindings,
        "wasi:http/types@0.2.0": wasmtime_wasi_http::bindings::http::types,
        "wasi:http@0.2.0": wasmtime_wasi_http::bindings::http,
        "wasi:filesystem/types/descriptor": MyDescriptorType,
    },

seem to be referencing the bindings module rather than the exports module. Given that no bindings module is generated for pkg:base-pkg because it only exports the types, I have updated my bindgen! invocation to:

bindgen!({
    path: "wit",
    with: {
        "pkg:super-pkg": pkg::super_pkg,
    },
    trappable_imports: true,
});

so that it now references the bindings generated for pkg:super-pkg. I am still getting an error albeit somewhat different:

error[E0432]: unresolved import `super::super::super::__with_name1::types::*`
   --> base-pkg/target/debug/build/wasmtime-component-macro-b0355b6157b884bd/out/super-pkg-world0.rs:239:21
    |
239 |             pub use super::super::super::__with_name1::types::*;
    |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cannot glob-import a module into itself

error[E0405]: cannot find trait `Host` in module `__with_name1::types`
  --> base-pkg/target/debug/build/wasmtime-component-macro-b0355b6157b884bd/out/super-pkg-world0.rs:99:37
   |
99 |             U: __with_name1::types::Host + cloudcad::super_pkg::base_pkg::Host,
   |                                     ^^^^ not found in `__with_name1::types`
   |
help: consider importing one of these items
  --> common/wasm-evaluators/src/super_pkg/mod.rs:6:1
   |
6  + use crate::super_pkg::cloudcad::super_pkg::base_pkg::Host;
  --> common/wasm-evaluators/src/super_pkg/mod.rs:6:1
   |
6  + use crate::base_pkg::cloudcad::base_pkg::db::Host;
  --> common/wasm-evaluators/src/super_pkg/mod.rs:6:1
   |
6  + use wasmtime_wasi::bindings::cli::environment::Host;
  --> common/wasm-evaluators/src/super_pkg/mod.rs:6:1
   |
6  + use wasmtime_wasi::bindings::cli::exit::Host;
   |
     and 34 other candidates
help: if you import `Host`, refer to it directly
   |
99 -             U: __with_name1::types::Host + cloudcad::super_pkg::base_pkg::Host,
99 +             U: Host + cloudcad::super_pkg::base_pkg::Host,
   |

error[E0425]: cannot find function `add_to_linker` in module `__with_name1::types`
   --> base-pkg/target/debug/build/wasmtime-component-macro-b0355b6157b884bd/out/super-pkg-world0.rs:101:34
    |
101 |             __with_name1::types::add_to_linker(linker, get)?;
    |                                  ^^^^^^^^^^^^^ not found in `__with_name1::types`
    |
help: consider importing one of these items
   --> common/wasm-evaluators/src/super_pkg/mod.rs:6:1
    |
6   + use crate::super_pkg::cloudcad::super_pkg::base_pkg::add_to_linker;
   --> |common/wasm-evaluators/src/super_pkg/mod.rs:6:1
    |
6   + use crate::base_pkg::cloudcad::base_pkg::db::add_to_linker;
   --> |common/wasm-evaluators/src/super_pkg/mod.rs:6:1
    |
6   + use wasmtime_wasi::bindings::cli::environment::add_to_linker;
   --> |common/wasm-evaluators/src/super_pkg/mod.rs:6:1
    |
6   + use wasmtime_wasi::bindings::cli::exit::add_to_linker;
    |
      and 36 other candidates
help: if you import `add_to_linker`, refer to it directly
    |
101 -             __with_name1::types::add_to_linker(linker, get)?;
101 +             add_to_linker(linker, get)?;
    |

error[E0412]: cannot find type `RelationshipTargetsRequest` in module `super::super::super::__with_name1::types`
   --> base-pkg/target/debug/build/wasmtime-component-macro-b0355b6157b884bd/out/super-pkg-world0.rs:122:59
    |
122 |                 super::super::super::__with_name1::types::RelationshipTargetsRequest;
    |                                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^ not found in `super::super::super::__with_name1::types`
    |
help: consider importing one of these items
    |
118 +             use crate::base_pkg::exports::cloudcad::base_pkg::description::RelationshipTargetsRequest;
    |
118 +             use crate::base_pkg::exports::cloudcad::base_pkg::types::RelationshipTargetsRequest;
    |
help: if you import `RelationshipTargetsRequest`, refer to it directly
    |
122 -                 super::super::super::__with_name1::types::RelationshipTargetsRequest;
122 +                 RelationshipTargetsRequest;
    |

error[E0412]: cannot find type `RelationshipType` in module `super::super::super::super::__with_name1::types`
   --> base-pkg/target/debug/build/wasmtime-component-macro-b0355b6157b884bd/out/super-pkg-world0.rs:253:70
    |
253 |                     super::super::super::super::__with_name1::types::RelationshipType;
    |                                                                      ^^^^^^^^^^^^^^^^ not found in `super::super::super::super::__with_name1::types`
    |
help: consider importing this enum
    |
249 +                 use crate::base_pkg::exports::cloudcad::base_pkg::types::RelationshipType;
    |
help: if you import `RelationshipType`, refer to it directly
    |
253 -                     super::super::super::super::__with_name1::types::RelationshipType;
253 +                     RelationshipType;
    |

Some errors have detailed explanations: E0405, E0412, E0425, E0432.
For more information about an error, try `rustc --explain E0405`.
error: could not compile `common-wasm-evaluators` (lib) due to 5 previous errors
warning: build failed, waiting for other jobs to finish...

What am I missing?

view this post on Zulip Victor Adossi (Jul 25 2024 at 06:25):

Hey @Samuel Mwangi I'd like to look at this -- can you upload your test case as a repository? I'd like to clone it and try locally

view this post on Zulip Samuel Mwangi (Jul 25 2024 at 11:31):

Hi, @Victor Adossi.

Thank you for your response. I have provided a test case in this repository. I have added just enough code to reproduce the issue.

Let me know if you need additional information.

Contribute to cryarchy/bindgen-deps-test development by creating an account on GitHub.

view this post on Zulip Alex Crichton (Jul 25 2024 at 14:15):

That error means that your with configuration isn't quite right, since it's projecting from the wrong path. That may be an extra path parameter needed or something else

view this post on Zulip Samuel Mwangi (Jul 25 2024 at 15:11):

Thank you, @Alex Crichton.

I have not figured out what issues exist in the with configuration. It aligns with the configuration information in the macro's documentation.

The package in the  provided test has only one world hence I have not specified the world to generate bindings for.

Contribute to cryarchy/bindgen-deps-test development by creating an account on GitHub.

view this post on Zulip Alex Crichton (Jul 25 2024 at 15:25):

This is a bit of a subtle problem, but the base issue is that there's a mismatch in the WIT and your with is invalid. In the generation of types for wit/deps/base-pkg all of the interfaces are exports, not imports. In the generation of the types for wit the types are imports, not exports. That means that the bindings don't actually line up.

To use with here the WIT structures need to be the same on the generation side and the consuming side.

view this post on Zulip Victor Adossi (Jul 25 2024 at 15:26):

Hey @Samuel Mwangi looking at your code -- it looks like you're trying to feed generated bindings through to another host implementation -- I'm not sure what you're trying to do.

Are you trying to call a host binding from another host binding?

base-pkg contains a ping function that it exports... Who is the intended caller? The host?
If you export the description interface, you get generated code that enables a call on a component (call_ping(store: S, arg0: &PingRequest) -- i.e. you can use a store.

super-pkg imports the base-pkg interface (completely unrelated to the package) which contains a resource, and exports ping-processor... Who is meant to call the ping-processor's process-ping?

Could you add your main.rs, and a a component binary that you're going to use to actually do the calls?

A sequence diagram/listing might be helpful:

  1. (Host|WebAssembly Component|???) calls x
  2. Host binding does y
  3. ???

Based on what I can tell, it seems that you need to import base-pkg/interface so that you can actually implement base-pkg-proxy in terms of calling the other ping underneath... But again, "calling the other ping" means *doing something with a store) -- which is something only the host can do.

That said, I did fork the repo and try to get it to build -- https://github.com/cryarchy/bindgen-deps-test/pull/1

Also, are you using cargo expand? https://crates.io/crates/cargo-expand

Not intending to merge this, but just as a good place for experimentation and discussion!

view this post on Zulip Samuel Mwangi (Jul 25 2024 at 20:09):

Hi, @Victor Adossi,

I have provided a **working** implementation that illustrates what I am trying to do.

To get it working, I had to manually implement the From<T> trait for similar types defined in wit/types.wit and wit/deps/base-pkg/types.wit.

The issue is how can I avoid defining similar types in the super-pkg and base-pkg WIT directories? If I could correctly configure the bindgen! macro's with property, I could avoid having to:

Contribute to cryarchy/bindgen-deps-test development by creating an account on GitHub.
Contribute to cryarchy/bindgen-deps-test development by creating an account on GitHub.
Contribute to cryarchy/bindgen-deps-test development by creating an account on GitHub.
Contribute to cryarchy/bindgen-deps-test development by creating an account on GitHub.

view this post on Zulip Samuel Mwangi (Jul 25 2024 at 20:15):

@Alex Crichton Aah! I see. That explains the cause of the error. That implies that I need to define the types on the host and import that into both the base-pkg and super-pkg. Let me give it a shot.

view this post on Zulip Samuel Mwangi (Jul 25 2024 at 21:49):

No luck. For now, I will implement the From<T> trait for the types. I suspect the wasi:* components are doing something similar to my goal. I did not manage to figure it out though. Shelving it off for now. Thank you very much, guys.

view this post on Zulip Victor Adossi (Jul 26 2024 at 03:01):

Hey @Samuel Mwangi thanks for the update here, glad you figured out how to get it working! The updated README is a ton clearer!

Just one last thing, I'm not sure if anyone has pointed it out, but... Have you seen WASI Virt?

https://github.com/bytecodealliance/WASI-Virt

Virtual implementations of WASI APIs. Contribute to bytecodealliance/WASI-Virt development by creating an account on GitHub.

view this post on Zulip Notification Bot (Jul 30 2024 at 14:40):

Victor Adossi has marked this topic as resolved.


Last updated: Nov 22 2024 at 17:03 UTC