Stream: wit-bindgen

Topic: Relation between imported and exported types

Slava Kuzmich (Jan 12 2024 at 12:20):

Greetings! I’m learning about WIT and trying to implement a guest binding generator for Kotlin. Could somebody help me figure out how non-builtin types that come from imported and exported interfaces should relate to each other in the generated source code?

For example, in tests/codegen/records.wit, there is an interface containing records, and it is both imported and exported from the world. Rust and Java guest bindings generate separate types (structs and classes) for imported and exported records. I’m assuming that users of the bindings would need to convert between “imported” and “exported” record types, even though they came from the same WIT interface.

It seems that types that are deeply composed of primitives don’t care if they are copyed from the host or created inside the component, and can be merged. But I can’t yet wrap my head around adding resource types to the mix. Resource types sometimes are created externally (for example, using the imported constructor method). Sometimes they need to be created internally when the constructor function is exported. Does anybody have thoughts on unifying resource types in bindings for statically typed languages?

Or should imported and exported WIT interfaces be treated as completely separate things/namespaces in bindings, requiring users to convert between different flavors of the same WIT type manually?

Lann Martin (Jan 12 2024 at 15:49):

Value types (i.e. all the WIT types except resources) are structurally typed, so binding generators can do whatever is appropriate for the target language.

Lann Martin (Jan 12 2024 at 15:50):

Resource types are always unique, even between two instances of the same component

Alex Crichton (Jan 12 2024 at 15:54):

Aha good questions! So from a mental-model perspective, structurally equivalent types are the same regardless whether they're exported or imported, except for resources. So yes if you import and export the same record it should be possible to use a single type in the guest language to represent that type. There's some minor trickiness where the memory management is often different, for example a string passed to an exported function is passed as an owned buffer of memory but a string passed to an import is just a ptr/len. For now though current bindings generators generate separate types for simplicity to not break resources but could also have an optimization to "unify" types where possible.

For resources as Lann mentioned they're always different types. So if the same WIT type is both exported and imported that actually generates two unique types for the component, one for the import and one for the export. That means that structs containing those resources would be distinct for example since their primitive components would be different.

So to answer your questions::

Does anybody have thoughts on unifying resource types in bindings for statically typed languages?

Currently this can't be done due to the nature of the component model where an imported/exported resource creates distinct types. This is reflected into languages with different static capabilities, patterns, representations, etc.

Or should imported and exported WIT interfaces be treated as completely separate things/namespaces in bindings, requiring users to convert between different flavors of the same WIT type manually?

Yes and no. Resources need to be distinct, and types built from those resources. Types that don't have distinct components, though, like records-of-strings, can be the same across both imports and exports.

Also FWIW both importing and exporting an interface is a bit of an esoteric thing to do in practice at the moment, so it's ok if a bindings generator doesn't fully handle it

Alex Crichton (Jan 12 2024 at 15:54):

also on that last bit that's why current bindings generators don't go out of their way to generate "nice" bindings right now because it's a chunk of work and doesn't seem too beneficial for anyone at this time

Joel Dice (Jan 12 2024 at 15:56):

Imported and exported versions of a resource must be separate types, despite the fact that they share a single TypeId in wit-parser's Resolve, meaning you have to keep track of whether you're generating code for imports or exports in order to determine which version of the resource is relevant. See the Rust, C, or Go binding generators for examples.

Joel Dice (Jan 12 2024 at 16:00):

Or, as a temporary measure, you could just detect if a resource is both imported and exported and raise an error in that case, which is what an early version of the C generator did

Slava Kuzmich (Jan 12 2024 at 16:50):

Thanks everyone! Am I understanding it correctly that there is no way to express a "passthrough" of a resource between imports and exports without wrapping it with another one?

Given this, it seems a bit problematic to have a default automatic "unification" of generated nominal types, considering that a small change of adding an optional resource handle type in a WIT package could cause a lot of generated types to split.

Lann Martin (Jan 12 2024 at 17:08):

Am I understanding it correctly that there is no way to express a "passthrough" of a resource between imports and exports without wrapping it with another one?

Could you give an example of what you mean?

Joel Dice (Jan 12 2024 at 17:27):

An exported function can refer to imported resource types, so that's no problem. You just can't import and export the same resource and try to unify them -- they're two different types implemented by two different components (or the host) and only share an interface -- the implementations could be completely unrelated.

Slava Kuzmich (Jan 12 2024 at 20:33):

Yes, I meant is referring to imported resource types in exported functions. What would it look like in WIT?
How to determine whether resource type is imported or exported (or both?) ?

Lann Martin (Jan 12 2024 at 20:37):

The world you are generating for will contain some set of exported and/or imported interfaces. If a resource is defined in a directly exported interface then that resource type is owned by the target guest. If a resource is defined in an imported interface (or an interface useed by an exported interface) then the target guest will be borrowing a different resource (handle).

Joel Dice (Jan 12 2024 at 20:41):

For example:

package foo:foo;

interface x {
  resource r;
}

interface y {
  use x.{r};

  foo: func(r: r);
}

world w {
  export y;
}

Joel Dice (Jan 12 2024 at 20:41):

As long as w doesn't try to export x, the r used in y will be imported by the component.

Joel Dice (Jan 12 2024 at 20:42):

@Alex Crichton and I have discussed making this more explicit in a future version of WIT, and possibly allowing you to refer to both an imported and exported versions of a resource in the same interface.

Lann Martin (Jan 12 2024 at 20:43):

It is a bit confusing since the import of x is implicit from useing it in exported interface y (right? :dizzy:)

Joel Dice (Jan 12 2024 at 20:43):

Yeah, I needed many conversations with Alex before I understood it :)

Joel Dice (Jan 12 2024 at 20:43):

Which is why I'd love to make it more explicit

Lann Martin (Jan 12 2024 at 20:46):

Your example (on its own) wouldn't be a useful world right? There would be no way to get an instance of r? nvm, the caller of foo could import something that exports x or even export it itself

Slava Kuzmich (Jan 13 2024 at 09:15):

It is a bit confusing, happy hear it was discussed :)

Joel Dice said:

As long as w doesn't try to export x, the r used in y will be imported by the component.

How would w exporting x should change the exportness of r used in exported y? I tried testing this with Rust, but got this error:

pub mod exports {
    pub mod foo {
        pub mod foo {
            #[allow(clippy::all)]
            pub mod x {
                pub use super::super::super::super::ERROR as R;

Slava Kuzmich (Jan 13 2024 at 09:22):

Also there is a syntax for resources declared directly in worlds world w { resource r; }. Experimenting with it, I see that it imported. I would be happy if someone confirm its semantics here, WIT spec document is very light on explaining this one.

Ralph (Jan 15 2024 at 17:05):

Joel Dice said:

Yeah, I needed many conversations with Alex before I understood it :)

someone's going to make a mint just writing that bit up.....

Joel Dice (Jan 16 2024 at 15:57):

Slava Kuzmich said:

It is a bit confusing, happy hear it was discussed :)

Joel Dice said:

As long as w doesn't try to export x, the r used in y will be imported by the component.

How would w exporting x should change the exportness of r used in exported y? I tried testing this with Rust, but got this error:

pub mod exports {
    pub mod foo {
        pub mod foo {
            #[allow(clippy::all)]
            pub mod x {
                pub use super::super::super::super::ERROR as R;

Here's what I did:

// foo.wit
package foo:foo;

interface x {
  resource r;
}

interface y {
  use x.{r};

  foo: func(r: r);
}

world w {
  export x; // Note that I've added this, which makes the `use x.{r};` in `y` refer to the exported version of `r`
  export y;
}

Then I generated the bindings using wit-bindgen rust --stubs foo.wit. If you look at the output of that command and compare it to what you get if you omit the export x; above, you'll see that the one with the export x; uses an exported version of r, while the other one uses an imported version of r.

Till Schneidereit (Jan 19 2024 at 12:09):

Ralph said:

someone's going to make a mint just writing that bit up.....

Where does that mint come from, I wonder?

Ralph (Jan 19 2024 at 12:14):

damn, now you want a business model for sharing knowledge.... I hate that.

Dan Gohman (Jan 19 2024 at 12:59):

business model: share knowledge and collect the commissions they pay you when they train AI on your content

Ralph (Jan 19 2024 at 14:09):

Dan is not helping

Ralph (Jan 19 2024 at 14:09):

but he is feeling perky

Last updated: Nov 22 2024 at 16:03 UTC