Stream: wit-bindgen

Topic: ✔ Guest resource gets duplicated


view this post on Zulip Karel Hrkal (kajacx) (May 03 2024 at 12:52):

Since my last post was pretty toxic, let me start over.

I am having trouble using resources. I have made a minimal reproducible example here, but for the sake of convenience, I'll add the important files here:

protocol.wit

Guest code

Host code

The main problem is that there seem to be two "Employee" resources both on guest and on host. On the guest, there is example::protocol::employees::Employee and exports::example::protocol::employees::Employee. The exported roundtrip method (the one we need to implement on the guest) uses the second type, while the imported roundtrip method (the one that is provided to us by the host) uses the first one.

I think this is happening because I use the employee resource type in the host_fns imported interface, but I don't know how to work with these 2 different types or if it is possible to convert one into the other.

On the host, there are also two "employee" resources: there is exports::example::protocol::employees::GuestEmployee, obtained with my_world.example_protocol_employees().employee(). This seems to be the "correct" guest resource, as using it works without running into the todo!() panics in the host Employee implementaion (see later).

And then there is example::protocol::employees::HostEmployee. It looks like wit bindgen thinks there is a employee resource that the host should implement, but that's not what I wanted. Maybe I have some problem in the wit file?

Finally, In the employee_roundtrip imported method on the host, I don't know how to work with that resource. How do I get the name, for example? It seems like it is the host employee "fake" resource, but Employee should be a guest resource.

The wit format page doesn't talk about the difference between host and guest resource, nor how to use them. This example by Christof Petig intentially uses the same resource as both host and a guest resource, and it doesn't have any methods that use either the host or the guest resource.

So in the end I am lost and don't know how to proceed. Is there a way I can change the wit file for example, so that I still have a employee guest resource (method implementation provided by the guest), a company host resource (method implementation provided by the host) and both roundtrip methods for both resources both on guest and host (with the guest roundtrip method calling the host one)?

Various projects related to WASM, each in it's own branch, so that I don't have 50 repositories. - GitHub - kajacx/wasm-playground at resources-roundtrip
Repository for design and specification of the Component Model - WebAssembly/component-model
A demo showing WASM component model resources in various environments - cpetig/resource-demo

view this post on Zulip Lann Martin (May 03 2024 at 13:18):

I think the problem may be the duplication of host-fns and guest-fns. A single interface is meant to support both the import and export sides depending on the world being targeted. If the world exports an interface, the guest will be expected to implement that interface. If the world imports an interface, an implementation will be provided to the guest. Wasmtime's host bindings generation is special in that it inverts this import/export relationship, but it still uses the same single WIT interface as the guest.

view this post on Zulip Joel Dice (May 03 2024 at 13:19):

The key thing to notice here is that the resources world both imports and exports the employees interface. It imports it implicitly by importing host-fns which uses employees.{employee} (which causes employees to be imported as well), and it exports it explicitly.

The result is that the component refers to two entirely different resource types which happen to share the same name and interface: the imported version of employee and the exported version of employee. Those two types are completely independent of each other as far as the component model is concerned, and the bindings generated by the host and guest bindings generators reflect that. Any conversions you might want to do between the two types would need to be done manually.

From what you wrote, I believe your intention was to only have a single type: the exported version of employee, and refer to it via both imported and exported interfaces. That is unfortunately not possible; see https://github.com/WebAssembly/component-model/issues/272 for details. You are allowed to reference an imported resource type from either imports or exports, but you can only reference an exported resource type from other exports.

The Component Model does not currently allow you to refer to exported types in imports, which feels asymmetric given that one can easily refer to imported types in exports. This becomes a practical...

view this post on Zulip Karel Hrkal (kajacx) (May 03 2024 at 13:31):

So the host can never use an exported Resource in any of it's (imported) functions?

view this post on Zulip Joel Dice (May 03 2024 at 13:37):

Not directly, no. You could work around the issue by maintaining a table of objects in the guest and referring to them by e.g. u32 indexes in any imports, e.g.:

interface ersatz-employees {
  type employee = u32;
  new-employee: func(name: string, min-salary: u32);
  employee-get-name: func(employee: employee) -> string;
  employee-get-min-salary: func(employee: employee) -> u32;
}

view this post on Zulip Joel Dice (May 03 2024 at 13:39):

Then:

interface host-fns {
  use ersatz-employees.{employee};
  use companies.{company};

  employee-roundtrip: func(employee: employee) -> employee;
  company-roundtrip: func(company: company) -> company;
}

view this post on Zulip Joel Dice (May 03 2024 at 13:40):

Since, in that scenario, employee is just a u32, there's no circularity, and thus only a single representation of the type that can be used in both imports and exports.

view this post on Zulip Joel Dice (May 03 2024 at 13:45):

It's not an ideal solution, obviously. I'd be interested if anyone has ideas for improving on it, given that I expect this will be a very common paper cut for folks.

view this post on Zulip Karel Hrkal (kajacx) (May 03 2024 at 13:47):

I guess that could work. The host still cannot get the name of the resource in the imported function, because it would be re-entering the component?

Now when I think about it, it makes sense that the host cannot "handle" guest resources in imported functions, because any work with them would involve re-entering the component, which is a fundamental problem that cannot be fixed with a better host implementation.

You could (in theory) handle opaque handles to guest resources in imported host functions (which is effectively what the "u32" workaround does), but I guess re-implementing large portions of the host runtime isn't worth the effort.

view this post on Zulip Ralph (May 03 2024 at 14:16):

My understanding is that this is currently the case, but that the next preview may or may not address the "one shared type" scenario. @Joel Dice is that your understanding as well?

view this post on Zulip Joel Dice (May 03 2024 at 14:20):

I'm not aware of any plans to allow referring to exported resource types in imported interfaces for the next preview or ever, nor any any plans to make the pattern I described above more type-safe or ergonomic. Maybe it was on the list @Luke Wagner presented at the plumbers conference and I missed it?

view this post on Zulip Karel Hrkal (kajacx) (May 03 2024 at 14:30):

Now when I think about it some more, I'm not sure if re-entering a component (calling an exported function from an imported function) is actually bad. Even when you have a mutable reference going in the guest while you call the exported function, it shouldn't be possible to read that data from the re-entrant code because of the type system. Or maybe there are other problems with re-entering that I don't see?

view this post on Zulip Joel Dice (May 03 2024 at 14:32):

AFAIK, calling an exported function from an imported function is not prohibited. My understanding is that the potential circularity we're concerned with here is at the static, type system level rather than the runtime level.

view this post on Zulip Joel Dice (May 03 2024 at 14:35):

oh, maybe they are prohibited: https://github.com/WebAssembly/component-model/blob/main/design/mvp/Explainer.md#component-invariants

Repository for design and specification of the Component Model - WebAssembly/component-model

view this post on Zulip Karel Hrkal (kajacx) (May 03 2024 at 14:35):

In that case, using guest resources in host imported function should (in theory) be fine. The host could just call the resource's methods, re-entering the guest component.

I will admit I did not really understand what the github issue was about, it looked like it concerned connecting many components together, but that is not my use case, I just want one component with exported and imported functions.

EDIT: never mind, re-entering is prohibited, as explained in the link above.

view this post on Zulip Joel Dice (May 03 2024 at 14:37):

Any interface defined in the component model needs to be valid both for interacting with the host and with other components, hence some of the restrictions.

view this post on Zulip Notification Bot (May 03 2024 at 14:38):

Karel Hrkal (kajacx) has marked this topic as resolved.

view this post on Zulip Joel Dice (May 03 2024 at 14:43):

Note that the rules about reentrance are expected to be loosened with the introduction of async lifts and lowers (which is what I'm currently working on). I'm not sure if that will directly help with your use case or not yet, but possibly.


Last updated: Dec 23 2024 at 12:05 UTC