Stream: general

Topic: Returning dynamic WASM component resource to guest from host

SeanOMik (Nov 11 2024 at 22:13):

@Robin Brown we talked on fedi a bit about a "dynamically-typed wasm component resource."

I'm working on implementing WASM scripting in my game engine. The game engine has an ECS, which can store ECS Resources. Often times, these ECS resources can be kind of big, so I wouldn't want to marshal them to the guest. Instead I would want to create a WASM component resource. The issue I'm running into is that in the get-resource WIT function on the World interface, I must return a typed resource. Rust represents a resource with Resource<T>, so I must know the type when writing the function implementation.

You mentioned that I could offer

(A) way to fallibly "downcast" a resource to a target type or to turn it into a variant of the possible concrete types either as a method or standalone function

I'm interested in how you could represent that in WIT. I'm experienced in Rust, but still relatively new to WASM and Wit. Thanks for the help!!

Robin Brown (Nov 11 2024 at 22:29):

Here's how what I was talking about maps to WIT roughly

Version with "downcast" (it's just a function)

interface ecs {
   resource ecs-resource {
      ...
      // have a method for each one if there's a fixed number of these known ahead of time
      to-camera: func() -> result<camera>;
   }

  resource camera { ... }
}

// or a separate function for each one if they need to be added by additional interfaces
interface camera {
   use ecs.{ecs-resource};

   to-camera: func(r: ecs-resource) -> result<camera>;
}

Version where you convert to a variant of concrete types

interface ecs {
   resource ecs-resource {
      ...
      // have a method that turns your resource into the corresponding concrete type
      to-concrete: func() -> concrete;
   }

  resource camera { ... }

   variant concrete {
      camera(camera),
      ...
   }
}

Robin Brown (Nov 11 2024 at 22:31):

very rushed pseudocode wit. happy to elaborate later. gtg though!

Robin Brown (Nov 11 2024 at 22:36):

also, I think it'd be interesting to zoom out and look more at how you want the system you're building to work and what the code your users write will look at sometime!

SeanOMik (Nov 11 2024 at 23:03):

Ohh, that makes sense. I actually do similar things like that with Asset handles, not sure why I didn't think about doing it for this.

Robin Brown said:

also, I think it'd be interesting to zoom out and look more at how you want the system you're building to work and what the code your users write will look at sometime!

Yeah, I'm not sure how else I would do something like this with WIT though. I do intend that the user would create a higher level api over the low level api provided through the WIT files. Right now I feel like I'm working around WIT's restrictions tbh. I felt similarly when implementing Lua though tbf.

It would be great to have a wit function that directly translates to World:get_resource<T>(), but I have to expose a lower level api, then the user can create higher level wrappers that expose a similar api to what the host would have. That's actually what I'm doing with a C# package.

Robin Brown (Nov 12 2024 at 15:59):

The issue is that get-resource<T>: func() -> T isn't a concrete type. It's a family of functions (or a "type constructor") and two components could only soundly compose if they provide and use the same concrete functions. Otherwise, what happens when I call get-resource<U> for some U the exporting interface hasn't implemented?

On the other hand, it makes a lot of sense to be able to want to write WIT generically over some types and let a given implementation fill in those types to choose its concrete type. However, this is no longer a concrete interface or world. We've taken to calling these higher-level constructs "WIT templates" and the parameters "holes" iirc.

There has been some discussion and experiments around what this might look like but I believe it will be quite some time (on the scale of years) before they're high enough priority compared to other work (like preview 3 async) to get fully implemented since they're very complex.

SeanOMik (Nov 12 2024 at 22:45):

Yeah, "templates" or "generics" would be perfect for this. But like you said, it will take some time until that is implemented. I think the only option currently I have is to return an error, or none if U doesn't implement that interface. If you have any other ideas for ways I could implement this in current WIT spec, I'm open ears!

Joel Dice (Nov 12 2024 at 23:09):

My colleague Brian Hardock and I started to implement WIT Templates last year, but shelved it because it was becoming a distraction from more important things. See https://github.com/bytecodealliance/wasm-tools/pull/964 and https://github.com/bytecodealliance/wit-bindgen/pull/541 (both of which are hopelessly out of date by now).

I'd love to pick that back up someday, but as Robin said, it's still not a high priority.

Last updated: Nov 22 2024 at 16:03 UTC