Improving Rust resource codegen · wit-bindgen

The codegen for resources in Rust isn't really ideal and feels far from idiomatic. I was wondering if anyone has done any thinking of how we might improve this.

I've recently had to interact with the resource heavy wasi-http and wrote some wrappers to provide more ergonomic bindings. Perhaps this could be an inspiration for where we might go?

struct Fields<'a> {
    guest: bindings::exports::wasi::http::types::GuestFields<'a>,
    resource: wasmtime::component::ResourceAny,
}

impl<'a> Fields<'a> {
    pub fn new<T>(
        instance: &'a VirtualizedApp,
        store: &mut wasmtime::Store<T>,
    ) -> anyhow::Result<Self> {
        let guest = instance.wasi_http_types().fields();
        let resource = guest.call_constructor(store)?;
        Ok(Self { guest, resource })
    }

    pub fn append<T>(
        &self,
        store: &mut wasmtime::Store<T>,
        name: &String,
        value: &Vec<u8>,
    ) -> anyhow::Result<Result<(), HeaderError>> {
        self.guest.call_append(store, self.resource, name, value)
    }

    fn entries(
        &self,
        store: &mut wasmtime::Store<super::StoreData>,
    ) -> wasmtime::Result<Vec<(String, Vec<u8>)>> {
        self.guest.call_entries(store, self.resource)
    }
}

The biggest questions, and the reason why bindings aren't yet ergonomic, is that you typically need to keep track of four things when using a particular resource:

Guests tend to be paramterized on some lifetime, so one could imagine parameterizing a resource type over borrows to an Instance, Store, and Guest but this obviously limits the user to only using the resource for the lifetime of that borrow.

My wrappers typically do this for everything but the Store which is instead passed in for every method call:

struct OutgoingBody<'a> {
    instance: &'a VirtualizedApp,
    guest: bindings::exports::wasi::http::types::GuestOutgoingBody<'a>,
    resource: wasmtime::component::ResourceAny,
}

impl<'a> OutgoingBody<'a> {
    fn write<T>(&self, store: &mut wasmtime::Store<T>) -> anyhow::Result<Result<OutputStream, ()>> {
        let stream = match self.guest.call_write(store, self.resource)? {
            Ok(s) => s,
            Err(()) => return Ok(Err(())),
        };
        Ok(Ok(OutputStream {
            instance: self.instance,
            resource: stream,
        }))
    }
}

Would appreciate any thoughts. Depending on the conversation here, I'd be happy to write up an issue and if we can come to an agreement on what the improvement should look like, perhaps even do the implementation.

Lann Martin (Jun 14 2024 at 15:33):

Just to clarify, you're talking about wasmtime's bindgen, not wit-bindgen itself, right?

Ryan Levick (rylev) (Jun 14 2024 at 15:33):

Alex Crichton (Jun 14 2024 at 15:38):

The initial implementation of resources was where I basically wanted everything to work as opposed to everything working well. We were under a fair amount of pressure to get everything working as opposed to everything working well, hence the current status where the bindings basically give up and don't do anything fancy. That being said I think it'd be great to improve bindings. The trickiness is always in handling the free-form nature of WIT where bindings need to be uniformly handled across all possible WIT interfaces, not just a few specific shapes. Even wasi:http doesn't exercise all the various possibilities of where a resource could show up.

Even with that though there's no reason we can't have improvements still which don't necessarily apply to all shapes of APIs. I don't really know how to balance this though because everyone seems to expect that bindgen! generates a "perfect" API which is the most ergonomic, but that's an extremely difficult task to take on for all possible WIT documents.

Joel Dice (Jun 14 2024 at 16:59):

My personal opinion is that the ergonomics of guest bindings matters more than the ergonomics of host bindings, based on the assumption that the latter will be used by a much smaller pool of developers who may need fine-grained control and deeper domain knowledge to match the "super host powers" they've been entrusted with, and it seems natural that the bindings should reflect that. It's analogous to user space programming vs. kernel programming. The latter is often more difficult and complicated, but for good reasons.

Joel Dice (Jun 14 2024 at 17:03):

(None of which is to say we shouldn't improve the host bindings where appropriate)

Ryan Levick (rylev) (Jun 17 2024 at 09:01):

All good points. I'm raising this issue because the bindings feel more complicated to me than they necessarily have to. My hand made wrappers around them improve usability quite a bit, and they feel quite mechanical. That all being said, I'm certainly not discounting the possibility that automating these wrappers is not actually possible and we've reached the point at which the generated code is as ergonomic as we can get it. It just doesn't feel that way to me yet. I'll spend some time thinking about this and seeing if I see any way for improvement.

Stream: wit-bindgen