How to work with resources on the host? · wit-bindgen

Ok, I have been looking into resources in wit, and while the concept is very interesting and works well on the guest, it's a confusing mess on the host. Let's take this wit file:

Wit protocol

package example:protocol;

interface companies {
  resource company {
    constructor(name: string, max-salary: u32);
    get-name: func() -> string;
    get-max-salary: func() -> u32;
  }
}

interface employees {
  resource employee {
    constructor(name: string, min-salary: u32);
    get-name: func() -> string;
    get-min-salary: func() -> u32;
  }
}

world my-world {
  import companies;
  use companies.{company};

  export employees;
  use employees.{employee};

  export find-job: func(employee: employee, companies: list<company>) -> option<company>;
  export find-employee: func(company: company, employees: list<employee>) -> option<employee>;

  export company-roundtrip-export: func(company: company) -> company;
  import company-roundtrip-import: func(company: company) -> company;

  export employee-roundtrip-export: func(employee: employee) -> employee;
  import employee-roundtrip-import: func(employee: employee) -> employee;
}

It's pretty straight forward - company is a host resource (method impl defined by host), while employee is a guest resource (method impl defined by guess). And the guess code is very nice and intuitive:

Guest file

wit_bindgen::generate!({
    path: "../protocol.wit",
    world: "my-world",
});

struct MyEmployees;

impl exports::example::protocol::employees::Guest for MyEmployees {
    type Employee = MyEmployee;
}

pub struct MyEmployee {
    name: String,
    min_salary: u32,
}

impl exports::example::protocol::employees::GuestEmployee for MyEmployee {
    fn new(name: String, min_salary: u32) -> Self {
        Self { name, min_salary }
    }

    fn get_name(&self) -> String {
        self.name.clone()
    }

    fn get_min_salary(&self) -> u32 {
        self.min_salary
    }
}

impl Guest for MyEmployees {
    fn find_job(employee: Employee, companies: Vec<Company>) -> Option<Company> {
        companies
            .into_iter()
            .find(|company| employee.get_min_salary() <= company.get_max_salary())
    }

    fn find_employee(company: Company, employees: Vec<Employee>) -> Option<Employee> {
        employees
            .into_iter()
            .find(|employee| employee.get_min_salary() <= company.get_max_salary())
    }

    fn company_roundtrip_export(company: Company) -> Company {
        let _name = company.get_name();
        company_roundtrip_import(company)
    }

    fn employee_roundtrip_export(employee: Employee) -> Employee {
        let _name = employee.get_name();
        employee_roundtrip_import(employee)
    }
}

export!(MyEmployees);

We define what the "employee" resource will be with an associative type, then implement the resource's methods on in, and in the exported functions, we can call the resource methods on both resources. Very nice and idiomatic.

Host file

use example::protocol::{
    companies,
    employees::{self, HostEmployee},
};
use std::collections::HashMap;
use wasmtime::{
    component::{Component, Linker, Resource, ResourceAny},
    Config, Engine, Result, Store,
};

wasmtime::component::bindgen!({
    path: "../protocol.wit",
    world: "my-world"
});

#[derive(Default, Clone)]
struct MyCompany {
    name: String,
    max_salary: u32,
}

#[derive(Default, Clone)]
struct ResHolder<T> {
    resources: HashMap<u32, T>,
    next_id: u32,
}

impl<T> ResHolder<T> {
    fn new(&mut self, item: T) -> u32 {
        let id = self.next_id;
        self.resources.insert(id, item);
        self.next_id += 1;
        id
    }

    fn get(&self, id: u32) -> Option<&T> {
        self.resources.get(&id)
    }

    fn drop(&mut self, id: u32) -> Result<(), ()> {
        self.resources.remove(&id).map(|_| ()).ok_or(())
    }
}

#[derive(Default, Clone)]
struct State {
    companies: ResHolder<MyCompany>,
}

impl companies::HostCompany for State {
    fn new(&mut self, name: String, max_salary: u32) -> Result<Resource<companies::Company>> {
        Ok(Resource::new_own(
            self.companies.new(MyCompany { name, max_salary }),
        ))
    }

    fn get_name(&mut self, self_: Resource<companies::Company>) -> Result<String> {
        Ok(self.companies.get(self_.rep()).unwrap().name.clone())
    }

    fn get_max_salary(&mut self, self_: Resource<companies::Company>) -> Result<u32> {
        Ok(self.companies.get(self_.rep()).unwrap().max_salary)
    }

    fn drop(&mut self, rep: Resource<companies::Company>) -> Result<()> {
        self.companies.drop(rep.rep()).unwrap();
        Ok(())
    }
}

impl companies::Host for State {}

impl HostEmployee for State {
    fn drop(
        &mut self,
        rep: wasmtime::component::Resource<employees::Employee>,
    ) -> wasmtime::Result<()> {
        Ok(())
    }

    fn get_min_salary(
        &mut self,
        self_: wasmtime::component::Resource<employees::Employee>,
    ) -> wasmtime::Result<u32> {
        Ok(50_000)
    }

    fn get_name(
        &mut self,
        self_: wasmtime::component::Resource<employees::Employee>,
    ) -> wasmtime::Result<String> {
        Ok("Anon".into())
    }

    fn new(
        &mut self,
        name: String,
        min_salary: u32,
    ) -> wasmtime::Result<wasmtime::component::Resource<employees::Employee>> {
        Ok(Resource::new_own(0))
    }
}

impl employees::Host for State {}

impl MyWorldImports for State {
    fn company_roundtrip_import(
        &mut self,
        company: wasmtime::component::Resource<Company>,
    ) -> wasmtime::Result<wasmtime::component::Resource<Company>> {
        let name = &self.companies.get(company.rep()).unwrap().name;
        println!("Company roundtrip: {name}");
        Ok(company)
    }

    fn employee_roundtrip_import(
        &mut self,
        employee: wasmtime::component::Resource<Employee>,
    ) -> wasmtime::Result<wasmtime::component::Resource<Employee>> {
        println!("Employee name: ???");
        Ok(employee)
    }
}

fn main() {
    println!("Starting...");

    let mut config = Config::new();
    config.wasm_component_model(true);

    let engine = Engine::new(&config).unwrap();
    let mut store = Store::new(&engine, State::default());

    let component_bytes =
        std::fs::read("../guest-rust/target/wasm32-unknown-unknown/release/guest_rust.wasm")
            .expect("component bytes");

    let component = Component::new(&store.engine(), &component_bytes).expect("create component");

    let mut linker = Linker::new(store.engine());

    MyWorld::add_to_linker(&mut linker, |state| state).unwrap();
    let (my_world, _instance) = MyWorld::instantiate(&mut store, &component, &linker).unwrap();

    let employees = my_world.example_protocol_employees().employee();
    let employee: ResourceAny = employees
        .call_constructor(&mut store, "Mike".into(), 50_000)
        .unwrap();
    println!(
        "Employee name: {}",
        employees.call_get_name(&mut store, employee).unwrap()
    );

    let company1 = Resource::new_own(store.data_mut().companies.new(MyCompany {
        name: "Company1".into(),
        max_salary: 30_000,
    }));

    let employee = Resource::new_own(0);

    let result = my_world
        // .example_protocol_employees()
        .call_find_job(&mut store, employee, &[company1])
        .unwrap();
    println!("Find first job: {result:?}");

    let company1 = Resource::new_own(store.data_mut().companies.new(MyCompany {
        name: "Company1".into(),
        max_salary: 30_000,
    }));
    let company2 = Resource::new_own(store.data_mut().companies.new(MyCompany {
        name: "Company2".into(),
        max_salary: 60_000,
    }));

    let employee = Resource::new_own(0);

    let result = my_world
        // .example_protocol_employees()
        .call_find_job(&mut store, employee, &[company1, company2])
        .unwrap();
    println!("Find second job: {result:?}");
    println!(
        "Second job name: {}",
        store
            .data()
            .companies
            .get(result.unwrap().rep())
            .unwrap()
            .name
    );
}

Now, there are many problems. First, I have to handle the companies myself in a custom ResHolder implementation. I would expect wasmtime to provide resource holding for host resources automatically, just like it provides resource holding for guest resources automatically.

Next, working with any resources on the host is beyond painful. You get a handle, and need to manually find the resource you want. Wasmtime should manage the resources automatically for you and you should implement methods that takes a proper resource object, just like it is on the quest.

But what is by far the most "what the fuck" thing is that there are actually two "employee" resources: one "host resource", and one "guest resource". How on earth does this happen??

Also having to store the guest resource as ResourceAny is just sad, no type safety for what kind of resource it is. Is there any plan to properly fix all of this in a future version?

Christof Petig (May 02 2024 at 12:53):

I perfectly understand why you are horrified seeing this manual process as described in the examples or docs, IIRC. I have been in a similar place when I wrote host code with resources.

But I feel this is or was hidden information, I mostly learned about it in wasmtime's source code.

Karel Hrkal (kajacx) (May 02 2024 at 13:12):

Well, the Resource table makes managing resources slightly less painful, but you still need to convert the handle into the resource type instead of getting the resource type directly.

Also, this doesn't address any of the other issues, but I 100% agree that there aren't really any good tutorials or examples that would explain how to work with resources.

Till Schneidereit (May 02 2024 at 13:14):

@Karel Hrkal (kajacx) without commenting on the substance of your feedback about the API: please stop using this kind of tone. People wrote this code, and they didn't write it to offend you—and it's not okay to be offensive towards them.

You used a question as the topic for this conversation, which is the right approach. Please stick to that approach in how you engage as well, instead of coming down extremely hard on other people's work

Karel Hrkal (kajacx) (May 02 2024 at 14:07):

@Till Schneidereit Hello and sorry if I was too negative. I honestly don't know if this is the intended way to use resources or not, if I'm missing something (like a function somewhere) I would appreciate some help.

Stream: wit-bindgen

Topic: How to work with resources on the host?

Karel Hrkal (kajacx) (May 02 2024 at 11:09):

Christof Petig (May 02 2024 at 12:53):

Karel Hrkal (kajacx) (May 02 2024 at 13:12):

Till Schneidereit (May 02 2024 at 13:14):

Karel Hrkal (kajacx) (May 02 2024 at 14:07):