Can records under the C-M be transparently handled hostside · general

Basically, I want to define a set of services using WASM and the component model. Each service has a state. The host runtime does not need to know what the state contains. Each service is defined using WIT. The WIT file is used to generate the guest code.

// First service.
interface service1 {
    record service1-state {
        last_seen_id: u64,
        last_username: string,
        last_diagnostic_data: list<string>,
    }

    init: func() -> service1-state;
    process: func(service1-state) -> service1-state;
}

// Second service.
interface service2 {
    record service2-state {
        username: string,
    }

    init: func() -> service2-state;
    process: func(service2-state) -> service2-state;
}

Given that each service processes their old state and transforms it into a new state, can the host run any type of service that adheares to that interface? I.e., take _some_ kind of state as input to the process function and return the new state. Store the state temporarily until the next process execution, pass the stored record to the process function call, repeat.

As far as I can tell, this does not seem to be possible. The host (in my case wasmtime) has to know all states during compile time.

Victor Adossi (Mar 24 2025 at 10:23):

Does encoding the state in a serializable format (Text, JSON, Protobuf, etc) work as a solution to your problem? Adding a function that derives/hydrates a service-state from an existing serialized format would be the most flexible way (and maybe only way) to achieve your goal presently.

mainrs (Mar 24 2025 at 10:41):

I can encode them. I was hoping that I could avoid that, because the state is rather large and deeply nested, so there would be some overhead.

In my current approach (no C-M), host writes the state into the WASM module's linear memory and passes the pointer during the process method invocation.

I was thinking about how to convert this to the C-M, since that comes with a lot of (future) benefits like result types, async, resources and more.

Victor Adossi (Mar 24 2025 at 11:54):

Is it possible for you to use some in-memory structures that might make serialization a bit easier?

As far as a component-model native solution, I'd be happy to be wrong here but I don't think there are any solutions for this at present -- I think the closest thing that I've seen might be https://github.com/WebAssembly/component-model/issues/172

Sketch: allow a variable number of imports/exports of fixed type in Wit · Issue #172 · WebAssembly/component-model

Motivation Let's say I'd like to build a component that consumes 3 configuration values a, b and c (which in a 12-factor app I'd take as 3 environment variables). I could define a component with ty...

mainrs (Mar 24 2025 at 12:21):

My modules and host are written in C/++. Wouldn't a simple memcpy into a buffer be enough for "serialization", as C ABI is stable? My other idea was to rely on a zero-copy de/ser framework. At least that should make it more efficient when compared to JSON et al.

mainrs (Mar 24 2025 at 12:22):

Thanks for the issue link! It does look similar to my "issue". I'll keep an eye on it!

Victor Adossi (Mar 24 2025 at 13:41):

If you know the relevant components and host are C/C++ that could certainly work (and most other languages too, given that C FFI is pretty well used)

Yeah my thought was more on the zero copy side -- thinking of serialization formats that allow for in-memory structures to stay unchanged. I think either approach could work if you can stomach the memcpy!

Stream: general