Wasm Components and service-oriented architecture · wasm

I'm seeking ideas about how best to think about WebAssembly Components in the context of service-oriented architecture (SOA). My immediate use case is a personal project that is all-in on Wasm Components, so it's fine for anything that isn't either a Component or the Rust host to be a second-class citizen. But I'm also thinking about this more broadly, so any related ideas are very much welcome and appreciated.

TL;DR: Try to have a single WIT-defined interface for each service, or define a first-class "local" WIT interface and a second-class REST-like "remote" interface (maybe WIT, maybe not) on top of it?

The last dot point here in particular keeps things simple, but it's also a kind of lowest-common-denominator approach that's emerged from a lack of better options.

Wasm Components complicate things by offering more sophisticated tools (resource handles!) which are awesome but re-open the can of worms about how to deal with the distinction between remote and local calls. And even within one process, some "services" will probably exist in different component instances / stores with a limited number of instances — e.g. anything that needs to manage a connection pool (some small n), long-lived exclusive access to a file (n = 1), etc.

Has anybody else with a vaguely similar use case come to any conclusions? I'd love to hear anything, be it advice, ideas, stories, etc.

Jeff Parsons (Nov 02 2023 at 05:18):

How is it imagined that this would work in implementations? Is the idea that the bucket resource on a client side would represent just some data (bucket identifier) and possibly a session token or something like that?

I think this is probably a good example to think about, because I have several services with similar structures — i.e. resources with their own access controls, then sub-resources within those. I imagine the reasoning behind however wasi-kv might carry over pretty well...

Lann Martin (Nov 02 2023 at 13:34):

I've thought about this quite a bit. In the short-term, I think the pragmatic approach would be to define your schema for an existing RPC system you want to use like gRPC (protobuf) or something JSON-based and generate adapters to/from WIT.

Lann Martin (Nov 02 2023 at 13:39):

In the long term I could imagine integrating resources into an RPC system with capabilities like CapTP (see also https://capnproto.org/, but be aware that lots of the magic described there isn't actually implemented [or wasn't last time I looked carefully])

Jeff Parsons (Nov 05 2023 at 04:40):

Thanks for the feedback, Lann. I've had a read of those links. I'm keen to use (possibly some subset of) WIT for authoritative API definitions, and there's only one host even if guests can be written in other languages, so I'm hoping to keep the guests oblivious of details of the wire protocol etc.

I think I'm going to poke around a bit more to better understand the design space before committing too hard to anything.

Lann Martin (Nov 06 2023 at 14:13):

Its definitely possible if you avoid resources, and even resources aren't impossible per se, just very tricky to nail down semantics/constraints. I have an old (pre-resources) poc server that exposes an input component as an HTTP/JSON service which might be useful as reference: https://github.com/lann/component-rpc

Jeff Parsons (Nov 06 2023 at 17:04):

Oh wow, that demonstrates most of what I had in mind quite neatly — very helpful. Thanks!

I can certainly start without resources and then worry about if/how to do them later. I'll see if whatever I cook up seems like it might be generally useful and spin it out into a separate crate/repo if so. (The parent project is a bit pie-in-the-sky and so might not ever be published.)

Jeff Parsons (Nov 09 2023 at 00:13):

For reference, this is what I'm doing for my same-OS-process-but-different-store calls:

use std::{
    collections::HashMap,
    sync::{Arc, Mutex},
};

use anyhow::Context;
use wasmtime::{component, Store};

pub struct ComponentInstanceProxy<T> {
    export_instance_name: String,
    store: Arc<Mutex<Store<T>>>,
    funcs: HashMap<String, component::Func>,
}

impl<T: Send + 'static> ComponentInstanceProxy<T> {
    /// Validates that the instance and all the requested functions exist up-front.
    ///
    /// TODO: Don't require the caller to pass in a list of functions;
    /// figure it out yourself, and return `None` if the instance wasn't found.
    pub fn new(
        root_component_instance: &component::Instance,
        store: Arc<Mutex<Store<T>>>,
        export_instance_name: &str,
        func_names: &[&str],
    ) -> anyhow::Result<Self> {
        let mut funcs = HashMap::new();

        {
            let mut store = store.lock().expect("Mutex was poisoned");
            let mut exports = root_component_instance.exports(&mut *store);
            let mut export_instance =
                exports.instance(export_instance_name).with_context(|| {
                    format!(
                        "Export instance \"{}\" doesn't exist in root component instance.",
                        export_instance_name
                    )
                })?;
            for func_name in func_names {
                let func = export_instance.func(func_name).with_context(|| {
                    format!(
                        "Function \"{}\" doesn't exist on export instance \"{}\".",
                        func_name, export_instance_name
                    )
                })?;
                funcs.insert(func_name.to_string(), func);
            }
        }

        Ok(Self {
            export_instance_name: export_instance_name.to_string(),
            store,
            funcs,
        })
    }

    pub fn add_to_linker<U>(
        &self,
        component: &component::Component,
        linker: &mut component::Linker<U>,
    ) -> anyhow::Result<()> {
        let mut import_instance =
            linker
                .instance(&self.export_instance_name)
                .with_context(|| {
                    format!(
                        "Import instance \"{}\" doesn't exist in source root component instance.",
                        self.export_instance_name
                    )
                })?;
        for (func_name, func) in &self.funcs {
            let store = self.store.clone();
            let func = *func;
            let func_new_res =
                import_instance.func_new(component, func_name, move |_store, inputs, outputs| {
                    let mut store = store.lock().expect("Mutex was poisoned");

                    // REVISIT: Wasmtime appears to happily accept _structurally equivalent_
                    // types here, even if they are defined in different stores.
                    // But I can't see mention of this in the docs. So make sure we understand
                    // what the actual guarantees are here, and whether we should be
                    // doing any conversion ourselves!
                    func.call(&mut *store, inputs, outputs)?;
                    func.post_return(&mut *store)
                });
            if func_new_res.is_err() {
                // TODO: Convert all this into logging/tracing.
                eprintln!(
                    "Function \"{}\" doesn't exist on import instance \"{}\". Skipping.",
                    func_name, self.export_instance_name
                );
            }
        }
        Ok(())
    }
}

I'm kinda surprised that this works (see comment about type equivalence across stores). Reading the implementation I can kinda see how that behaviour would fall out the other end, but it's not obvious to me whether it's actually deliberate that this should work.

Lann Martin (Nov 09 2023 at 13:36):

All current component model value types (param and result field types) except resources are structural.

Ralph (Nov 13 2023 at 19:52):

Apologies for the late sail, but KubeCon disturbs all who enter its event horizon. Speaking as one who helped implement WS-* systems, your take on SOA is pretty good, and I'd always advise anyone to read Lann's examples for ideas. But the one thing that SOA was meant to do was NOT to model the pipe, and that was kind of a mistake. Turns out what worked was a simple pipe. (Mind you, this is one of my takes, possibly not yours.) But it is now quite ironic that all rpcs can be modeled as message exchanges; which means that using wasm components do not have a stance on anything about the pipe, only about the surface of things. SOA tried to define a surface for everyone; wasm components enable everyone to define THEIR surface -- out of which some standards will appear.

Ralph (Nov 13 2023 at 19:53):

that said, it merely means you could create a SOA system easily enough, but only SOA would try to have some opinion what actually was implemented on one side in terms of types.

Stream: wasm

Topic: Wasm Components and service-oriented architecture

Jeff Parsons (Nov 02 2023 at 03:09):