Stream: general

Topic: how to send a resource from one component to another


view this post on Zulip David Craven (Jul 22 2026 at 06:53):

I have a plugin system that has a shared interface the content resource:

interface content {
  use mime-registry.{mime-info};

  type error = string;

  resource content-reader {
    stream-from: func(offset: u64) -> result<stream<u8>, error>;
    read-to-end: async func() -> result<list<u8>, error>;
  }

  record content {
    url: string,
    info: mime-info,
    length: option<u64>,
    content: content-reader,
  }
}

interface network-provider {
  use content.{content, error};
  set-supported-encodings: async func(encodings: list<string>);
  fetch: async func(url: string) -> result<content, error>;
}

world network {
  import mime-registry;
  export content;
  export network-provider;
}

and the view interface consumes the content:

interface view-provider {
  use content.{content, error};

  variant input-event {
    window-resize(tuple<u32, u32>),
  }

  record frame {
    width: u32,
    height: u32,
    bytes: list<u8>,
  }

  load: async func(content: content) -> result<u8, string>;
  on-input: func(event: input-event);
  needs-redraw: func() -> bool;
  render: func() -> frame;
}

world view {
  import content;
  export view-provider;
}

this is then the host interface:

world omnishell {
  include network;
  include archive;
  include compression;
  include view;
}

the plugin interface looks like this:

pub mod network {
    pub use super::bindings::exports::omnishell::provider::network_provider::*;
    use super::{Component, Linker, OmnishellManifest, Store};
    use url::Url;

    pub struct NetworkPlugin {
        schemes: Vec<String>,
        component: Component,
    }

    impl NetworkPlugin {
        pub fn new(manifest: OmnishellManifest, component: Component) -> Result<Self, String> {
            Ok(Self {
                schemes: manifest.provides.schemes,
                component,
            })
        }

        pub fn schemes(&self) -> &[String] {
            &self.schemes
        }

        pub async fn fetch(
            &self,
            store: &mut Store,
            linker: &Linker,
            url: &Url,
            supported_encodings: Vec<String>,
        ) -> Result<Content, String> {
            let instance_pre = linker
                .instantiate_pre(&self.component)
                .map_err(|e| format!("failed to pre-instantiate: {e}"))?;
            let guest_indices = GuestIndices::new(&instance_pre)
                .map_err(|e| format!("failed to read guest indices: {e}"))?;
            let instance = instance_pre
                .instantiate_async(&mut *store)
                .await
                .map_err(|err| format!("failed to instantiate: {err}"))?;
            let guest = guest_indices.load(&mut *store, &instance).unwrap();
            let content = store
                .run_concurrent(async |accessor| {
                    guest
                        .call_set_supported_encodings(accessor, supported_encodings)
                        .await
                        .map_err(|e| e.to_string())?;
                    guest
                        .call_fetch(accessor, url.to_string())
                        .await
                        .map_err(|e| e.to_string())?
                })
                .await
                .map_err(|e| e.to_string())??;
            Ok(content)
        }
    }
}

and finally the open function that ties it together:

    pub async fn open(&self, url: Url) -> Result<(Url, WasmWidgetState), String> {
        let mut store = Store::new(
            &self.engine,
            HostState {
                table: ResourceTable::new(),
                ctx: wasmtime_wasi::WasiCtxBuilder::new()
                    .inherit_stdout()
                    .build(),
                http_ctx: wasmtime_wasi_http::WasiHttpCtx::new(),
            },
        );
        let content = self.fetch(&mut store, url).await?;
        let url: Url = content
            .url
            .parse()
            .map_err(|err| format!("invalid url {}: {err}", &content.url))?;
        let plugin = self.view_plugin(&content.info.mime)?;
        let guest = plugin
            .load(&mut store, &self.linker, content)
            .await
            .unwrap();
        Ok((url, WasmWidgetState::new(store, guest)))
    }

however I can't figure out how to link the export content of the network_provider to the import content of the view_provider. can you point me to an example or docs on how to do this dynamically at runtime, since the plugin instance depends on the returned mime type, I can't just wasm-tools compose them. and the Linker seems to be only useful for satisfying host exports/imports.

view this post on Zulip David Craven (Jul 22 2026 at 07:41):

essentially I'm looking for something like this:

use wasmtime::component::{Component, Linker, Store};

// 1. Instantiate the dependency first
let mut linker: Linker<MyState> = Linker::new(&engine);
let dependency_instance = linker.instantiate(&mut store, &dependency_component)?;

// 2. Setup a fresh, clean linker for the plugin
let mut plugin_linker: Linker<MyState> = Linker::new(&engine);

// 3. THE PREMADE WAY: Fetch the entire interface block from the dependency
let content_export = dependency_instance
    .get_export(&mut store, None, "ns:pkg/content")
    .expect("Dependency must export the interface");

// 4. Feed the entire exported block into the plugin's required import slot
plugin_linker.instance("ns:pkg/content")?
    .add_export(&content_export)?; // This maps all functions AND resources natively

// 5. Instantiate the plugin safely
let plugin_instance = plugin_linker.instantiate(&mut store, &plugin_component)?;

view this post on Zulip Alex Crichton (Jul 22 2026 at 14:49):

Ah yeah direct component-to-component linking isn't supported in Wasmtime right now, for that you'll have to either intermediate on the host (e.g. with func_new) or pre-compose ahead of time

view this post on Zulip David Craven (Jul 22 2026 at 15:30):

sorry going to need a little bit more help than that. currently it just doesn't work. For example the wasmtime::component::bindgen, just makes the ContentReader {}, so its not clear how I can even read the data on the host side. then it forces some HostContentReader and HostContentReaderWithStore traits on the state, which I guess if I could somehow call the methods on ContentReader I could implement to forward the calls from one guest component to another

view this post on Zulip Alex Crichton (Jul 22 2026 at 17:30):

the rough idea is that you would implement the traits by calling the guest component, so you'd store the one instance within the T of Store<T> and then the implementation of HostContentReader* would use this instance as a recursive call into wasm

view this post on Zulip Alex Crichton (Jul 22 2026 at 17:31):

bindgen! auto-creates types for resources but you can customize those if you wish, in general though resources at that layer can be considered "type helpers" more than required state

view this post on Zulip Alex Crichton (Jul 22 2026 at 17:31):

(sorry I know this is probably a lot of jargon, but I don't know how else to answer this)

view this post on Zulip David Craven (Jul 22 2026 at 19:27):

still stuck on missmatched resource types. this is what I tried this time:

  1. after fetch I convert the guest content reader to a host content reader:
            let content_reader = GuestContentReader::new(&mut *store, &instance, content.content);
            let resource = store.data_mut().table.push(content_reader).unwrap();
            content.content = resource.try_into_resource_any(&mut *store).unwrap();
            Ok(content)

2: GuestContentReader just holds all the stuff to read the content:

    #[derive(Clone)]
    pub struct GuestContentReader {
        guest: Guest,
        reader: ResourceAny,
    }

    impl GuestContentReader {
        pub fn new(store: &mut Store, instance: &Instance, reader: ResourceAny) -> Self {
            let guest_indices = GuestIndices::new(&instance.instance_pre(&mut *store)).unwrap();
            let guest = guest_indices.load(&mut *store, &instance).unwrap();
            Self { guest, reader }
        }

        pub async fn read_to_end(&self, accessor: &Accessor) -> Result<Vec<u8>, String> {
            self.guest
                .content_reader()
                .call_read_to_end(accessor, self.reader)
                .await
                .unwrap()
        }

        pub fn stream_from(
            &self,
            store: impl AsContextMut,
            offset: u64,
        ) -> Result<StreamReader<u8>, String> {
            self.guest
                .content_reader()
                .call_stream_from(store, self.reader, offset)
                .unwrap()
        }
    }
  1. finally the host traits forward the calls:
    impl content::Host for HostState {}

    impl content::HostContentReader for HostState {
        fn stream_from(
            &mut self,
            rep: Resource<content::ContentReader>,
            offset: u64,
        ) -> Result<StreamReader<u8>, String> {
            let rep: Resource<GuestContentReader> = unsafe { std::mem::transmute(rep) };
            let reader = self
                .table
                .get(&rep)
                .map_err(|e| format!("Failed to look up reader: {e}"))?;
            //reader.stream_from(self, offset)
            todo!()
        }

        fn drop(&mut self, rep: Resource<content::ContentReader>) -> Result<(), wasmtime::Error> {
            self.table.delete(rep)?;
            Ok(())
        }
    }

    impl content::HostContentReaderWithStore<HostState> for HasSelf<HostState> {
        async fn read_to_end(
            cx: &Accessor,
            rep: Resource<content::ContentReader>,
        ) -> Result<Vec<u8>, String> {
            let rep: Resource<GuestContentReader> = unsafe { std::mem::transmute(rep) };
            let reader = cx
                .with(|mut access| {
                    let host_state = access.data_mut();
                    let reader: &GuestContentReader = host_state
                        .table
                        .get(&rep)
                        .map_err(|e| format!("Reader handle not found: {e}"))?;
                    Ok::<_, String>(reader.clone())
                })
                .unwrap();
            reader.read_to_end(cx).await.map_err(|e| e.to_string())
        }
    }

am I on the right track here or way off?

view this post on Zulip Alex Crichton (Jul 23 2026 at 14:44):

Hm yeah that all looks correct to me (although I'd recommend replacing the transmute with a safe call to rep + new instead). Would you be able to share a small repro of the error that I could poke around?

view this post on Zulip David Craven (Jul 23 2026 at 16:10):

here is a repo that shows the issue:
https://github.com/dvc94ch/omnishell-repro

view this post on Zulip Alex Crichton (Jul 23 2026 at 23:55):

Ok had a chance to dig into this now and see what's going on. What you're running into is subtelties of how resources work in the component model. To get this example to work you have to get everything to line up exactly such that the resource exported by one component is exactly the resource imported by another component.

One issue is that the omnishell world is serving a dual-purpose of bindings for both components. I think you'll probably want to generate separate bindings for the two components with two separate worlds. Through that you can configure what exactly is the imported resource's type for one of the worlds, and that's where you'll specify your GuestContentReader structure for example. I ran out of time today to build out an example of this in your repo, but this is hopefully a bit more specific than some hand-waving!

view this post on Zulip David Craven (Jul 24 2026 at 13:13):

ok, finally made some progress. thanks for your guidance. I'm currently running into the issue that the loading of state is async in the view_provider but the rendering and input handling is synchronous. It has to be synchronous as the ui framework is synchronous and somehow making it async would be a major pain. also I don't see exactly why they have to be async. However wasmtime complains with: "value: store configuration requires that *_async functions are used instead". another thing I don't understand is why bother making wit functions async at all in the first place, since you end up having to use wstd::runtime::block_on if you're doing any actual io which is a bit confusing.

view this post on Zulip David Craven (Jul 24 2026 at 15:21):

and testing the compression/archive plugins that both take and return a ContentReader, we're back to the same issue "missmatched resource types". the issue seems to be that when the same interface is used as an argument and return type, the Resource<ContentReader> becomes ResourceAny which doesn't seem to work

view this post on Zulip Alex Crichton (Jul 24 2026 at 15:24):

For sync bits you'll want to change this to add_to_linker_sync. If you never add anything async to a linker/store then you'll be able to use sync interfaces. Note though that all I/O will block the main thread in this case.

For the mismatch, mind updating the repo to poke at it? There's a lot of possible causes for that and it pretty heavily depends on the exact shape of things to determine what the cause is

view this post on Zulip David Craven (Jul 24 2026 at 16:04):

I updated the repo to reproduce the issue. Thanks again for you help, really appreciate it.

I worked around the Store issue by just using futures::executor::block_on on "sync" methods, which is something that could probably be generated if the wit interface is sync and its just a matter of wiring it through wasmtime.

I assume the reactor will be provided by wasmtime in the future to avoid wstd::runtime::block_on for futures/streams that need to register an io/poll thingie?

view this post on Zulip Alex Crichton (Jul 24 2026 at 17:09):

as a heads up I'm technically on PTO today so may not get to this for a bit

view this post on Zulip David Craven (Jul 27 2026 at 15:44):

@Alex Crichton were you able to give it a try?

and do HostWithStore methods have some kind of a timeout? because the HostWithStore fetch_bytes implementation is dropping the tokio::sync::oneshot::Receiver its awaiting, which is puzzling unless a higher power is dropping the fetch_bytes future

view this post on Zulip David Craven (Jul 27 2026 at 15:48):

impl shell_provider::HostWithStore<HostState> for HasSelf<HostState> {
    async fn fetch_bytes(cx: &Accessor, url: String) -> Option<(String, Vec<u8>)> {
        let url = url.parse().ok()?;
        let shell = cx.with(|mut access| access.data_mut().shell.clone());
        let (url, bytes) = shell.fetch_bytes(url).await?;
        Some((url.to_string(), bytes))
    }

    async fn fetch_view(cx: &Accessor, url: String) -> Option<u32> {
        let url = url.parse().ok()?;
        let shell = cx.with(|mut access| access.data_mut().shell.clone());
        let (_url, view) = shell.fetch_view(url).await?;
        let id = cx.with(|mut access| {
            let children = &mut access.data_mut().children;
            let id = children.len();
            children.push(view);
            id
        });
        Some(id as u32)
    }
}
#[derive(Clone)]
pub struct ShellAction {
    tx: mpsc::UnboundedSender<ShellCommand>,
}

impl ShellAction {
    pub fn navigate(&self, url: Url) {
        self.tx.send(ShellCommand::Navigate(url)).unwrap();
    }

    pub async fn fetch_bytes(&self, url: Url) -> Option<(Url, Vec<u8>)> {
        let (tx, rx) = oneshot::channel();
        self.tx
            .send(ShellCommand::FetchBytes(url.clone(), tx))
            .unwrap();
        tracing::info!("waiting for response");
        // HERE future is dropped, how is it possible??
        let (url, bytes) = match rx.await {
            Ok((url, bytes)) => (url, bytes),
            Err(_) => {
                tracing::error!("fetch_bytes {url} oneshot tx dropped");
                return None;
            }
        };
        tracing::info!("received bytes for url {url}");
        Some((url, bytes))
    }

    pub async fn fetch_view(&self, url: Url) -> Option<(Url, WasmWidgetState)> {
        let (tx, rx) = oneshot::channel();
        self.tx.send(ShellCommand::FetchView(url, tx)).unwrap();
        rx.await.ok()
    }
}

view this post on Zulip Alex Crichton (Jul 27 2026 at 20:09):

I haven't yet, no, busy day

view this post on Zulip Alex Crichton (Jul 27 2026 at 22:26):

I'd still recommend removing transmutes, for example this patch compiles with no transmutes. Unsure if this'd run though.

Otherwise I believe what's happening is that this is using the wrong input resource. For the world you're calling it's got import content and export content. This means that the content.content-reader resource used by export transform-provider is actually the export content, not the import content. The resource you're passing in is the host's resource, which satisfies the import but not the export. The export expects a resource created by the guest itself, but you haven't invoked the guest to create the resource.

view this post on Zulip Alex Crichton (Jul 27 2026 at 22:26):

Depending on the exact layout you're trying to achieve here it may not be possible with WIT today

view this post on Zulip Alex Crichton (Jul 27 2026 at 22:27):

for example WIT can't describe one export referring to a resource imported, and another wit export referring to the same resource being exported (e.g. if an interface is both imported and exported)

view this post on Zulip Alex Crichton (Jul 27 2026 at 22:27):

the component model can handle that just fine, but WIT isn't expressive enough yet

view this post on Zulip David Craven (Jul 28 2026 at 07:35):

any suggestions for how to work around this limitation in wit? I think the two demos that need to work to show off omnishell are: https://omnishell.com/book.zip/index.md and https://omnishell.com/app.wasm and I'm close to a pretty cool demo. except for the built in file:// and application/wasm handlers, everything else are purely modular wasm plugins.

view this post on Zulip David Craven (Jul 28 2026 at 10:44):

just fyi gemini generated a cool wasm demo using vello-cpu and pointer events. so the only pieces missing for an mvp are:

  1. fix content -> content plugin types
  2. fetch_bytes drops the oneshot preventing the webview from fetching css/fonts
  3. use fetch_view in the html plugin to reduce its size and use the image and svg plugins by registering the img/svg tag as custom html elements
  4. read all the config from a config file

Last updated: Jul 29 2026 at 05:03 UTC