Stream: wit-bindgen

Topic: Building WASM components that don't require WASI

Cameron M (May 10 2023 at 06:33):

Hi, this is somewhat related to my previous thread, but feels different enough to warrant it's own thread.

I'm trying to build a WASM component that exposes a very simple interface:

default world myapp {
  record foo {
    bar: u64,
    baz: u64,
  }

  import some-host-func: func(s: string)

  export do-the-thing: func(data: foo)
}

I'm building the WASM binary with Rust + wasm-tools component new ... --adapt wasi_snapshot_preview1.wasm.

The issue is, this then creates a component that expects the WASI functions to be present, which they are not in my Host. Currently, I'm getting errors when I try to load the module saying things like drop-input-stream has the wrong type, expected func found nothing. My understanding is that this is a function defined in WASI, and this error message is basically saying:

• this WASM component expects the host to provide a function called drop-input-stream
• the host doesn't provide such a function

Is there any way to build a WASM component in such a way that it doesn't try to link against WASI functions?

Steven Johnson (May 10 2023 at 07:33):

I was working on this issue with Cameron, and it does look odd.
Specifically the WASI standard says "The WebAssembly System Interface is not a monolithic standard system interface, but is instead a modular collection of standardized APIs. None of the APIs are required to be implemented to have a compliant runtime. Instead, host environments can choose which APIs make sense for their use cases."

So, it seems strange that without explicitly defining particular WASI interfaces to be used that the wasm binary is being forced to use stdin/stdout, which in turn pulls in the streaming and fs API's. In addition to environment.

A wit dump of the resulting file shows this:

interface streams {
  type input-stream = u32

  type output-stream = u32

  record stream-error {
  }

  drop-input-stream: func(this: input-stream)

  write: func(this: output-stream, buf: list<u8>) -> result<u64, stream-error>

  blocking-write: func(this: output-stream, buf: list<u8>) -> result<u64, stream-error>

  drop-output-stream: func(this: output-stream)
}

interface filesystem {
  use self.streams.{output-stream}

  type descriptor = u32

  type filesize = u64

  enum descriptor-type {
    unknown,
    block-device,
    character-device,
    directory,
    fifo,
    symbolic-link,
    regular-file,
    socket,
  }

  enum error-code {
    access,
    would-block,
    already,
    bad-descriptor,
    busy,
    deadlock,
    quota,
    exist,
    file-too-large,
    illegal-byte-sequence,
    in-progress,
    interrupted,
    invalid,
    io,
    is-directory,
    loop,
    too-many-links,
    message-size,
    name-too-long,
    no-device,
    no-entry,
    no-lock,
    insufficient-memory,
    insufficient-space,
    not-directory,
    not-empty,
    not-recoverable,
    unsupported,
    no-tty,
    no-such-device,
    overflow,
    not-permitted,
    pipe,
    read-only,
    invalid-seek,
    text-file-busy,
    cross-device,
  }

  write-via-stream: func(this: descriptor, offset: filesize) -> output-stream

  append-via-stream: func(this: descriptor) -> output-stream

  get-type: func(this: descriptor) -> result<descriptor-type, error-code>

  drop-descriptor: func(this: descriptor)
}

interface environment {
  get-environment: func() -> list<tuple<string, string>>
}

interface preopens {
  use self.streams.{input-stream, output-stream}
  use self.filesystem.{descriptor}

  record stdio-preopens {
    stdin: input-stream,
    stdout: output-stream,
    stderr: output-stream,
  }

  get-stdio: func() -> stdio-preopens

  get-directories: func() -> list<tuple<descriptor, string>>
}

interface exit {
  exit: func(status: result)
}

default world component {
  import streams: self.streams
  import filesystem: self.filesystem
  import environment: self.environment
  import preopens: self.preopens
  import exit: self.exit
  import some-host-func: func(s: string)
  record foo {
    bar: u64,
    baz: u64,
  }

  export do-the-thing: func(data: foo)
}

Eduardo Rodrigues (May 10 2023 at 08:39):

i believe the issue is with the adapt parameter that you are passing. if you are specifying a WASI adapter, it will expect the default interfaces to be available

Cameron M (May 10 2023 at 08:41):

In the other thread, I was recommended to check out https://github.com/bytecodealliance/preview2-prototyping and add it on the host side. My understanding is that this provides those interfaces on the host side, but I'm not certain. Do you know of any resources/examples that show how to do this?

Ramon Klass (May 10 2023 at 11:38):

sorry I forgot to answer yesterday, I think your component needs the whole wasi spec because you adapt the whole snapshot_preview1 polyfill, but I'm not entirely sure if that's correct. My own code is still using preview2, but looking at https://github.com/bytecodealliance/componentize-js/tree/main/example it seems to not be needed anymore.
that componentize-js example comes with a rust host that can run the example component, I used it as a reference to get my host to work

Ramon Klass (May 10 2023 at 11:54):

I think it boils down to "you need to use wasmtime from git or wait for the next release" since the wasi host is now in wasmtime but there was no release since

Scott Waye (May 18 2023 at 01:20):

This sounds related to this topic so will post here, sorry if I am hijacking. If you don't --adapt .... Are you still required to import the wasi interface? It seems so because otherwise wasm-tools component new gives an error

PS C:\github\rustimpl\rustimpl> wasm-tools component new ./target/wasm32-wasi/debug/rustimpl.wasm -o my-component.wasm
error: failed to encode a component from module

Caused by:
    0: module requires an import interface named `wasi_snapshot_preview1`
PS C:\github\rustimpl\rustimpl>

Scott Waye (May 18 2023 at 01:24):

My wit

interface floats {
  float32-param: func(x: float32)
  float64-param: func(x: float64)
  float32-result: func() -> float32
  float64-result: func() -> float64
}

default world the-world {
  import imports: self.floats
  export exports: self.floats
}

Salim Afiune Maya (May 18 2023 at 04:39):

I have the same question. https://bytecodealliance.zulipchat.com/#narrow/stream/206238-general/topic/.E2.9C.94.20Unable.20to.20bind.20vanilla.20Rust.20host.2Fguess.20component/near/358751939

If there is a way to not import the wasi interface, I would also like to know.

Scott Waye (May 18 2023 at 12:47):

I guess I need to build with a different target, i.e. not wasm32-wasi, if there is one. Alternatively, how do you import that interface in rust. Is it in the wit, or the rust? Sorry I'm .net really, just trying the tools for rust<->c#

Salim Afiune Maya (May 18 2023 at 19:12):

No worries, Scott. I am in the same boat :smile:

I don't know if there is another target but, I used the adapter (look at the tags/releases) https://github.com/bytecodealliance/preview2-prototyping to build the component and then, on the Host side, not the guest (module), I imported the wasi-common crate from the same repo:

cargo add --git https://github.com/bytecodealliance/preview2-prototyping wasi-common

And then loaded the module via:

    let mut config = Config::new();
    config.wasm_backtrace_details(wasmtime::WasmBacktraceDetails::Enable);
    config.wasm_component_model(true);
    config.async_support(true);
    info_!("Loading Component on Engine");
    let engine = Engine::new(&config)?;
    let component = Component::new(&engine, &bytes)?;

    info_!("Creating Linker");
    let mut linker = Linker::new(&engine);
    wasi::streams::add_to_linker(&mut linker, |x| x)?;
    wasi::filesystem::add_to_linker(&mut linker, |x| x)?;
    wasi::environment::add_to_linker(&mut linker, |x| x)?;
    wasi::preopens::add_to_linker(&mut linker, |x| x)?;
    wasi::exit::add_to_linker(&mut linker, |x| x)?;

Salim Afiune Maya (May 18 2023 at 19:13):

I used this as an example https://github.com/bytecodealliance/preview2-prototyping/blob/5be99d645c177198b9f5e4d76561669796f7b379/host/tests/reactor.rs#L57-L75

Peter Huene (May 18 2023 at 20:19):

The wasm32-unknown-unknown target will forego the WASI imports (so cargo build --target wasm32-unknown-unknown or cargo component build --target wasm32-unknown-unknown if using cargo-component).

That said, it's still best to target WASI from your components and enable preview2 in the host (which is just about to land into Wasmtime itself with this PR).

Scott Waye (May 18 2023 at 20:38):

Thanks for the target and the link. Look forward to trying preview2

DougAnderson444 | PeerPiper.io (Jun 08 2023 at 01:21):

What's the latest way to do this linker? I see the preview2-prototyping repo is deprecating... is the replacement ready to use as of now? I'm still getting the drop-input-stream error no matter what I do. Would love if someone could share an end to end hello world from component to host!

Ramon Klass (Jun 08 2023 at 11:59):

https://bytecodealliance.zulipchat.com/#narrow/stream/219900-wasi/topic/preview2.20in.20wasmtime.2010.2E0.20release/near/364067517

I think what Pat said means wasmtime repo from git should work right now but wasmtime 10.0 that comes out in 1.5 weeks will likely also not work out of the box

Last updated: Nov 22 2024 at 16:03 UTC