Stream: git-wasmtime

Topic: wasmtime / PR #10036 Split off wasmtime-wasi-io crate fro...

view this post on Zulip Wasmtime GitHub notifications bot (Jan 16 2025 at 21:03):

pchickey opened PR #10036 from bytecodealliance:pch/wasi_io_crate to bytecodealliance:main:

Follow-on to https://github.com/bytecodealliance/wasmtime/pull/10016

This takes the bindings generation for the wasi-io crate, the definitions of the Subscribe, HostInputStream, and HostOutputStream traits, and the freshly separated IoView machinery, and puts it into its own crate wasmtime-wasi-io.

Users of wasmtime-wasi and others never have to know or care about this change, the wasmtime-wasi crate re-exports everything now defined inwasmtime-wasi-io at the same names as prior to this PR.

Optionally, crates which depend on wasmtime-wasi for just the wasi-io primitives May depend directly on wasmtime-wasi-io directly. I made changes in this PR so that wasmtime-wasi-http changes its imports to be from wasmtime-wasi-io.

wasmtime-wasi-http can't break its dep on wasmtime-wasi for two reasons: it depends on wasmtime_wasi::runtime to provide common code for in_tokio sync wrappers of async and task spawning, and its linker needs to provide the std{in,out,err} interfaces from the wasi-cli implementation in wasmtime-wasi.

The big motivator of this change is to provide a common definition of wasi-io to be reusable by alternative (to wasmtime-wasi) implementations of the rest of WASI. In particular, this crate is compatible with !#[no_std] Rust.

I had to make one minor functional change to ResourceTable::iter_entries and impl poll::Host { fn poll(...) {...} } to use a BTreeMap in place of a HashMap in order to enable building wasmtime-wasi-io without std enabled in the wasmtime crate features.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 16 2025 at 21:03):

pchickey requested fitzgen for a review on PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 16 2025 at 21:03):

pchickey requested wasmtime-default-reviewers for a review on PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 16 2025 at 21:03):

pchickey requested wasmtime-core-reviewers for a review on PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 16 2025 at 21:05):

pchickey updated PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 16 2025 at 22:56):

pchickey updated PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 16 2025 at 22:58):

pchickey updated PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 16 2025 at 23:04):

pchickey updated PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 16 2025 at 23:08):

pchickey updated PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 16 2025 at 23:12):

pchickey updated PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 16 2025 at 23:18):

pchickey updated PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 16 2025 at 23:19):

pchickey updated PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 16 2025 at 23:24):

pchickey requested alexcrichton for a review on PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 16 2025 at 23:24):

pchickey commented on PR #10036:

I updated and wrote a bunch of docs for this PR that were not included in #10016 - ready for review now @alexcrichton

view this post on Zulip Wasmtime GitHub notifications bot (Jan 17 2025 at 00:53):

pchickey updated PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 17 2025 at 00:59):

pchickey edited PR #10036:

Follow-on to https://github.com/bytecodealliance/wasmtime/pull/10016

The big motivator of this change is to provide a common definition of wasi-io to be reusable by alternative (to wasmtime-wasi) implementations of the rest of WASI. In particular, this crate is compatible with !#[no_std] Rust.

Additionally, it contains renames for things that have been bugging me for a year or two.

New wasmtime-wasi-io crate

This takes the bindings generation for the wasi-io crate, the definitions of the Subscribe, HostInputStream, and HostOutputStream traits, and the freshly separated IoView machinery, and puts it into its own crate wasmtime-wasi-io.

Users of wasmtime-wasi and others never have to know or care about the change to the crate, the wasmtime-wasi crate re-exports everything now defined inwasmtime-wasi-io prior to the PR. However, names have changed!

Optionally, crates which depend on wasmtime-wasi for just the wasi-io primitives May depend directly on wasmtime-wasi-io directly. I made changes in this PR so that wasmtime-wasi-http changes its imports to be from wasmtime-wasi-io.

wasmtime-wasi-http can't break its dep on wasmtime-wasi for two reasons: it depends on wasmtime_wasi::runtime to provide common code for in_tokio sync wrappers of async and task spawning, and its linker needs to provide the std{in,out,err} interfaces from the wasi-cli implementation in wasmtime-wasi.

Trait and type renaming

Programmers using the direct bindings got to see the wit Pollable, InputStream, and OutputStream names inside Resource<...>. However, mapping from the resources to the traits is a boring bit of boilerplate that most programmers only ever encounter very briefly, and then spend most of their time interacting with the traits. The traits are the inconsistently-named Subscribe, HostInputStream, and HostOutputStream. Host... prefix these days means bindgen-created, while back in wasi-common meant not-bindgen-created. And Subscribe doesn't describe that trait at all!

So, I renamed according to the following:

    resource type Pollable -> DynPollable
    resource type InputStream -> DynInputStream
    resource type OutputStream -> DynOutputStream

    trait Subscribe -> Pollable
    trait HostInputStream -> InputStream
    trait HostOutputStream -> OutputStream

    type alias PollableFuture -> DynFuture (little-used)
    type alias ClosureFuture deleted (never used)

etc

I had to make one minor functional change to ResourceTable::iter_entries and impl poll::Host { fn poll(...) {...} } to use a BTreeMap in place of a HashMap in order to enable building wasmtime-wasi-io without std enabled in the wasmtime crate features.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 17 2025 at 01:11):

pchickey updated PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 17 2025 at 18:51):

alexcrichton submitted PR review.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 17 2025 at 18:51):

alexcrichton created PR review comment:

As a proof-of-concept would it be possible to not add the dependency here? Or is it required that those using wasmtime-wasi have to also depend on wasmtime-wasi-io?

view this post on Zulip Wasmtime GitHub notifications bot (Jan 17 2025 at 19:24):

pchickey submitted PR review.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 17 2025 at 19:24):

pchickey created PR review comment:

Yes, its possible to not add the dependency and use everything through wasmtime-wasi. I'm happy to convert it over to that if you think that is valuable, I could go either way.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 17 2025 at 22:25):

alexcrichton submitted PR review.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 17 2025 at 22:25):

alexcrichton created PR review comment:

Yeah if you wouldn't mind doing that I think it'd be good to have an example and basically a test for that. In Spin for example I'd hope to ideally only need to refactor a bit and not have to explicitly pick up wasmtime-wasi-io (otherwise we might need to extend docs more to mention it)

view this post on Zulip Wasmtime GitHub notifications bot (Jan 22 2025 at 00:14):

pchickey updated PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 22 2025 at 00:14):

pchickey submitted PR review.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 22 2025 at 00:14):

pchickey created PR review comment:

Done - hitting merge.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 22 2025 at 00:15):

pchickey has enabled auto merge for PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 22 2025 at 00:16):

pchickey updated PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 22 2025 at 05:28):

pchickey updated PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 22 2025 at 05:28):

pchickey has enabled auto merge for PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 22 2025 at 17:40):

pchickey updated PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 22 2025 at 17:52):

pchickey has enabled auto merge for PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 22 2025 at 19:09):

pchickey updated PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 22 2025 at 19:11):

pchickey updated PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 22 2025 at 19:22):

pchickey has enabled auto merge for PR #10036.

view this post on Zulip Wasmtime GitHub notifications bot (Jan 22 2025 at 19:47):

pchickey merged PR #10036.

Last updated: Jan 24 2025 at 00:11 UTC