Stream: general

Topic: examples of wasi-http integration for client HTTP calls?

Zeeshan Lakhani (Jan 25 2024 at 15:22):

More of a general question, though I'm probably overlooking stuff. I've seen https://github.com/sunfishcode/hello-wasi-http, but was instead looking for examples that focus on outgoing requests made from the host. I came across https://github.com/smithy-lang/smithy-rs/pull/2520, but wasn't sure what the state of that was. We embed wasmtime within our runtime, so any examples demonstrating working with the preview2 bindings would be great as well. Thanks all.

Joel Dice (Jan 25 2024 at 15:26):

This example does outgoing requests: https://github.com/bytecodealliance/wasmtime/blob/main/crates/test-programs/src/bin/api_proxy_streaming.rs

Zeeshan Lakhani (Jan 25 2024 at 15:28):

Joel Dice said:

This example does outgoing requests: https://github.com/bytecodealliance/wasmtime/blob/main/crates/test-programs/src/bin/api_proxy_streaming.rs

Ok, yeah, this is what I was expecting/looking for. Thank you @Joel Dice.

Joel Dice (Jan 25 2024 at 15:34):

I should note that that example is overkill if you just want to do simple, blocking requests -- it includes a custom async executor and uses it to do a bunch of fancy concurrency. Someone could write a much simpler example, but I don't know of an existing one offhand.

Zeeshan Lakhani (Jan 25 2024 at 15:57):

@Joel Dice actually, I def was looking for something on the streaming side haha!

Lann Martin (Jan 25 2024 at 15:58):

Another (not particularly simple) example, integrating with hyper's types: https://github.com/lann/wasi-hyperium

Ralph (Jan 25 2024 at 16:08):

I think https://github.com/landonxjames/wasi-http-test/blob/main/src/generated-js-runtime-bindings/exports/wasm-version-tests-component-clients.d.ts also does this....

Gareth (Jan 26 2024 at 00:14):

Joel Dice said:

I should note that that example is overkill if you just want to do simple, blocking requests -- it includes a custom async executor and uses it to do a bunch of fancy concurrency. Someone could write a much simpler example, but I don't know of an existing one offhand.

Thanks for the advice, I had exactly the same question and was looking for a simple example of making an outgoing http request. I've tried running the example you suggested, api_proxy_streaming.rs using the latest version of wasmtime. Are you able to advise the wasmtime command I need to use to run this example?

Currently I'm getting the following error: 1: unknown import: wasi:http/types@0.2.0::[constructor]fields has not been defined , or if I run ./target/debug/wasmtime run --wasi http target/wasm32-wasi/release/api_proxy_streaming.wasm, I get an Error: Cannot enable wasi-http for core wasm modules.

I built wasmtime from source using cargo build --features component-model,wasi-http
Any help would be much appreciated as I am very new to running wasm modules with wasmtime.

Joel Dice (Jan 26 2024 at 00:14):

You'll need to use wasmtime serve instead of wasmtime run.

Joel Dice (Jan 26 2024 at 00:16):

e.g. ./target/debug/wasmtime serve --wasi common target/wasm32-wasi/release/api_proxy_streaming.wasm

Joel Dice (Jan 26 2024 at 00:19):

oh, and you'll probably need to convert the module to a component using wasm-tools component new prior to running it (or else use cargo-component to build from source, which will take care of the componentization step for you).

Gareth (Jan 26 2024 at 00:20):

thx for the quick reply, yes I tried the above and got an error, Error: The serve command currently requires a component

Joel Dice (Jan 26 2024 at 00:20):

Consider using https://github.com/sunfishcode/hello-wasi-http as a template, which takes care of the details.

Joel Dice (Jan 26 2024 at 00:21):

(i.e. clone that repo, replace the contents of the lib.rs with those of api_proxy_streaming.rs, and follow the README.md directions for building and running)

Gareth (Jan 26 2024 at 00:22):

great thx for your help, I'll give this a go

Joel Dice (Jan 26 2024 at 00:24):

Although I just noticed the README.md refers to Wasmtime 14, but recent commits refer to the rc-2023-12-05 snapshot of WASI, which came later. Anyway, you might need to grab the wit files from the same commit you built Wasmtime from and got api_proxy_streaming.rs from to make sure everything lines up.

Joel Dice (Jan 26 2024 at 00:25):

Sorry this is such a process. Now that WASI 0.2.0 is out, we won't need to juggle versions as much going forward.

Dan Gohman (Jan 26 2024 at 00:28):

I'm working up updating hello-wasi-http to Wasmtime 17 and WASI Preview 2. First step, update cargo-component.

Peter Huene (Jan 26 2024 at 00:29):

There will be a (breaking change) release of cargo-component this afternoon

Peter Huene (Jan 26 2024 at 00:29):

that will require a few minor tweaks to your Cargo.toml and main.rs/lib.rs (I'll have update instructions in the release announcement)

Joel Dice (Jan 26 2024 at 00:29):

somewhat relatedly, I'm planning to do the same for componentize-py early tomorrow

Gareth (Jan 28 2024 at 14:51):

Joel Dice said:

Although I just noticed the README.md refers to Wasmtime 14, but recent commits refer to the rc-2023-12-05 snapshot of WASI, which came later. Anyway, you might need to grab the wit files from the same commit you built Wasmtime from and got api_proxy_streaming.rs from to make sure everything lines up.

Thanks for your help, I tried to implement your suggestion using https://github.com/sunfishcode/hello-wasi-http as a template.

• I am using the latest version of wasmtime, 17.0.
• I added wit-bindgen = { version = "0.16.0", default-features = false, features = ["realloc"] } and crate-type = ["cdylib"] to my cargo.toml
• I've copied over http/wit/deps from wasmtime to make sure these are the latest version.

This worked well and I was able to make a http request when running as a server with:
./wasmtime/target/debug/wasmtime serve --wasi common try_http_outbound/target/wasm32wasi/debug/try_http_outbound.wasm

...however I would like to be able to run this it via the commandline using:
./wasmtime/target/debug/wasmtime run --wasi common try_http_outbound/target/wasm32-wasi/debug/try_http_outbound.wasm

I have amended my imports for lib.rs and am using https://github.com/bytecodealliance/wasmtime/blob/main/crates/test-programs/src/http.rs (this file is unchanged from the wasmtime repo) to make the outgoing http request. My code compiles successfully using cargo component build.
imports to my lib.rs are:

mod http;
use bindings::wasi::http::types::{Method, Scheme};
use bindings::exports::wasi::cli::run::Guest;

use anyhow::Context;
use serde_json::{Value, json};
use url::Url;

struct Component;
impl Guest for crate::Component {
    fn run() -> Result<(), ()> {
        main();
        Ok(())
    }
}

My world.wit file is as follows:

package sunfishcode:try-http-outbound;

world target-world {
  include wasi:cli/command@0.2.0;
  import wasi:http/outgoing-handler@0.2.0;
}

However when running:
./wasmtime/target/debug/wasmtime run --wasi common try_http_outbound/target/wasm32-wasi/debug/try_http_outbound.wasm
I get the following error :

Error: failed to run main module `try_http_outbound/target/wasm32-wasi/debug/try_http_outbound.wasm`
Caused by:
    0: import `wasi:http/types@0.2.0` has the wrong type
    1: instance export `fields` has the wrong type
    2: expected resource found nothing

I have checked and confirmed that all the wit files match those of my wasmtime version. Any help would be much appreciated as I’m not sure what I’ve missed here.

Joel Dice (Jan 28 2024 at 15:23):

Have you tried adding --wasi http to your wasmtime run command?

Dan Gohman (Jan 28 2024 at 16:33):

To run the hello-wasi-http example, using wasmtime serve rather than wasmtime run.

Gareth (Jan 28 2024 at 20:22):

Joel Dice said:

Have you tried adding --wasi http to your wasmtime run command?

amazing thx, this worked!

Joel Dice (Jan 28 2024 at 20:23):

For future reference, wasmtime run -S help will list all the WASI features you can enable or disable.

Dan Gohman (Feb 06 2024 at 00:26):

The hello-wasi-http demo repo is now updated to use Preview 2 and work with Wasmtime 17.

Dan Gohman (Feb 06 2024 at 00:44):

This uses the new --proxy flag to cargo component new, which tells cargo component to use a special adapter that avoids pulling in get-environment etc., so it works in wasmtime serve out of the box, without needing -S common.

Last updated: Dec 23 2024 at 12:05 UTC