tbrockman opened PR #11843 from ezdevlol:req_into_http to bytecodealliance:main:
Discussion: #wasi > wasi-http Request and pub(crate) Body @ 💬
Largely copies the implementation from
Response::into_httpas suggested, contains one basic test asserting it works in the happy path.
tbrockman requested wasmtime-wasi-reviewers for a review on PR #11843.
dicej submitted PR review:
Thanks for doing this; LGTM.
Please run
cargo fmtwhen you have a chance, which should address the CI failure.
alexcrichton commented on PR #11843:
Would it also be possible to update the
default_send_requestto use this new code path as well? I would naively expect the two to be pretty similar in what they're doing.
tbrockman updated PR #11843.
tbrockman commented on PR #11843:
Thanks for doing this; LGTM.
Please run cargo fmt when you have a chance, which should address the CI failure.
No problem! Seems a fair exchange for the past/future questions I'll have about
wasmtime:grimacing:Would it also be possible to update the default_send_request to use this new code path as well? I would naively expect the two to be pretty similar in what they're doing.
I looked into this, but I think I might not quite understand the ask as the
reqparameter already appears to be anhttp::Request?
tbrockman edited a comment on PR #11843:
Thanks for doing this; LGTM.
Please run cargo fmt when you have a chance, which should address the CI failure.
No problem! Seems a fair exchange for the past/future questions I'll have about
wasmtime:grimacing:Would it also be possible to update the default_send_request to use this new code path as well? I would naively expect the two to be pretty similar in what they're doing.
I looked into this, but I think I might not quite understand the ask as the
reqparameter todefault_send_requestalready appears to be anhttp::Request?
tbrockman edited a comment on PR #11843:
Thanks for doing this; LGTM.
Please run cargo fmt when you have a chance, which should address the CI failure.
No problem! Seems a fair exchange for the past/future questions I'll have about
wasmtime:grimacing:Would it also be possible to update the default_send_request to use this new code path as well? I would naively expect the two to be pretty similar in what they're doing.
I looked into this, but I think I might not quite understand the ask as the
reqparameter todefault_send_requestalready appears to be anhttp::Request?Edit: Ah, guessing you meant this code which eventually calls
default_send_request, which does indeed have some overlap here.
tbrockman edited a comment on PR #11843:
Thanks for doing this; LGTM.
Please run cargo fmt when you have a chance, which should address the CI failure.
No problem! Seems a fair exchange for the past/future questions I'll have about
wasmtime:grimacing:Would it also be possible to update the default_send_request to use this new code path as well? I would naively expect the two to be pretty similar in what they're doing.
I looked into this, but I think I might not quite understand the ask as the
reqparameter todefault_send_requestalready appears to be anhttp::Request?Edit: Ah, guessing you meant this code which eventually calls
default_send_request, which does indeed have some overlap here (if you can confirm that this is what you meant I can dig more into it).
alexcrichton commented on PR #11843:
Ah yes indeed! That eventually bottoms out here and I think yeah you're right that what you've added here is more suitable for replacing/augmenting the implementation of
handleas opposed todefault_send_requestitself (which is already working at the abstraction level of Hyper types)
tbrockman commented on PR #11843:
@alexcrichton So I looked into it a bit, and given my limited understanding of
handleand how it uses various channels (and perhaps channels, pinning, and tasks in Rust in general), it's not immediately straightforward to me how to integrateRequest::into_httpin a way that doesn't require adding too much complexity toRequest::into_http, without prompting workarounds/redundant code to preserve existing functionality inhandle:thinking:Might need a bit of hand holding here :sweat_smile:
alexcrichton commented on PR #11843:
Ah ok no worries, I can try to take a look after this merges
tbrockman commented on PR #11843:
Looks like it leaked some memory during testing as well? :disappointed:
================================================================= ==18584==ERROR: LeakSanitizer: detected memory leaks Indirect leak of 72 byte(s) in 1 object(s) allocated from: #0 0x563417655224 (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x14cb224) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #1 0x563417756761 (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x15cc761) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #2 0x5634176fc8f9 (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x15728f9) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #3 0x5634177a923d (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x161f23d) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #4 0x5634176ae4da (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x15244da) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #5 0x56341771965c (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x158f65c) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #6 0x56341775d61d (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x15d361d) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #7 0x5634176ad96d (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x152396d) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #8 0x5634177a894d (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x161e94d) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #9 0x5634177c1c98 (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x1637c98) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #10 0x5634177ffef4 (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x1675ef4) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #11 0x5634177d68e3 (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x164c8e3) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #12 0x5634177da219 (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x1650219) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #13 0x563419c47c6e (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x3abdc6e) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #14 0x563417652b96 (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x14c8b96) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) Indirect leak of 32 byte(s) in 1 object(s) allocated from: #0 0x563417655224 (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x14cb224) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #1 0x5634176fdb34 (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x1573b34) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #2 0x5634177a95f7 (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x161f5f7) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #3 0x5634176ae4da (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x15244da) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #4 0x56341771965c (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x158f65c) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #5 0x56341775d61d (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x15d361d) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #6 0x5634176ad96d (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x152396d) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #7 0x5634177a894d (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x161e94d) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #8 0x5634177c1c98 (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x1637c98) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #9 0x5634177ffef4 (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x1675ef4) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #10 0x5634177d68e3 (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x164c8e3) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #11 0x5634177da219 (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x1650219) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #12 0x563419c47c6e (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x3abdc6e) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) #13 0x563417652b96 (/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521+0x14c8b96) (BuildId: ebb66c38ee1e4fe09240bde1dc91b9090cdf87d4) SUMMARY: AddressSanitizer: 104 byte(s) leaked in 2 allocation(s). error: test failed, to rerun pass `-p wasmtime-wasi-http --lib` Caused by: process didn't exit successfully: `/home/runner/work/wasmtime/wasmtime/target/x86_64-unknown-linux-gnu/debug/deps/wasmtime_wasi_http-e2c9f65d23950521` (exit status: 1) note: test exited abnormally; to see the full output pass --no-capture to the harness. Error: Process completed with exit code 1.
rvolosatovs created PR review comment:
This should either error or just panic, see https://github.com/bytecodealliance/wasmtime/blob/ab2486b449d9b8377ffaa2e7a224da390c94a810/crates/wasi-http/src/p3/host/handler.rs#L277-L294
rvolosatovs submitted PR review.
rvolosatovs submitted PR review.
rvolosatovs created PR review comment:
I'm not sure we can actually implement such conversion even without "unwrapping" the body, since e.g.
schememay end up unset, while thewasi-httpimplementation defaults toctx.default_scheme(). See alsowasi-httpdocs https://github.com/WebAssembly/wasi-http/blob/14a19b388f4828604a08ede2af8ee1285c020113/wit-0.3.0-draft/types.wit#L278-L284
rvolosatovs created PR review comment:
We should also test this conversion with scheme unset
rvolosatovs created PR review comment:
Like I pointed out above, I'd expect the scheme to be set here using the context, like in the
handleimplementation
rvolosatovs submitted PR review.
rvolosatovs created PR review comment:
nit; we've been using this pattern in the p3 implementation so far
.map_err(|x| match x {})
rvolosatovs submitted PR review.
rvolosatovs created PR review comment:
let http_req = req.into_http(&mut store, async { Ok(()) } ).unwrap();I don't think passing
futyou received from the implementation here is valid, in fact that may be the cause for the memory leak
rvolosatovs submitted PR review.
rvolosatovs created PR review comment:
After consuming the body, we should probably also verify that
futresolves to success by polling it once using https://doc.rust-lang.org/std/task/struct.Waker.html#method.noop
rvolosatovs edited PR review comment.
alexcrichton commented on PR #11843:
As for the leak, it looks like that's caused by the test added here so I don't think it's spurious. The error message isn't great, however, but if you try running locally it might provide a better error message when
llvm-symbolizeris in your$PATH
tbrockman commented on PR #11843:
Thanks for the detailed review and guidance @rvolosatovs! I'll incorporate the changes when I get a chance.
tbrockman updated PR #11843.
tbrockman updated PR #11843.
tbrockman updated PR #11843.
rvolosatovs submitted PR review.
rvolosatovs created PR review comment:
why not use
Emptyinstead? https://docs.rs/http-body-util/latest/http_body_util/struct.Empty.html
rvolosatovs created PR review comment:
If I understand correctly,
into_httpfails before it converts the body, which causes this - that makes sense to me, but I think converting the body should be one of the very first things the function should do, which would maybe fix the issue? (see also comments above as to why the order matters)
rvolosatovs created PR review comment:
If this would error, it would leak memory with no way for the user to fix that.
I still don't think we should implementTryFromfor this conversion
rvolosatovs created PR review comment:
All of these error cases could leak, since the body handles may not be disposed of correctly, in
handleimplementation we prevent that by careful ordering of operations - first converting the body (ensuring it will be correctly disposed of onDrop) and then constructing the request https://github.com/bytecodealliance/wasmtime/blob/6b156f234249eb697b6749f369bd36fb5d219d5b/crates/wasi-http/src/p3/host/handler.rs#L219-L299
rvolosatovs submitted PR review.
rvolosatovs created PR review comment:
refs https://github.com/bytecodealliance/wasmtime/pull/11440#discussion_r2326139381
tbrockman submitted PR review.
tbrockman created PR review comment:
Woops! Think I forgot to remove this (or failed to commit it), it's no longer called, wasn't insisting on keeping it.
tbrockman submitted PR review.
tbrockman created PR review comment:
Thanks for the context (and direct link to the PR discussion!), that makes sense, I'll revert the changes in ordering.
tbrockman submitted PR review.
tbrockman created PR review comment:
First, some context:
So (apologies if this explanation is unnecessary) this is due to the one section that did mirror what
HostWithStore::handle<T>currently does, which returns an error if the content length is invalid first (which is used to determine whether to produce aLimited/UnlimitedGuestBodyConsumer), which is then matched and used to explicitly drop the body:This drops the sender side of the channel, which results in the
elseclause here to be evaluated:Then, the question:
Is this how we want things to be handled? A valid content length header is not strictly necessary (so an error could result in setting
content_length = None, and the rest of the code could happily proceed), but it does also make sense that creating aRequestwith an invalidContent-Lengthheader would result in an error, but if so I think I would expect to learn of that error infut...that is, if there weren't also this docstring onRequest::new:
tbrockman edited PR review comment.
tbrockman edited PR review comment.
tbrockman edited PR review comment.
tbrockman edited PR review comment.
tbrockman updated PR #11843.
rvolosatovs submitted PR review.
rvolosatovs created PR review comment:
I don't think this is valid. The
Request::newassumes that the host has performed the validation if the request is of foreign origin. For example, if request was received usinghyper, it would have to be a valid HTTP request (e.g. with compliantcontent-length)This is not a valid
content-lengthheader value though, soRequest::newshould not be called with it
tbrockman updated PR #11843.
tbrockman edited PR review comment.
tbrockman submitted PR review.
tbrockman created PR review comment:
Ah okay, changed the error scenario to something which should throw on building the URI instead.
rvolosatovs submitted PR review.
rvolosatovs created PR review comment:
IIRC GET requests cannot carry a body by the spec
tbrockman updated PR #11843.
tbrockman submitted PR review.
tbrockman created PR review comment:
Changed to POST (though I think per the spec sending a body with GET just doesn't have defined semantics), also rebased and included changes to use
boxed_unsync(which were failing tests).
tbrockman edited PR review comment.
tbrockman edited PR review comment.
tbrockman updated PR #11843.
tbrockman edited PR review comment.
rvolosatovs submitted PR review.
rvolosatovs has enabled auto merge for PR #11843.
rvolosatovs merged PR #11843.
alexcrichton commented on PR #11843:
Question for y'all: the new method here looks to more-or-less be a copy/paste of this functionality. Would it be possible to unify the two, and if not, how come? The handling of
Body::Hostfor example is quite different between the two and I naively am not entirely sure why.
rvolosatovs commented on PR #11843:
Question for y'all: the new method here looks to more-or-less be a copy/paste of this functionality. Would it be possible to unify the two, and if not, how come? The handling of
Body::Hostfor example is quite different between the two and I naively am not entirely sure why.I'm working on a PR, which will align the two
rvolosatovs commented on PR #11843:
alexcrichton commented on PR #11843:
Makes sense, thanks!
Last updated: Dec 06 2025 at 06:05 UTC