Stream: wasmtime

Topic: Slow file reads due to small buffer sizes?

Justin Fagnani (Oct 13 2026 at 17:12):

I'm working on streaming file contents read from wasi:fs and passed out to wasi:http and I noticed that the throughput was pretty slow even though by forwarding the WASI stream, my guest code shouldn't be in the byte-serving path. I was getting about 261 MB/s on my server vs 3.4 GB/s for Python serving a random 64 MB file.

It seems like the issue might be the 8k DEFAULT_BUFFER_CAPACITY. I had an agent run a benchmark against different values, and got these results:

There's not currently a way for an embedder to directly change the buffer size, but you can accomplish that by overriding read-via-stream (as an agent did for me here: https://github.com/elematic/zena/pull/324).

Would it be worth a way to allow an embedder to easily change this, and/or increasing the default value?

I'd also be interesting in a way to set buffer sizes from guest code, but that would be a WASI question.

fitzgen (he/him) (Oct 13 2026 at 17:45):

I would be pretty skeptical of letting a guest set its own buffer sizes, as the guest is inherently untrusted

that said, I don't see any reason not to the embedder to customize this, but also the WASI implementation is the area of the codebase that I am least familiar with. perhaps @Pat Hickey could say more

Pat Hickey (Oct 13 2026 at 17:48):

Thats the p3 implementation which I'm actually not familiar with, will have to pass that off to @Roman Volosatovs or @Adam Bratschi-Kaye (he)

Pat Hickey (Oct 13 2026 at 17:49):

we have made some attempt to make those buffer limits settable by the embedder elsewhere, but we may have done an imperfect job, and in general we are open to letting the host set that

Pat Hickey (Oct 13 2026 at 17:49):

basically the trade off is how much host space a guest can consume, which can be a DoS vector, so theres not really a one size fits all

Justin Fagnani (Oct 13 2026 at 18:08):

Could the guest possibly supply a hint? The guest might have some idea of the type of workload it's dealing with. I settled on 256 KB for my embedder, but I don't have reason to think that's a universally good setting.

Pat Hickey (Oct 13 2026 at 18:26):

the host can permit up to any size, and the guest can opt into using as much as it wants to, so idk what a hint could do here

Justin Fagnani (Oct 13 2026 at 18:42):

The guest can't request a specific buffer size yet, so that would be the hint. It'd be a hint if the host doesn't promise to use that size.

Justin Fagnani (Oct 13 2026 at 18:48):

Something like:

read-via-stream: func(offset: filesize, buffer-size: option<u32>) -> tuple<stream<u8>, future<result<_, error-code>>>;

Adam Bratschi-Kaye (he) (Oct 13 2026 at 19:06):

That would need to be proposed as a change to the WASI interface which would be trickier. It seems to me that it's more natural for the embedder to control the buffer size anyway, especially because it can be aware of what else is going on in the system and how the stream is being read. For your use case, why is guest configuration preferable to embedder configuration?

Justin Fagnani (Oct 13 2026 at 19:13):

Because the guest might know the size of files it's typically dealing in. You don't need a 1MB buffer for small files.

Adam Bratschi-Kaye (he) (Oct 13 2026 at 19:14):

True, but the host also knows the actual file size, right?

Till Schneidereit (Oct 13 2026 at 20:33):

fwiw I also think that we should do something meaningfully larger than 8kb here: I've run into this as a meaningful perf issue as well.

Ideally we'd let the guest set buffer sizes, but make it easy to enforce overall memory limits that capture host allocations together with guest memory + table sizes and all that

Till Schneidereit (Oct 13 2026 at 20:34):

I agree though that auto-sizing buffers on the host works in many cases, including files, but also http bodies if content-length is set

Justin Fagnani (Oct 14 2026 at 14:49):

Should I file an issue, and can I make a starter PR to make DEFAULT_BUFFER_CAPACITY override-able by embedders?

Roman Volosatovs (Oct 16 2026 at 17:13):

Making DEFAULT_BUFFER_CAPACITY configurable by embedders definitely makes sense to me, and I think it would be the lowest-hanging fruit providing the biggest immediate benefit.

If I understand this correctly, we're in the host<->host case, where the wasi:filesystem producer is streamed to a wasi:http consumer. If the other side were the guest, the host would know the guest's buffer capacity and could do what's optimal, however in this case the buffer is preallocated using DEFAULT_BUFFER_CAPACITY, so ultimately I think we want the consumer to be able to say how many elements it needs, rather than having the producer use an arbitrary value. In some sense I think we want this to behave more like https://docs.rs/tokio/latest/tokio/io/trait.AsyncRead.html

In other words, we'd need https://github.com/bytecodealliance/wasmtime/blob/3c4c9651687c4c9abd1ad6333a1b0416c0ac76fc/crates/wasmtime/src/runtime/component/concurrent/futures_and_streams.rs#L399-L417 to return a value for host consumers as well, although I'm not sure we'd want to structure the API that way

Long term, I think we want to make DEFAULT_BUFFER_CAPACITY redundant altogether in this case (although we'd still want a configurable cap)

IIRC we actually discussed this a while back with @Joel Dice

Joel Dice (Oct 19 2026 at 13:49):

Yeah, communicating the consumer's capacity to the producer for host-to-host streaming seems reasonable to me.

Last updated: Oct 31 2026 at 23:03 UTC