Stream: git-wasmtime

Topic: wasmtime / issue #7260 Add an asynchronous `wasi:io/outpu...

Wasmtime GitHub notifications bot (Oct 16 2023 at 23:30):

pchickey commented on issue #7260:

More feedback, based on discussion with @elliottt:

• Do we need to express an output-stream that can potentially have an unbounded number of input-streams forwarded to it? It would perhaps be much simpler if forward consumed both the input-stream and output-stream, and future-forward-result gave back the output-stream as well as the u64 in the some(ok(ok(...)) case.
• I am concerned this could be too powerful, and force us to make some difficult implementation choices, when applied to all streams. We have the use case of forwarding http bodies. Could we instead implement this as a static method on outgoing-body as an alternative to finish which is something like finish-by-forwarding: static func(this: outgoing-body, forward: input-stream) -> some-future. That still requires us to keep that input-stream alive beyond the lifetime of the wasm execution, which could be the same can of worms...

Wasmtime GitHub notifications bot (Oct 16 2023 at 23:33):

pchickey edited a comment on issue #7260:

More feedback, based on discussion with @elliottt:

• Do we need to express an output-stream that can potentially have an unbounded number of input-streams forwarded to it? It would perhaps be much simpler if forward consumed both the input-stream and output-stream, and future-forward-result gave back the output-stream as well as the u64 in the some(ok(ok(...)) case.
• I am concerned this could be too powerful, and force us to make some difficult implementation choices, when applied to all streams. We have the use case of forwarding http bodies. Could we instead implement this as a static method on outgoing-body as an alternative to finish which is something like finish-by-forwarding: static func(this: outgoing-body, forward: input-stream) -> some-future. That still requires us to keep that input-stream alive beyond the lifetime of the wasm execution, which could be the same can of worms... for example, what happens when that input-stream is derived from a socket, or a file. now we need to be able to keep those open beyond the lifetime of wasm execution... we'd basically have to keep the entire Table alive (or have some sort of fancy garbage collection) until all forwarding is over, so all we'd be able to free during forward is the linear memory.

Wasmtime GitHub notifications bot (Oct 16 2023 at 23:35):

pchickey edited a comment on issue #7260:

More feedback, based on discussion with @elliottt:

• Do we need to express an output-stream that can potentially have an unbounded number of input-streams forwarded to it? It would perhaps be much simpler if forward consumed both the input-stream and output-stream, and future-forward-result gave back the output-stream as well as the u64 in the some(ok(ok(...)) case.
• I am concerned this could be too powerful, and force us to make some difficult implementation choices, when applied to all streams. We have the use case of forwarding http bodies. Could we instead implement this as a static method on outgoing-body as an alternative to finish which is something like finish-by-forwarding: static func(this: outgoing-body, forward: input-stream) -> some-future. That still requires us to keep that input-stream alive beyond the lifetime of the wasm execution, which could be the same can of worms... for example, what happens when that input-stream is derived from a socket, or a file. now we need to be able to keep those open beyond the lifetime of wasm execution... we'd basically have to keep the entire Table alive (or have some sort of fancy garbage collection) until all forwarding is over, so all we'd be able to free during forward is the linear memory.

This was the can of worms I was trying to avoid opening, and I'm worried about painting ourselves into a corner by requiring certain behavior happen in the background and beyond the lifetime of the component.

Wasmtime GitHub notifications bot (Oct 16 2023 at 23:37):

pchickey edited a comment on issue #7260:

More feedback, based on discussion with @elliottt:

• Do we need to express an output-stream that can potentially have an unbounded number of input-streams forwarded to it? It would perhaps be much simpler if forward consumed both the input-stream and output-stream, and future-forward-result gave back the output-stream as well as the u64 in the some(ok(ok(...)) case.
• I am concerned this could be too powerful, and force us to make some difficult implementation choices, when applied to all streams. We have the use case of forwarding http bodies. Could we instead implement this as a static method on outgoing-body as an alternative to finish which is something like finish-by-forwarding: static func(this: outgoing-body, forward: input-stream) -> some-future. That still requires us to keep that input-stream alive beyond the lifetime of the wasm execution, which could be the same can of worms... for example, what happens when that input-stream is derived from a socket, or a file. now we need to be able to keep those open beyond the lifetime of wasm execution... we'd basically have to keep the entire Table alive (or have some sort of fancy garbage collection) until all forwarding is over, so all we'd be able to free during forward is the linear memory.

This was the can of worms I was trying to avoid opening, and I'm worried about painting ourselves into a corner by requiring certain behavior happen in the background and beyond the lifetime of the component. Could we scope this down to an interaction that only happens as long as the future-forwarded-result is alive, and not try to return from execution until forwarding has complete, just waiting in a poll_list until the forwarding is complete? That will save on lots of back and forth interaction between host and guest, at least.

Wasmtime GitHub notifications bot (Nov 09 2023 at 18:51):

alexcrichton commented on issue #7260:

I believe the current conclusion is to defer this to after the 0.2.0 release of WASI, so I'm going to close this as it's probably best to pick back up discussion after that happens, but please reopen if you disagree

Last updated: Nov 22 2024 at 16:03 UTC