Stream: git-wasmtime

Topic: wasmtime / Issue #939 Exporting dec_ciovec_slice and ciov...

Wasmtime GitHub notifications bot (Apr 06 2020 at 17:17):

jeffcharles commented on Issue #939:

Glad to see #701 has been merged. Unfortunately I can't seem to figure out how to use the new FileContents trait as a destination for stdout or stderr. The stdout and stderr methods on the builder only appear to take Files and not something that implements FileContents. As well, it's not clear to me if there's a way to wire up preopened_virt to provide destinations for stdout and stderr though I do see a guard in the code that they should not among the pre-opens so I'm guessing that won't work.

How do I map stdout and stderr to a custom implementation of FileContents?

Wasmtime GitHub notifications bot (Apr 20 2020 at 18:29):

jeffcharles commented on Issue #939:

Is there anything on the roadmap for supporting what I've proposed above? Another option is if we could get access to the pre-opened file descriptors in the WASI context, then we could wrap fd_write with something that passes through but with a different file descriptor (e.g., if we see a write for file descriptor 1, then pass it through to wasi-common as a write to file descriptor 4), but it doesn't look like that's exposed anywhere other than making a series of WASI calls.

Wasmtime GitHub notifications bot (Apr 20 2020 at 21:43):

pchickey commented on Issue #939:

Sorry for the delay. I think @kubkon is currently working on that aspect of the code. I haven't kept up with every change that has landed & is underway over there.

Wasmtime GitHub notifications bot (Apr 21 2020 at 15:04):

kubkon commented on Issue #939:

Hi @jeffcharles! I haven't read into your issue here in too much detail, but I think what you might be looking for, I'm just now trying to bake in into wasi-common. See #1561 for details. Oh, and just to sum up and check that I understand correctly, it all boils down for you to be able to attach your own virtualfs handle as stdio, is it? The PR I've mentioned should hopefully bring us a step closer to that at the very least.

Wasmtime GitHub notifications bot (Apr 21 2020 at 16:05):

jeffcharles deleted a comment on Issue #939:

it all boils down for you to be able to attach your own virtualfs handle as stdio, is it?

Almost, I want to attach our own virtual files to stdout and stderr. Attaching our own virtualfs to stdio would accomplish that. Really I just want some sort of interface that gives us byte arrays or io slices that we can operate on when an fd_write call occurs which the virtual file stuff does except for calls to file descriptors 1 or 2.

Wasmtime GitHub notifications bot (Apr 21 2020 at 16:09):

jeffcharles commented on Issue #939:

it all boils down for you to be able to attach your own virtualfs handle as stdio, is it?

Almost, I want to attach our own virtual files to stdout and stderr. Attaching our own virtualfs to stdio would accomplish that. Really I just want some sort of interface that gives us byte arrays or io slices that we can operate on when an fd_write call occurs which the virtual file stuff does except for calls to file descriptors 1 or 2.

Wasmtime GitHub notifications bot (Apr 21 2020 at 16:22):

jeffcharles commented on Issue #939:

it all boils down for you to be able to attach your own virtualfs handle as stdio, is it?

Almost, I want to attach our own virtual files to stdout and stderr. Attaching our own virtualfs to stdio would accomplish that. Really I just want some sort of interface that gives us byte arrays or io slices that we can operate on when an fd_write call occurs which the virtual file stuff does except for calls to file descriptors 1 or 2.

Wasmtime GitHub notifications bot (Apr 25 2020 at 07:42):

kubkon commented on Issue #939:

@jeffcharles Cool! Let me see what I can conjure up! If I have any questions, is it OK if I ping you now and then? Also, if it's any more convenient for you, we can sync up on [BA's zulip chat] as well. Let me know what works best!

[BA's zulip chat]: https://bytecodealliance.zulipchat.com/

Wasmtime GitHub notifications bot (Apr 26 2020 at 08:18):

kubkon commented on Issue #939:

@pchickey @jeffcharles ok, I've now created a draft PR (#1600) that hopefully addresses your requirements. With this PR, you should now be able to pass in any implementor of the Handle trait as a valid handle to stdio. Lemme know if that fixes it or if I (very likely!) misunderstood what the problem here actually is :-D

Wasmtime GitHub notifications bot (Apr 27 2020 at 17:12):

jeffcharles deleted a comment on Issue #939:

it all boils down for you to be able to attach your own virtualfs handle as stdio, is it?

Almost, I want to attach our own virtual files to stdout and stderr. Attaching our own virtualfs to stdio would accomplish that. Really I just want some sort of interface that gives us byte arrays or io slices that we can operate on when an fd_write call occurs which the virtual file stuff does except for calls to file descriptors 1 or 2.

Wasmtime GitHub notifications bot (Apr 27 2020 at 17:55):

jeffcharles commented on Issue #939:

It looks like that draft PR does. Just to confirm my understanding of how this would work as a consumer, I should be able to define a custom implementation for FileContents and pass that implementation in a call to InMemoryFile::new and then specify that InMemoryFile I get back from that call to stdout and stderr on WasiCtxBuilder which will result in my custom FileContents implementation's pwrite or pwritev's methods being invoked when a WASI fd_write hostcall occurs?

Wasmtime GitHub notifications bot (Apr 27 2020 at 19:01):

kubkon commented on Issue #939:

If I read it right, then yeah, that's the basic idea. In fact, in that draft PR, anything that conforms to the Handle trait will do the job, and in this case, specifying your own implementation of FileContents and then injecting that into InMemoryFile which you then add to the context as a relevant stdio handle should do the job:

let virt_stdout = InMemoryFile::new(Box::new(custom_file_contents));
let ctx: WasiCtx = WasiCtxBuilder::new().stdout(in_mem).build()?;

While we're here, you'll note that you have to provide an instance that implements the Handle trait which effectively should be an internal trait of wasi-common. In the future, we'd like to make it completely independent of the underlying runtime and WASI implementation. @sunfishcode has drafted out some really good ideas about this, where the dynamic dispatch problem of this type would be handled at a layer one above the WASI implementation. This essentially implies that as a consumer of WASI you won't have to worry about implementing any traits specific to the runtime at hand. I hope I got the basic idea right. @sunfishcode could you confirm/clarify? :-)

Wasmtime GitHub notifications bot (Apr 27 2020 at 19:02):

kubkon edited a comment on Issue #939:

If I read it right, then yeah, that's the basic idea. In fact, in that draft PR, anything that conforms to the Handle trait will do the job, and in this case, specifying your own implementation of FileContents and then injecting that into InMemoryFile which you then add to the context as a relevant stdio handle should do the job:

let virt_stdout = InMemoryFile::new(Box::new(custom_file_contents));
let ctx: WasiCtx = WasiCtxBuilder::new().stdout(virt_stdout).build()?;

While we're here, you'll note that you have to provide an instance that implements the Handle trait which effectively should be an internal trait of wasi-common. In the future, we'd like to make it completely independent of the underlying runtime and WASI implementation. @sunfishcode has drafted out some really good ideas about this, where the dynamic dispatch problem of this type would be handled at a layer one above the WASI implementation. This essentially implies that as a consumer of WASI you won't have to worry about implementing any traits specific to the runtime at hand. I hope I got the basic idea right. @sunfishcode could you confirm/clarify? :-)

Wasmtime GitHub notifications bot (Jun 09 2020 at 18:19):

kubkon closed Issue #939:

I have a use-case where I'd like to perform some operations on the data being sent through an fd_write call on the host rather than sending it through to an underlying file descriptor. The existing fd_write call doesn't appear to offer the ability to work with the input data on the host.

What I think I want (and feel free to correct me if it isn't) is something that would convert the scatter gather IO vectors passed into fd_write to a u8 slice. I think combining dec_ciovec_slice and ciovec_to_host and flattening the resulting IO slices _should_ do that. Would there be any interest in either exporting these functions or exporting a function that would use these two functions to return a slice of the data provided as input to fd_write?

Last updated: Dec 23 2024 at 12:05 UTC