Stream: git-wasmtime

Topic: wasmtime / issue #7530 Move Subscribe trait's static life...

Wasmtime GitHub notifications bot (Nov 13 2023 at 16:30):

alexcrichton commented on issue #7530:

Thanks for the PR! Could you confirm that this works for your app though? There's currently no way for a non-'static Subscribe implementation to work with Table, so while this may fix some errors it may also shift where the error is happening and still not fully resolve the issue.

Wasmtime GitHub notifications bot (Nov 13 2023 at 16:57):

langyo commented on issue #7530:

Thanks for the PR! Could you confirm that this works for your app though? There's currently no way for a non-'static Subscribe implementation to work with Table, so while this may fix some errors it may also shift where the error is happening and still not fully resolve the issue.

Yes, as you stated, this fix alone doesn't actually seem to completely solve the problem...

However, I feel that even if this modification does not completely solve my problem, I still think this modification is beneficial. It seems not very elegant to directly bind a fixed static life cycle to the trait.

Wasmtime GitHub notifications bot (Nov 13 2023 at 17:01):

alexcrichton commented on issue #7530:

Ok makes sense. Elegance is of course always subjective, but one purpose of the bounds on the trait, as you're finding out, is that even if removed it still won't work. By placing it on the trait itself it weeds out these sorts of errors sooner and additionally avoids the need to restate the 'static bound at callsites, as this PR is required to do so

Wasmtime GitHub notifications bot (Nov 13 2023 at 17:13):

langyo commented on issue #7530:

Ok makes sense. Elegance is of course always subjective, but one purpose of the bounds on the trait, as you're finding out, is that even if removed it still won't work. By placing it on the trait itself it weeds out these sorts of errors sooner and additionally avoids the need to restate the bound at callsites, as this PR is required to do so'static

In terms of quickly locating the wrong location, you are right...

It's time for me to try to find other ideas to solve the problem. :(

Wasmtime GitHub notifications bot (Nov 13 2023 at 17:39):

alexcrichton commented on issue #7530:

Could you detail a bit more what you're doing with non-'static data? I might be able to try to help out with a possible solution

Wasmtime GitHub notifications bot (Nov 14 2023 at 01:25):

langyo commented on issue #7530:

Could you detail a bit more what you're doing with non-'static data? I might be able to try to help out with a possible solution

Thank you very much. I can just summarize the questions I have accumulated about wasmtime in the recent period.

I am building a server-side container facility based on wasmtime and some routing libraries, such as axum. I packaged a virtual machine container library to support the real-time creation and destruction of WASI containers, which is somewhat similar to docker's approach.

The wasmtime WASI support is not the most sound, because of the rust compiler's stable target wasm-wasi, only support the preview1 ABI. So I only used stdio, a stable feature that can adapt to all ABIs at the same time, for internal and external communication.

Since stdio is used for communication, it is necessary to transfer text streams to each other, just like different CLIs under Linux interact with each other using text streams.

I tried to take over the stdio of wasmtime and parse and generate messages by myself. At this time, I encountered the problem I encountered this time. I can't deserialize the data correctly with serde the moment I receive the message because I'm running into a lifecycle conflict while converting...

serde will inevitably need to apply for additional heap memory when deserializing, and rust believes that this batch of heap memory may cause dangling references, and then I got stuck here...

By the way, I have actually considered many other solutions, such as using WIT to generate fixed ABI bindings,or using the network communication capabilities privately provided by wasmer for internal and external communication. But they are all worse than the current solution of using stdio for plain text communication. The former required to implement additional heap memory maintained by the browser's JavaScript environment like web-sys privately, while the latter is a feature that is not written into the standard. I dare not use it and will probably be replaced in the future.

Wasmtime GitHub notifications bot (Nov 14 2023 at 01:26):

langyo edited a comment on issue #7530:

Could you detail a bit more what you're doing with non-'static data? I might be able to try to help out with a possible solution

Thank you very much. I can just summarize the questions I have accumulated about wasmtime in the recent period.

I am building a server-side container facility based on wasmtime and some routing libraries, such as axum. I packaged a virtual machine container library to support the real-time creation and destruction of WASI containers, which is somewhat similar to docker's approach.

The wasmtime WASI support is not the most sound, because of the rust compiler's stable target wasm-wasi only support the preview1 ABI. So I only used stdio, a stable feature that can adapt to all ABIs at the same time, for internal and external communication.

Since stdio is used for communication, it is necessary to transfer text streams to each other, just like different CLIs under Linux interact with each other using text streams.

I tried to take over the stdio of wasmtime and parse and generate messages by myself. At this time, I encountered the problem I encountered this time. I can't deserialize the data correctly with serde the moment I receive the message because I'm running into a lifecycle conflict while converting...

serde will inevitably need to apply for additional heap memory when deserializing, and rust believes that this batch of heap memory may cause dangling references, and then I got stuck here...

By the way, I have actually considered many other solutions, such as using WIT to generate fixed ABI bindings,or using the network communication capabilities privately provided by wasmer for internal and external communication. But they are all worse than the current solution of using stdio for plain text communication. The former required to implement additional heap memory maintained by the browser's JavaScript environment like web-sys privately, while the latter is a feature that is not written into the standard. I dare not use it and will probably be replaced in the future.

Wasmtime GitHub notifications bot (Nov 14 2023 at 05:44):

pchickey commented on issue #7530:

If you are just trying to direct stdio around you should only need to provide impls of HostInputStream and HostOutputStream, and never worry about impling Subscribe yourself. You will also will require an additional small wrapper for StdinStream and StdoutStream which exists because Preview 2 programs get to create many (effectively unbounded, in reality always a small number) resources that all point to the same stdin / stdout / stderr. Pass those to your WasiCtxBuilder and it should all just work from there. https://docs.rs/wasmtime-wasi/latest/wasmtime_wasi/preview2/struct.WasiCtxBuilder.html#method.stdin

Wasmtime GitHub notifications bot (Nov 14 2023 at 06:32):

langyo commented on issue #7530:

If you are just trying to direct stdio around you should only need to provide impls of HostInputStream and HostOutputStream, and never worry about impling Subscribe yourself. You will also will require an additional small wrapper for StdinStream and StdoutStream which exists because Preview 2 programs get to create many (effectively unbounded, in reality always a small number) resources that all point to the same stdin / stdout / stderr. Pass those to your WasiCtxBuilder and it should all just work from there. https://docs.rs/wasmtime-wasi/latest/wasmtime_wasi/preview2/struct.WasiCtxBuilder.html#method.stdin

I found the default impl from here, but it wasn't working while I have returned Box<dyn HostXXputStream> on these codes...

The other problem is, the parameters for the methods .stdin(), .stdout() and .stderr() not only use impl StdxxxStream trait but also needs 'static tag.

I found that the problem seems to be more than just removing the 'static tag for a Subscriber...XD

Wasmtime GitHub notifications bot (Nov 14 2023 at 15:47):

alexcrichton commented on issue #7530:

It sounds like the source of the lifetime issues is coming from serde? Would it be possible to avoid the use of lifetimes there and use types like String instead of &str?

Wasmtime GitHub notifications bot (Nov 14 2023 at 15:53):

langyo commented on issue #7530:

It sounds like the source of the lifetime issues is coming from serde? Would it be possible to avoid the use of lifetimes there and use types like String instead of &str?

There is nothing wrong with this idea. In fact, I am already improving the previous code with this idea. The disadvantage is that I have to write additional code for parsing. :D

I don’t have any questions for the time being. I would close this PR now, and you can leave a message at any time if you have any additional ideas.

Last updated: Nov 22 2024 at 16:03 UTC