Stream: git-wasmtime

Topic: wasmtime / Issue #1987 Question: current status and the f...

view this post on Zulip Wasmtime GitHub notifications bot (Jul 07 2020 at 08:04):

daubaris opened Issue #1987:

Hi, could someone explain the current status of lightbeam? Since wasmtime offers multiple compilation strategies I think it currently lacks some information. Lightbeam is a bit of an unknown area, at least at a first glance. For the past few months most of the commits were regarding the updates of dependencies. I was wondering if it is a slowly dying part of the project and will it be removed in the future or if there is still a plan to develop it further?

There is also this TODO comment about supporting streams in jit crate’s compiler.rs file. According to the git blame the comment is about two years old. This raises a few questions. Does this mean that even though lightbeam is said to be streaming, the feature is not exploited by the runtime? Similarly, I am unsure how come lightbeam is streaming if the binary needs to be loaded all at once by the translate method. In addition, the description in the documentation is a bit inconsistent as README.md in the lightbeam crate describes it as "... an optimising one-pass streaming compiler", whereas in docs.rs it's only "a single-pass code generator". Please correct me if I am wrong, I might be missing some important details! There are also quite a few unimplemented! cases in the lightbeam’s module.rs file which kind of makes me wonder how crucial are these for a module to be compiled?

To sum up, these are the questions that might clear the confusion:

view this post on Zulip Wasmtime GitHub notifications bot (Jul 07 2020 at 09:29):

bjorn3 commented on Issue #1987:

How come lightbeam is streaming? As I understand, translate method accepts the whole binary and only then it is passed to the translate_only method which reads it section by section?

The "one-pass streaming compiler" refers to the fact that translate scans through the given wasm file linearly without looking back at any point. It also doesn't build any ir that doesn't get linearly scanned through itself. Lightbeam reads the wasm code and turns it into its own "microwasm" ir. At any point it looks at at most the last two wasm instructions for this. This "microwasm" is then immediately turned into machine code, again without looking at more than the last few "microwasm" instructions. (I think just the last one) At no point does it need to have more than a small slice of the wasm file in memory. That FIXME refers to the fact that the current user interface doesn't expose this streaming behavior.

Does wasmtime support streaming at all?

Not right now. The Cranelift backend doesn't support it by definition, while the Lightbeam backend does technically support it, but doesn't expose it.

The present state of lightbeam?

I think most of the WASM MVP works, but I am not sure.

Any plans for future development?

I have no clue.

view this post on Zulip Wasmtime GitHub notifications bot (Jul 07 2020 at 09:51):

Robbepop commented on Issue #1987:

From what I know as @bjorn3 already said the Lightbeam project technically supports streaming compilation. However, the wasmparser crate which it uses to parse the incoming Wasm file does not. There have been some attempts by the Lightbeam team to experiment with streaming parsing that can be found here: https://github.com/Vurich/wasm-reader
If you want to contribute or read more about the streaming parsing for wasmparser:

view this post on Zulip Wasmtime GitHub notifications bot (Jul 08 2020 at 09:03):

daubaris closed Issue #1987:

Hi, could someone explain the current status of lightbeam? Since wasmtime offers multiple compilation strategies I think it currently lacks some information. Lightbeam is a bit of an unknown area, at least at a first glance. For the past few months most of the commits were regarding the updates of dependencies. I was wondering if it is a slowly dying part of the project and will it be removed in the future or if there is still a plan to develop it further?

There is also this TODO comment about supporting streams in jit crate’s compiler.rs file. According to the git blame the comment is about two years old. This raises a few questions. Does this mean that even though lightbeam is said to be streaming, the feature is not exploited by the runtime? Similarly, I am unsure how come lightbeam is streaming if the binary needs to be loaded all at once by the translate method. In addition, the description in the documentation is a bit inconsistent as README.md in the lightbeam crate describes it as "... an optimising one-pass streaming compiler", whereas in docs.rs it's only "a single-pass code generator". Please correct me if I am wrong, I might be missing some important details! There are also quite a few unimplemented! cases in the lightbeam’s module.rs file which kind of makes me wonder how crucial are these for a module to be compiled?

To sum up, these are the questions that might clear the confusion:

Last updated: Dec 23 2024 at 12:05 UTC