Stream: git-wasmtime

Topic: wasmtime / Issue #2112 Cache mode

Wasmtime GitHub notifications bot (Aug 07 2020 at 10:26):

sergiizubchevskyy opened Issue #2112:

Hello!

I'm currently running some benchmarks and observed some strange behavior when testing Wasmtime with cache enabled.

Basically there are no changes in a CPU or a RAM performance when using the cache mode. For the CPU benchmarking I used CoreMark and for the RAM I used Massif. Here are the snippets of the results:



Here is my cache configuration file:

With cache enabled I use Wasmtime in the same way as if cache was disabled, i.e. just passing the .wasm file.
Am I using it wrong? Because I was expecting to have at least lower RAM usage since file was already compiled.

Thank you in advance.

Wasmtime GitHub notifications bot (Aug 07 2020 at 11:10):

bjorn3 commented on Issue #2112:

Basically there are no changes in a CPU or a RAM performance when using the cache mode.

No change in CoreMark score is expected as I think it doesn't start measuring until the wasm file starts running, on which the cache has no effect and in any case the runtime of CoreMark is likely much bigger than the time it takes for wasmtime to compile it, making it very hard to see any significant change in total execution time.

That there is pretty much no change in memory usage is also expected I think, as Cranelift compiles one function at a time, meaning that during the whole compilation there is just a relatively small increase in memory usage on top of reading the wasm file into memory.

Wasmtime GitHub notifications bot (Aug 07 2020 at 12:03):

sergiizubchevskyy commented on Issue #2112:

Thank you for your explanation!

Could you clarify than what is the meaning of cache in a context of Wasmtime? I probably misunderstood something since I was expecting for it to behave a bit like ahead of time mode thus expected lower RAM usage since no compilation was required.

Sorry to bother with what are probably stupid questio

Wasmtime GitHub notifications bot (Aug 07 2020 at 12:03):

sergiizubchevskyy edited a comment on Issue #2112:

Thank you for your explanation!

Could you clarify than what is the meaning of cache in a context of Wasmtime? I probably misunderstood something since I was expecting for it to behave a bit like ahead of time mode thus expected lower RAM usage since no compilation was required.

Sorry to bother with what are probably stupid questions.

Wasmtime GitHub notifications bot (Aug 07 2020 at 12:06):

tschneidereit commented on Issue #2112:

My guess would be that compilation has a lower peak memory consumption than execution, so it doesn't affect the overall peak. Caching however has a higher peak, so you see that in the first run with caching enabled.

Wasmtime GitHub notifications bot (Aug 07 2020 at 12:20):

bjorn3 commented on Issue #2112:

Could you clarify than what is the meaning of cache in a context of Wasmtime? I probably misunderstood something since I was expecting for it to behave a bit like ahead of time mode thus expected lower RAM usage since no compilation was required.

The cache is for faster startup times.

Wasmtime GitHub notifications bot (Aug 07 2020 at 12:20):

sergiizubchevskyy commented on Issue #2112:

I have a Massif graphs here

This is execution of Wasmtime without cache option:

This is a first run of Wasmtime with cache enabled:

This is a second run of Wasmtime with cache enabled:

And this is a third run of Wasmtime with cache enabled:

So I think compilation has a higher peak than execution in all cases.

Wasmtime GitHub notifications bot (Aug 07 2020 at 12:21):

sergiizubchevskyy commented on Issue #2112:

Could you clarify than what is the meaning of cache in a context of Wasmtime? I probably misunderstood something since I was expecting for it to behave a bit like ahead of time mode thus expected lower RAM usage since no compilation was required.

The cache is for faster startup times.

And how is this achieved? Pre-compilation?

Wasmtime GitHub notifications bot (Aug 07 2020 at 12:22):

bjorn3 commented on Issue #2112:

Yes, the cache is a serialized version of the compilation results. It is created the first time you compile with caching enabled and can then be loaded again on future runs.

Wasmtime GitHub notifications bot (Aug 07 2020 at 12:35):

tschneidereit commented on Issue #2112:

So I think compilation has a higher peak than execution in all cases.

Agreed, that is what it looks like — thank you for sharing that analysis. We do a couple of copies of the code memory during cache deserialization right now, which should eventually go away through more optimization. Once that's done, this should change.

Out of curiosity, is this a problem for you right now?

Wasmtime GitHub notifications bot (Aug 07 2020 at 12:45):

sergiizubchevskyy commented on Issue #2112:

No, this is not a problem at all and again, thank you for such a fast and explicit answers!

Our team is currently doing some investigation to select runtime for our use case, the benchmarks showed above are part of our investigation. We are bit concerned with the RAM and the CPU performance that is why we are looking at those metrics.

Some time ago I had an idea that Wasmtime "had" AOT mode, however no such information is being found anywhere right now, so I thought that cache mode is some sort of alternative.

We are currently comparing Wasmtime to WAMR and in terms of RAM it is possible to achieve relatively low RAM usage (with patch provided by WAMR team member we can lower the RAM consumption from 8MiB to ~600kiB), this is why I was looking for a way to lower the RAM usage with Wasmtime.

Wasmtime GitHub notifications bot (Aug 07 2020 at 12:55):

tschneidereit commented on Issue #2112:

Some time ago I had an idea that Wasmtime "had" AOT mode, however no such information is being found anywhere right now, so I thought that cache mode is some sort of alternative.

AOT mode is planned, and we recently did a lot of refactoring of the compilation pipeline and caching logic to work towards that, see e.g. #1931.

Wasmtime GitHub notifications bot (Aug 07 2020 at 15:12):

alexcrichton commented on Issue #2112:

It looks like the goal here is to reduce memory usage, and looking at the memory usage graphs it looks like a huge chunk of memory comes from zstd. With Module::serialize and Module::deserialize you can choose how to store the serialized module, and while the default caching system with Wasmtime uses zstd you don't have to use that (you could even eschew compression entirely). To test that out you'd need to write the binding code yourself (e.g. use the wasmtime crate), but you may be able to get more specialized numbers than the CLI.

Wasmtime GitHub notifications bot (Aug 10 2020 at 09:44):

sergiizubchevskyy commented on Issue #2112:

Understood.

Thank you for explanation. I am clarified and as such will close this issue.

Best regards.

Wasmtime GitHub notifications bot (Aug 10 2020 at 09:44):

sergiizubchevskyy closed Issue #2112:

Hello!

I'm currently running some benchmarks and observed some strange behavior when testing Wasmtime with cache enabled.

Basically there are no changes in a CPU or a RAM performance when using the cache mode. For the CPU benchmarking I used CoreMark and for the RAM I used Massif. Here are the snippets of the results:



Here is my cache configuration file:

With cache enabled I use Wasmtime in the same way as if cache was disabled, i.e. just passing the .wasm file.
Am I using it wrong? Because I was expecting to have at least lower RAM usage since file was already compiled.

Thank you in advance.

Last updated: Nov 22 2024 at 16:03 UTC