Wasmtime GitHub notifications bot (Oct 18 2021 at 17:51): Wasmtime GitHub notifications bot (Oct 18 2021 at 17:51):kirawi opened issue #3460:
Feature
I would like it if Wasmtime would publish benchmarks of native Rust code versus it compiled down to Wasm.
Benefit
It would help decide how performant it is compared to a direct scripting language, and if its portability outweighs its worse performance. There isn't a source truth for such comparisons AFAIK.
Implementation
Outside of my knowledge.
Alternatives
Don't implement/publish such benchmarks.
Wasmtime GitHub notifications bot (Oct 18 2021 at 17:51):kirawi edited issue #3460:
Feature
I would like it if Wasmtime would publish benchmarks of native Rust code versus it compiled down to Wasm.
Benefit
It would help decide how performant it is compared to a direct scripting language, and if its portability outweighs its worse performance. There isn't a source truth for such comparisons AFAIK except https://00f.net/2021/02/22/webassembly-runtimes-benchmarks/
Implementation
Outside of my knowledge.
Alternatives
Don't implement/publish such benchmarks.
Wasmtime GitHub notifications bot (Oct 18 2021 at 17:59):cfallin commented on issue #3460:
Some folks have been working on benchmarking infrastructure using Sightglass with possibly some CI integration and/or historical views -- @jlb6740 what's the latest status on that?
Wasmtime GitHub notifications bot (Oct 19 2021 at 00:52):kirawi commented on issue #3460:
It doesn't look like it benchmarks the native equivalent.
Wasmtime GitHub notifications bot (Oct 19 2021 at 15:17):cfallin commented on issue #3460:
In the past, at least in presentations I've seen, Sightglass results have included performance ratios against the same code running natively; and I think that's the plan going forward, but @jlb6740 or @fitzgen could comment further?
Wasmtime GitHub notifications bot (Oct 19 2021 at 16:42):fitzgen commented on issue #3460:
The benchmark suite is explicitly interally focused (is this PR a regression/improvement? what is our perf trend over time?) and we de-emphasized external presentation, "benchmarketing", and direct comparisons to other wasm engines or native, per the RFC.
@kirawi You could probably get the benchmarks running on native without a _ton_ of work if you're motivated to do that, but it isn't really the focus of our benchmarking efforts.
Wasmtime GitHub notifications bot (Oct 19 2021 at 23:25):kirawi commented on issue #3460:
I think it would still be valuable to have benchmarks against native code to help in deciding whether a project like Wasmtime fits your requirements.
Wasmtime GitHub notifications bot (Oct 19 2021 at 23:26):kirawi edited a comment on issue #3460:
I think it would still be valuable to have benchmarks against native code to help in deciding whether a project like Wasmtime fits your requirements. Wasm will obviously be overall slower, but _how slow_ is a big deciding factor.
Wasmtime GitHub notifications bot (Oct 19 2021 at 23:27):kirawi edited a comment on issue #3460:
I think it would still be valuable to have benchmarks against native code to help in deciding whether a project like Wasmtime fits design requirements. Wasm will obviously be overall slower, but _how slow_ is a big deciding factor.
Last updated: Nov 22 2024 at 17:03 UTC