Stream: git-wasmtime

Topic: wasmtime / issue #3460 Publicly viewable benchmarks again...


view this post on Zulip Wasmtime GitHub notifications bot (Oct 18 2021 at 17:51):

kirawi opened issue #3460:

Feature

I would like it if Wasmtime would publish benchmarks of native Rust code versus it compiled down to Wasm.

Benefit

It would help decide how performant it is compared to a direct scripting language, and if its portability outweighs its worse performance. There isn't a source truth for such comparisons AFAIK.

Implementation

Outside of my knowledge.

Alternatives

Don't implement/publish such benchmarks.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 18 2021 at 17:51):

kirawi edited issue #3460:

Feature

I would like it if Wasmtime would publish benchmarks of native Rust code versus it compiled down to Wasm.

Benefit

It would help decide how performant it is compared to a direct scripting language, and if its portability outweighs its worse performance. There isn't a source truth for such comparisons AFAIK except https://00f.net/2021/02/22/webassembly-runtimes-benchmarks/

Implementation

Outside of my knowledge.

Alternatives

Don't implement/publish such benchmarks.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 18 2021 at 17:59):

cfallin commented on issue #3460:

Some folks have been working on benchmarking infrastructure using Sightglass with possibly some CI integration and/or historical views -- @jlb6740 what's the latest status on that?

view this post on Zulip Wasmtime GitHub notifications bot (Oct 19 2021 at 00:52):

kirawi commented on issue #3460:

It doesn't look like it benchmarks the native equivalent.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 19 2021 at 15:17):

cfallin commented on issue #3460:

In the past, at least in presentations I've seen, Sightglass results have included performance ratios against the same code running natively; and I think that's the plan going forward, but @jlb6740 or @fitzgen could comment further?

view this post on Zulip Wasmtime GitHub notifications bot (Oct 19 2021 at 16:42):

fitzgen commented on issue #3460:

The benchmark suite is explicitly interally focused (is this PR a regression/improvement? what is our perf trend over time?) and we de-emphasized external presentation, "benchmarketing", and direct comparisons to other wasm engines or native, per the RFC.

@kirawi You could probably get the benchmarks running on native without a _ton_ of work if you're motivated to do that, but it isn't really the focus of our benchmarking efforts.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 19 2021 at 23:25):

kirawi commented on issue #3460:

I think it would still be valuable to have benchmarks against native code to help in deciding whether a project like Wasmtime fits your requirements.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 19 2021 at 23:26):

kirawi edited a comment on issue #3460:

I think it would still be valuable to have benchmarks against native code to help in deciding whether a project like Wasmtime fits your requirements. Wasm will obviously be overall slower, but _how slow_ is a big deciding factor.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 19 2021 at 23:27):

kirawi edited a comment on issue #3460:

I think it would still be valuable to have benchmarks against native code to help in deciding whether a project like Wasmtime fits design requirements. Wasm will obviously be overall slower, but _how slow_ is a big deciding factor.


Last updated: Dec 23 2024 at 12:05 UTC