Wasmtime GitHub notifications bot (Jul 06 2022 at 22:14): Wasmtime GitHub notifications bot (Jul 06 2022 at 22:14):fitzgen opened PR #4398 from trap-micro-benchmarks to main:
<!--
Please ensure that the following steps are all taken care of before submitting
the PR.
[ ] This has been discussed in issue #..., or if not, please tell us why
here.[ ] A short description of what this does, why it is needed; if the
description becomes long, the matter should probably be discussed in an issue
first.[ ] This PR contains test cases, if meaningful.
- [ ] A reviewer from the core maintainer team has been assigned for this PR.
If you don't know who could review this, please indicate so. The list of
suggested reviewers on the right can help you.Please ensure all communication adheres to the code of conduct.
-->
Wasmtime GitHub notifications bot (Jul 06 2022 at 22:14):fitzgen requested alexcrichton for a review on PR #4398.
Wasmtime GitHub notifications bot (Jul 06 2022 at 22:24):alexcrichton submitted PR review.
Wasmtime GitHub notifications bot (Jul 06 2022 at 22:24):alexcrichton created PR review comment:
How come this is re-instantiating each time? Could the same instance be used to use
b.iter(|| ...)?
Wasmtime GitHub notifications bot (Jul 06 2022 at 22:24):alexcrichton created PR review comment:
I think this may be a copy/paste error and there should only be one module in this benchmark?
Wasmtime GitHub notifications bot (Jul 06 2022 at 22:24):alexcrichton created PR review comment:
Personally I find that benchmarks like this aren't the best for measuring multithreaded contention. The threads aren't actually guaranteed to be interleaved here as starting up a thread may sometimes take just as long as the number of iterations. Additionally the numbers coming out of criterion itself are unrelated to other numbers below about depth and number of modules since here it's N threads doing 1000 traps where below it's time-per-trap.
Would you be up for restructuring this like the concurrent instantiation benchmark? That one has a scheme where:
- N threads are started in the background infinitely doing work.
- The main thread waits for all N threads to start
- The main thread then does normal criterion-things measuring per-iteration time (e.g. per-one-trap time)
- After the main thread finishes benchmarking it sends a shutdown signal to all threads and joins them
That should guarantee that there's concurrent work in the background and additionally yield comparable numbers to the other benchmarks below.
Wasmtime GitHub notifications bot (Jul 06 2022 at 22:24):alexcrichton submitted PR review.
Wasmtime GitHub notifications bot (Jul 06 2022 at 22:26):fitzgen submitted PR review.
Wasmtime GitHub notifications bot (Jul 06 2022 at 22:26):fitzgen created PR review comment:
Because one isn't really "supposed" to call back into a wasm instance after it traps.
Wasmtime GitHub notifications bot (Jul 06 2022 at 22:26):fitzgen submitted PR review.
Wasmtime GitHub notifications bot (Jul 06 2022 at 22:26):fitzgen created PR review comment:
Good eye.
Wasmtime GitHub notifications bot (Jul 06 2022 at 22:27):fitzgen created PR review comment:
Good idea!
Wasmtime GitHub notifications bot (Jul 06 2022 at 22:27):fitzgen submitted PR review.
Wasmtime GitHub notifications bot (Jul 06 2022 at 23:07):fitzgen updated PR #4398 from trap-micro-benchmarks to main.
Wasmtime GitHub notifications bot (Jul 06 2022 at 23:14):fitzgen submitted PR review.
Wasmtime GitHub notifications bot (Jul 06 2022 at 23:14):fitzgen created PR review comment:
Done.
Wasmtime GitHub notifications bot (Jul 06 2022 at 23:14):fitzgen submitted PR review.
Wasmtime GitHub notifications bot (Jul 06 2022 at 23:14):fitzgen created PR review comment:
Switched to one instance/store.
Wasmtime GitHub notifications bot (Jul 06 2022 at 23:14):fitzgen submitted PR review.
Wasmtime GitHub notifications bot (Jul 06 2022 at 23:14):fitzgen created PR review comment:
Fixed.
Wasmtime GitHub notifications bot (Jul 06 2022 at 23:56):alexcrichton submitted PR review.
Wasmtime GitHub notifications bot (Jul 06 2022 at 23:56):alexcrichton has enabled auto merge for PR #4398.
Wasmtime GitHub notifications bot (Jul 07 2022 at 00:20):alexcrichton merged PR #4398.
Last updated: Nov 22 2024 at 17:03 UTC