Wasmtime GitHub notifications bot (Mar 31 2022 at 05:34): Wasmtime GitHub notifications bot (Mar 31 2022 at 05:34):kimdora commented on issue #1151:
Hello, is there further progress for tracking code coverage?
Wasmtime GitHub notifications bot (Mar 31 2022 at 05:54):cfallin commented on issue #1151:
Hi @kimdora, unfortunately no one has had time to look at this yet. If you or someone else is interested in trying to implement this, we would definitely be able to help guide the effort, though.
Wasmtime GitHub notifications bot (Mar 31 2022 at 08:16):kimdora commented on issue #1151:
Hi @cfallin, actually I'm interested in this. I wanna try to implement this, but unfortunately I'm not sure where to start. Is there any guide on things regarding this?
Wasmtime GitHub notifications bot (Mar 31 2022 at 16:08):fitzgen commented on issue #1151:
@kimdora our contributing docs are here: https://docs.wasmtime.dev/contributing.html (those are a little wasmtime specific but cranelift's process is pretty much identical)
Wasmtime GitHub notifications bot (Mar 31 2022 at 16:45):cfallin commented on issue #1151:
@kimdora to get started on this particular idea, a few things that might be helpful:
- Instrumentation for generated code run by Wasmtime is generally done in this file: it has hooks that are invoked by the Wasm-to-IR lowering at certain points (loop headers, before and after every instruction, etc).
- The runtime has a toplevel data structure called the
VMContextthat is accessible from generated code. It is defined implicitly by computing offsets here. In the lowering above, you'll see various references to the vmctx and loads of pointers from its fields, for example.- The IR opcodes (our IR is called "CLIF") that are available can be seen in the InstBuilder documentation.
My first step would probably be to design the data structures. To track code coverage, we probably want to keep counters of some sort, one counter per "location", and increment a counter when control-flow reaches that location in the generated code. A location might be a basic block, or an individual Wasm opcode; the latter may be simplest with the existing hooks, but will be significantly slower (2x-5x slowdown probably). Then given that data structure, allocate it during VMContext construction, and in the generated-code hooks, load a pointer to the array once at the beginning of the function and emit a load-add-store sequence at a particular offset before every Wasm opcode is lowered.
@fitzgen mentions above that there are tricks to reduce the number of points at which you increment a counter, and reconstruct later. For an example of the intuition, a loop with a straight-line-code body, i.e. no conditionals inside the loop, only needs a single counter increment for each loop iteration. And an if-else needs a counter for each path (or I guess one could recover one given the other), but the control flow merge doesn't need a counter because we know control must flow there. In general I think we can use the domtree to work this out. I know I've seen a paper about this but can't recall it now; anyway we can talk about this more when we need it and we definitely don't need to implement it right away!
I would estimate that the simplest version (array of counters one per Wasm opcode, instrumentation at each Wasm opcode) is probably ~100 lines of code; it shouldn't be too complex in an absolute sense, but I recognize this is probably a lot of new context to learn. We're happy to help explain more or work things out if you get stuck!
Wasmtime GitHub notifications bot (Mar 31 2022 at 16:57):cfallin commented on issue #1151:
Ah, and I should add that in the above I was assuming doing this for Wasm code running in Wasmtime specifically; were you interested in that or in coverage instrumentation for some other Cranelift frontend? In the latter case the basic steps would be the same but the implementation would probably best be done in whatever other frontend (since it requires interaction with runtime data structures) rather than purely in Cranelift.
Wasmtime GitHub notifications bot (Apr 01 2022 at 06:29):kimdora commented on issue #1151:
@cfallin, really thank you for your kind reply. I should carefully read your above comment. There are lots of information that you provided so that I haven't read all of things in detail at this moment... If there is any progress or question, I'll comment again! Also I think your comment is quite enough because I'm interested in tracking code coverage during running WebAssembly code running in Wasmtime!
Last updated: Oct 23 2024 at 20:03 UTC