Wasmtime GitHub notifications bot (Jun 27 2022 at 20:19): Wasmtime GitHub notifications bot (Jun 27 2022 at 20:19):alexcrichton opened issue #4338:
With https://github.com/bytecodealliance/wasmtime/issues/4322 recently I think one thing we could to do help improve our coverage when fuzzing is to re-integrate Binaryen's
wasm-opt -ttffuzzer. The wasm modules coming out of that have a different shape than those ofwasm-smithand might be interesting to fuzz in addition to those fromwasm-smithfor more coverage.Historically we had this but since removed it for a few reasons:
- We were using a crate for integration with Binaryen and the crate was somewhat slow to update the submodule, meaning it took quite some time to get fixes from Binaryen in to Wasmtime.
- Binaryen itself doesn't run under libfuzzer, but Wasmtime on oss-fuzz compiles it with fuzz options. This means that fuzzing was getting really good at exploring code in the binaryen wasm-module-generator itself and often found paths where binaryen took up too much memory or took too long to generate a wasm module. These are difficult and time consuming to track down for a foreign codebase, however.
- At the time we theorized that our coverage wasn't much better than with wasm-smith.
It would perhaps be prudent to reevaluate some of these. While the second point likely still stands we could perhaps improve extraction of the random seed and a "reproduction command" to send to upstream binaryen and avoid investigating too much ourselves to avoid adding work to our plates. Specifically the last point may be best empirically evaluated by looking at coverage reports for
cranelift-codegento see how many instructions are hit and how many lowerings are touched.
Wasmtime GitHub notifications bot (Jun 27 2022 at 20:19):alexcrichton labeled issue #4338:
With https://github.com/bytecodealliance/wasmtime/issues/4322 recently I think one thing we could to do help improve our coverage when fuzzing is to re-integrate Binaryen's
wasm-opt -ttffuzzer. The wasm modules coming out of that have a different shape than those ofwasm-smithand might be interesting to fuzz in addition to those fromwasm-smithfor more coverage.Historically we had this but since removed it for a few reasons:
- We were using a crate for integration with Binaryen and the crate was somewhat slow to update the submodule, meaning it took quite some time to get fixes from Binaryen in to Wasmtime.
- Binaryen itself doesn't run under libfuzzer, but Wasmtime on oss-fuzz compiles it with fuzz options. This means that fuzzing was getting really good at exploring code in the binaryen wasm-module-generator itself and often found paths where binaryen took up too much memory or took too long to generate a wasm module. These are difficult and time consuming to track down for a foreign codebase, however.
- At the time we theorized that our coverage wasn't much better than with wasm-smith.
It would perhaps be prudent to reevaluate some of these. While the second point likely still stands we could perhaps improve extraction of the random seed and a "reproduction command" to send to upstream binaryen and avoid investigating too much ourselves to avoid adding work to our plates. Specifically the last point may be best empirically evaluated by looking at coverage reports for
cranelift-codegento see how many instructions are hit and how many lowerings are touched.
Last updated: Dec 23 2024 at 12:05 UTC