https://github.com/fitzgen/wasm-smith
The README has the sales pitch. I've already used this to find a bunch of bugs in wasmparser
. Isn't published on crates.io yet, because I need to upstream some changes to the arbitrary
crate first.
cc @Yury Delendik @Alex Crichton @Jonathan Foote
Nice! Definitely looking forward to seeing this hooked up to wasmtime
@fitzgen (he/him) this might be good to include in the wasm tools repo?
Perhaps!
Now up on crates.io: https://crates.io/crates/wasm-smith
blog post describing its implementation, if anyone is interested: https://fitzgeraldnick.com/2020/08/24/writing-a-test-case-generator.html
and let the fuzz bugs begin...
@fitzgen (he/him) btw the auto-reduction oss-fuzz does is really good with wasm-smith
it reduced a 30kb input to 4kb which generated a module with just one type in it
so the pattern you're using for generating a wasm module I think is really amenable to auto-reduction
although you probably knew this already heh
yeah, I think it is a combo of two things:
wasm-opt -ttf
was cycling back over the raw input and doing funky stuff like xoring it with a counter, rather than simply padding the end of the input with zeroes. this meant that in order to reproduce a bug that involves something generated after it has cycled back around the input, you have to have the whole prefix. there is a lot of unintentional stuff entrained in the test case.
I switched a few things in the arbitrary crate to be more fuzzing friendly. we are more lax about errors (eg a string that isn't perfectly utf-8 or when we need more input than the fuzzer provided us) and I also made it so that collections are built up via a "continuation byte" rather than by reading a length and then reading that many elements. This latter one was inspired by a paper about test case reduction from the python hypothesis testing framework. See https://github.com/rust-fuzz/arbitrary/issues/52 for details
yeah I saw that about the keep_going
bool which I thought was pretty clever
I do think though that one part about wasm-smith is that it's difficult to have a stable corpus, because as we tweak wasm-smith a previous input could be reinterpreted totally differently (e.g. with the patch I just sent)
not really a huge issue, but also not sure how we'd fix it
agreed. I don't think it is a huge deal and I'm also not sure how to fix it where the fix wouldn't be worse than just rediscovering some coverage
Last updated: Dec 23 2024 at 12:05 UTC