Wasmtime GitHub notifications bot (Jul 28 2021 at 21:03): Wasmtime GitHub notifications bot (Jul 28 2021 at 21:03):github-actions[bot] commented on issue #3124:
Subscribe to Label Action
cc @fitzgen
<details>
This issue or pull request has been labeled: "fuzzing"Thus the following users have been cc'd because of the following labels:
- fitzgen: fuzzing
To subscribe or unsubscribe from this label, edit the <code>.github/subscribe-to-label.json</code> configuration file.
Learn more.
</details>
Wasmtime GitHub notifications bot (Jul 28 2021 at 23:03):abrown commented on issue #3124:
All, thanks for the comments. I've fixed most of the small stuff but I've left open the ones that I still need help on (how and when to build
wasm-spec-interpreter, e.g.).
Wasmtime GitHub notifications bot (Aug 03 2021 at 23:53):abrown commented on issue #3124:
I rebased this on
mainand condensed it to five commits. I think this is almost good for a final review except for the issue I raised in https://github.com/bytecodealliance/wasmtime/pull/3124#discussion_r678631805 about unsafe behavior. Something is not right there and we should wait to merge this until it is resolved. I can make the fuzzer crash by runningcargo +nightly fuzz run differential_speclocally:INFO: Running with entropic power schedule (0xFF, 100). INFO: Seed: 2512589415 INFO: Loaded 1 modules (699235 inline 8-bit counters): 699235 [0x5557756ba9da, 0x55577576553d), INFO: Loaded 1 PC tables (699235 PCs): 699235 [0x555775765540,0x555776210b70), INFO: 1425 files found in /home/abrown/Code/wasmtime/fuzz/corpus/differential_spec INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes INFO: seed corpus: files: 1425 min: 1b max: 63b total: 52307b rss: 72Mb #1024 pulse cov: 16016 ft: 34393 corp: 418/12380b exec/s: 33 rss: 429Mb === Execution rate (572 executed modules / 1000 tried modules): 57.199999999999996% === #2048 pulse cov: 17686 ft: 41583 corp: 838/28Kb exec/s: 28 rss: 429Mb === Execution rate (992 executed modules / 2000 tried modules): 49.6% === [...time passes] #3227 NEW cov: 18157 ft: 47066 corp: 1060/39Kb lim: 68 exec/s: 35 rss: 429Mb L: 47/68 MS: 1 CrossOver- #3268 NEW cov: 18157 ft: 47074 corp: 1061/39Kb lim: 68 exec/s: 35 rss: 429Mb L: 48/68 MS: 1 CrossOver- #3309 NEW cov: 18158 ft: 47076 corp: 1062/39Kb lim: 68 exec/s: 36 rss: 429Mb L: 64/68 MS: 1 InsertRepeatedBytes- #3346 NEW cov: 18158 ft: 47077 corp: 1063/39Kb lim: 68 exec/s: 36 rss: 429Mb L: 38/68 MS: 2 ChangeByte-InsertByte- #3397 NEW cov: 18158 ft: 47079 corp: 1064/39Kb lim: 68 exec/s: 37 rss: 429Mb L: 36/68 MS: 1 ChangeASCIIInt- #3445 NEW cov: 18158 ft: 47108 corp: 1065/39Kb lim: 68 exec/s: 37 rss: 429Mb L: 41/68 MS: 3 ChangeASCIIInt-InsertByte-CopyPart- #3452 NEW cov: 18158 ft: 47113 corp: 1066/39Kb lim: 68 exec/s: 37 rss: 429Mb L: 58/68 MS: 2 InsertRepeatedBytes-EraseBytes- ──────────────────────────────────────────────────────────────────────────────── Error: Fuzz target exited with signal: 11No core dump is associated with this crash so it's hard to tell what is going on. I would appreciate someone else trying to run this, or even any suggestions on how to narrow down what is going wrong.
Wasmtime GitHub notifications bot (Aug 05 2021 at 21:37):alexcrichton commented on issue #3124:
I tried again to get this working locally, and I believe my previous build error was due to an older
ocamlcin PATH than what was needed here (according to The Internet). I figured out how to wrangle opam and such and I got things working with 4.12.0.I got a different error than you did initially, and it's behavior I've never seen before. The fuzzer would just stop after a few hundreds runs and print that it detected a leak:
<details>
================================================================= ==44138==ERROR: LeakSanitizer: detected memory leaks Direct leak of 32 byte(s) in 1 object(s) allocated from: #0 0xaaaaac28c01c in calloc /checkout/src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:154 #1 0xffffb372e3f0 in __cxa_thread_atexit_impl /build/glibc-pfQmxL/glibc-2.28/stdlib/cxa_thread_atexit_impl.c:106:27 #2 0xaaaab0a73074 in std::sys::unix::thread_local_dtor::register_dtor::h901a681bd6e662af /rustc/25b764849625cb090e8b81d12d2bb2295d073788/library/std/src/sys/unix/thread_local_dtor.rs:36:9 #3 0xaaaaaea5e224 in std::thread::local::LocalKey$LT$T$GT$::with::h9f753ffa090280d6 (/home/acrichto/code/wasmtime/target/aarch64-unknown-linux-gnu/release/differential_spec+0x3df3224) #4 0xaaaaaeb0f3dc in wasmtime_runtime::traphandlers::tls::raw::replace::h66677875f2db8192 (/home/acrichto/code/wasmtime/target/aarch64-unknown-linux-gnu/release/differential_spec+0x3ea43dc) #5 0xaaaaac512d60 in wasmtime_runtime::traphandlers::tls::set::h4626657d9defd11e (/home/acrichto/code/wasmtime/target/aarch64-unknown-linux-gnu/release/differential_spec+0x18a7d60) #6 0xaaaaac676480 in wasmtime::func::invoke_wasm_and_catch_traps::hd21244aafcd451e9 (/home/acrichto/code/wasmtime/target/aarch64-unknown-linux-gnu/release/differential_spec+0x1a0b480) #7 0xaaaaac679638 in wasmtime::func::Func::call_impl::hb80934ccfb58865f (/home/acrichto/code/wasmtime/target/aarch64-unknown-linux-gnu/release/differential_spec+0x1a0e638) #8 0xaaaaac717d64 in wasmtime_fuzzing::oracles::run_in_wasmtime::hb0614ee0e39ac4f8 (/home/acrichto/code/wasmtime/target/aarch64-unknown-linux-gnu/release/differential_spec+0x1aacd64) #9 0xaaaaac715330 in wasmtime_fuzzing::oracles::differential_spec_execution::h71bb42d8a0672244 (/home/acrichto/code/wasmtime/target/aarch64-unknown-linux-gnu/release/differential_spec+0x1aaa330) #10 0xaaaaac4ad54c in rust_fuzzer_test_input (/home/acrichto/code/wasmtime/target/aarch64-unknown-linux-gnu/release/differential_spec+0x184254c) #11 0xaaaab09d2d50 in __rust_try (/home/acrichto/code/wasmtime/target/aarch64-unknown-linux-gnu/release/differential_spec+0x5d67d50) #12 0xaaaab09d2428 in LLVMFuzzerTestOneInput (/home/acrichto/code/wasmtime/target/aarch64-unknown-linux-gnu/release/differential_spec+0x5d67428) #13 0xaaaab09d7534 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/acrichto/code/wasmtime/target/aarch64-unknown-linux-gnu/release/differential_spec+0x5d6c534) #14 0xaaaab09db6d4 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) (/home/acrichto/code/wasmtime/target/aarch64-unknown-linux-gnu/release/differential_spec+0x5d706d4) #15 0xaaaab09dd4b0 in fuzzer::Fuzzer::MutateAndTestOne() (/home/acrichto/code/wasmtime/target/aarch64-unknown-linux-gnu/release/differential_spec+0x5d724b0) #16 0xaaaab09e0bbc in fuzzer::Fuzzer::Loop(std::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) (/home/acrichto/code/wasmtime/target/aarch64-unknown-linux-gnu/release/differential_spec+0x5d75bbc) #17 0xaaaab09f8974 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/acrichto/code/wasmtime/target/aarch64-unknown-linux-gnu/release/differential_spec+0x5d8d974) #18 0xaaaaac202930 in main (/home/acrichto/code/wasmtime/target/aarch64-unknown-linux-gnu/release/differential_spec+0x1597930) #19 0xffffb3719d20 in __libc_start_main /build/glibc-pfQmxL/glibc-2.28/csu/../csu/libc-start.c:308:16 #20 0xaaaaac202a5c in _start (/home/acrichto/code/wasmtime/target/aarch64-unknown-linux-gnu/release/differential_spec+0x1597a5c) SUMMARY: AddressSanitizer: 32 byte(s) leaked in 1 allocation(s). INFO: to ignore leaks on libFuzzer side use -detect_leaks=0. MS: 1 CopyPart-; base unit: 5080fd9c0bf811fe129fd0db4e1158b53926333c 0xa,0xf4,0xf4,0xf4,0xf4,0xf4,0xf4,0xf4,0xf4,0xf4,0xf4,0xf4,0xf4,0xf4,0xf4,0xf4,0xf4,0xf4,0xf4,0xf4,0xf4,0xff,0xf4,0xf4,0xf4, \x0a\xf4\xf4\xf4\xf4\xf4\xf4\xf4\xf4\xf4\xf4\xf4\xf4\xf4\xf4\xf4\xf4\xf4\xf4\xf4\xf4\xff\xf4\xf4\xf4 artifact_prefix='./'; Test unit written to ./leak-8ce318eb459e74f96cb3214962087900c5ce2cea Base64: CvT09PT09PT09PT09PT09PT09PT0//T09A==</details>
This doesn't really have anything to do with Wasmtime, it's just standard registration of a TLS destructor... Also I'm not sure why the fuzzer would just halt after a bunch of runs and it feels like it spuriously detects the leak. Unsure!
Anyway after I passed
-detect_leaks=0to the fuzzer it so far hasn't crashed yet. Right now I'm at=== Execution rate (124756 executed modules / 256000 tried modules): 48.7328125% ===I'm also on an AArch64 machine, which may make a difference. @abrown what version of ocaml are you using? Perhaps you also need to update to 4.12.0 and they fixed something in the meantime?
Wasmtime GitHub notifications bot (Aug 05 2021 at 22:20):abrown commented on issue #3124:
Good to see you got things running! I also have have version 4.12.0 of
ocamlcandocamlopt.Since this morning I tried to:
- run the fuzz target in LLDB but that eventually fails with
LeakSanitizer does not work under ptrace (strace, gdb, etc)- run the fuzz target in Valgrind but that immediately fails with
Shadow memory range interleaves with an existing memory mapping. ASan cannot proceed correctly. ABORTING.(I think libFuzzer's ASAN and Valgrind's ASAN and interfering?)- force OCaml's
Gc.compacton every invocation of the spec interpreter--still crashes- guard invocations of the spec interpreter with a mutex--still crashes
- avoid detecting leaks, like you did (e.g.
cargo +nightly fuzz run differential_spec -- -detect_leaks=0)--still crashesI guess I should try this out on another machine in case there is something specific about a compiler, library, etc. that is local to mine. In any case, @cfallin and I were going to get together to look at this tomorrow afternoon and it would be great fun (:grinning_face_with_smiling_eyes:) if you wanted to join.
Wasmtime GitHub notifications bot (Aug 06 2021 at 15:38):alexcrichton commented on issue #3124:
Sure I'm happy to help out!
Wasmtime GitHub notifications bot (Aug 06 2021 at 22:38):abrown commented on issue #3124:
The latest rebased commits happily fuzz away on my x86_64 Linux machine, where previously the fuzzer would crash after ~2k-4k cases:
$ cargo +nightly fuzz run differential_spec Finished release [optimized] target(s) in 0.10s Running `/home/abrown/Code/wasmtime/target/x86_64-unknown-linux-gnu/release/differential_spec -artifact_prefix=/home/abrown/Code/wasmtime/fuzz/artifacts/differential_spec/ /home/abrown/Code/wasmtime/fuzz/corpus/differential_spec` INFO: Running with entropic power schedule (0xFF, 100). INFO: Seed: 62986658 INFO: Loaded 1 modules (720350 inline 8-bit counters): 720350 [0x558bd3a2529a, 0x558bd3ad5078), INFO: Loaded 1 PC tables (720350 PCs): 720350 [0x558bd3ad5078,0x558bd45d2e58), INFO: 7169 files found in /home/abrown/Code/wasmtime/fuzz/corpus/differential_spec INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes INFO: seed corpus: files: 7169 min: 1b max: 180b total: 725327b rss: 79Mb #512 pulse cov: 12621 ft: 21502 corp: 178/4261b exec/s: 51 rss: 301Mb ... #22307 NEW cov: 22079 ft: 93916 corp: 5068/507Kb lim: 187 exec/s: 52 rss: 649Mb L: 111/187 MS: 4 CopyPart-CMP-ChangeBit-CrossOver- DE: "\xff\xff\xff\xff\xff\xff\xff("- #22438 REDUCE cov: 22079 ft: 93916 corp: 5068/507Kb lim: 187 exec/s: 52 rss: 649Mb L: 127/187 MS: 1 EraseBytes- #22580 NEW cov: 22079 ft: 93918 corp: 5069/508Kb lim: 187 exec/s: 52 rss: 649Mb L: 180/187 MS: 2 ChangeBinInt-ChangeByte- #22612 REDUCE cov: 22079 ft: 93918 corp: 5069/508Kb lim: 187 exec/s: 52 rss: 649Mb L: 119/187 MS: 2 CopyPart-EraseBytes- === Execution rate (5510 executed modules / 20000 tried modules): 27.55% ===The ratio of executable modules steadily decreases but that's a separate problem so I'm going to mark this ready for review.
Last updated: Dec 23 2024 at 13:07 UTC