Stream: git-wasmtime

Topic: wasmtime / PR #9457 Change how env vars work in `differen...


view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 16:06):

alexcrichton opened PR #9457 from alexcrichton:update-differential-fuzzer to bytecodealliance:main:

This commit updates the processing of the ALLOWED_* environment variables to work differently than before. Previously the list of engines and module-generation-strategies were filtered based on ALLOWED_* environment variables but this meant that the meaning of a fuzz input changed depending on environment variables. This commit instead replaces the handling of these environment variables to ensure that the meaning of the fuzz input doesn't change depending on their values. Instead fuzz test cases are early-thrown-out if they request an engine that's disabled or a module-generation-strategy that's disabled.

<!--
Please make sure you include the following information:

Our development process is documented in the Wasmtime book:
https://docs.wasmtime.dev/contributing-development-process.html

Please ensure all communication follows the code of conduct:
https://github.com/bytecodealliance/wasmtime/blob/main/CODE_OF_CONDUCT.md
-->

view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 16:07):

alexcrichton requested elliottt for a review on PR #9457.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 16:07):

alexcrichton requested wasmtime-fuzz-reviewers for a review on PR #9457.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 18:46):

github-actions[bot] commented on PR #9457:

Subscribe to Label Action

cc @fitzgen

<details>
This issue or pull request has been labeled: "fuzzing"

Thus the following users have been cc'd because of the following labels:

To subscribe or unsubscribe from this label, edit the <code>.github/subscribe-to-label.json</code> configuration file.

Learn more.
</details>

view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 18:52):

fitzgen submitted PR review.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 18:52):

fitzgen created PR review comment:

            log::debug!("test case uses a runtime-disabled engine");

view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 18:52):

elliottt submitted PR review:

Looks good, just curious about how we ensure that the fuzzer is staying productive.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 18:52):

elliottt created PR review comment:

            log::debug!("test case uses a runtime-disabled engine");

view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 18:52):

elliottt created PR review comment:

Do we have a way to categorize the early return cases, or can we find out if the fuzzer is spending all its time in unproductive configurations?

view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 18:56):

cfallin submitted PR review.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 18:56):

cfallin created PR review comment:

In the regalloc.rs fuzzer we used atomics to count stats (iirc similar thing, rejected cases) and the fuzz case that got a value 0 mod 1000 from the fetch-and-add printed a progress message with all stats -- this at least gave us a general sense watching interactively. I don't think there's a way to get such output from oss-fuzz though. IIRC there is a way to return a status to libFuzzer that tells it specifically the case is uninteresting?

view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 19:02):

abrown submitted PR review.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 19:02):

abrown created PR review comment:

It might not be clear in this diff but RuntimeStats is what does that here.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 19:25):

alexcrichton submitted PR review.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 19:25):

alexcrichton created PR review comment:

Can confirm what @abrown mentioned should work and it's the same idea as regalloc's fuzzers @cfallin mentions.

AFAIK though there's no way to tell libFuzzer something is uninteresting, but by not doing anything you're also saying "this introduces no new coverage" which means it should naturally get pruned and deduplicated from the corpus over time.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 19:25):

alexcrichton updated PR #9457.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 19:26):

alexcrichton has enabled auto merge for PR #9457.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 19:51):

alexcrichton merged PR #9457.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 22:34):

fitzgen submitted PR review.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 22:34):

fitzgen created PR review comment:

AFAIK though there's no way to tell libFuzzer something is uninteresting, but by not doing anything you're also saying "this introduces no new coverage" which means it should naturally get pruned and deduplicated from the corpus over time.

There actually is: if you return this https://docs.rs/libfuzzer-sys/latest/libfuzzer_sys/enum.Corpus.html#variant.Reject from a fuzz target, then it tells libfuzzer not to add that testcase into the corpus or reuse it for future mutations or whatever.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 22:41):

alexcrichton submitted PR review.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 11 2024 at 22:41):

alexcrichton created PR review comment:

Whoa fancy!


Last updated: Dec 23 2024 at 13:07 UTC