Stream: git-wasmtime

Topic: wasmtime / issue #4799 cranelift: Use `bugpoint` mutators...

Wasmtime GitHub notifications bot (Aug 27 2022 at 08:35):

afonso360 opened issue #4799:

:wave: Hey,

I had this thought overnight and I'm not sure if there are any hidden downsides.

Both cranelift-icache and clif-util bugpoint change the input functions. cranelift-icache is a bit more random in its approach and bugpoint has some constraints to preserve. However bugpoint is quite developed and can mutate functions in a bunch of ways that icache can't, we could try to use those mutators in icache and check if it breaks the cache or not.

cc: @bjorn3 @bnjbvr @jameysharp

Wasmtime GitHub notifications bot (Aug 27 2022 at 09:00):

bjorn3 commented on issue #4799:

Bugpoint's mutations are much less flexible as they replace for example int returning imstructions with iconst 0. Or they remove things. They never change or add things. In addition they don't affect function declarations at all if I recall correctly, despite this being one of the main things where icache can break. You can always try bugpoint's mutations, but it probably won't be all that useful and in any case will need a large corpus of existing functions to work on.

Wasmtime GitHub notifications bot (Aug 27 2022 at 11:27):

afonso360 commented on issue #4799:

Bugpoint's mutations are much less flexible as they replace for example int returning imstructions with iconst 0. Or they remove things. They never change or add things.

Sure, I wasn't proposing replacing all of icache's mutations, we still need those two sets of random replacing and randomly adding stuff. But we should have the removes somewhat covered without much effort.

In addition they don't affect function declarations at all if I recall correctly, despite this being one of the main things where icache can break.

I had a look at the icache source and it looks like we don't do those transforms either, so that would probably be a good addition.

And it would also be nice to have that on bugpoint, but only the remove half, and not the add random args part. Although I think we can share some code there.

You can always try bugpoint's mutations, but it probably won't be all that useful and in any case will need a large corpus of existing functions to work on.

The fuzzer specializes in generating huge amounts of random functions, so we should be covered there :big_smile:

Wasmtime GitHub notifications bot (Aug 27 2022 at 11:31):

afonso360 edited a comment on issue #4799:

Bugpoint's mutations are much less flexible as they replace for example int returning imstructions with iconst 0. Or they remove things. They never change or add things.

Sure, I wasn't proposing replacing all of icache's mutations, we still need those two sets of random replacing and randomly adding stuff. But we should have the removes somewhat covered without much effort.

In addition they don't affect function declarations at all if I recall correctly, despite this being one of the main things where icache can break.

I had a look at the icache source and it looks like we don't do those transforms either, so that would probably be a good addition.

And it would also be nice to have that on bugpoint, but only the remove half, and not the add random args part. I think we can share some code there.

You can always try bugpoint's mutations, but it probably won't be all that useful and in any case will need a large corpus of existing functions to work on.

The fuzzer specializes in generating huge amounts of random functions, so we should be covered there :big_smile:

Wasmtime GitHub notifications bot (Aug 29 2022 at 08:21):

bnjbvr commented on issue #4799:

Yes, that would make sense to try to reuse internals of bugpoint in the icache target indeed. Right now, only two "random" mutations are done in the icache target:

• either change an actual function target, and make sure it causes a cache hit when using the incremental cache.
• or change one thing in the IR, and cause a cache miss. This is done in a very adhoc way by looking for an instruction with one i32/i64 result, and change it to a constant value (and if it was a constant, just tweak it so it's the same +/- 1).

The second part could benefit from using bugpoint, in my opinion, by triggering more general changes. Ideally, the _change_ itself would be generated from libfuzz's binary input, instead of having our fuzz target generate this code for us: this would require generating both the Function as well as a valid location + instruction to replace one instruction in the IR.

Wasmtime GitHub notifications bot (Aug 29 2022 at 22:50):

jameysharp commented on issue #4799:

I like this idea!

Bugpoint's mutations are much less flexible as they replace for example int returning imstructions with iconst 0. Or they remove things. They never change or add things.

Sure, I wasn't proposing replacing all of icache's mutations, we still need those two sets of random replacing and randomly adding stuff. But we should have the removes somewhat covered without much effort.

Do mutators which add things test anything different than those that remove things? If you add something to fn1 to produce fn2, how is that different from removing something from fn2 to produce fn1?

If order does matter, you could consume a boolean from the fuzz input to decide whether to swap the two functions. But I'd hope order doesn't matter: my impression of the incremental cache is that there shouldn't be anything stateful at this level.

Similarly, I would think that replacing more complex instructions with less complex ones (e.g. iconst 0) tests the same things as the other way around. But that might not be general enough because maybe you want to go from one non-zero constant to another non-zero constant, or replace one binary operator with another, or one operand with another of the same type.

In any case I'm assuming that the underlying cranelift-fuzzgen function generator can produce the more complex function to begin with. But I like the idea of putting all the "add complicated stuff" effort into the function generator, and all the "remove stuff" effort in bugpoint.

Wasmtime GitHub notifications bot (Sep 02 2022 at 15:48):

akirilov-arm labeled issue #4799:

:wave: Hey,

I had this thought overnight and I'm not sure if there are any hidden downsides.

Both cranelift-icache and clif-util bugpoint change the input functions. cranelift-icache is a bit more random in its approach and bugpoint has some constraints to preserve. However bugpoint is quite developed and can mutate functions in a bunch of ways that icache can't, we could try to use those mutators in icache and check if it breaks the cache or not.

cc: @bjorn3 @bnjbvr @jameysharp

Wasmtime GitHub notifications bot (Sep 02 2022 at 15:48):

akirilov-arm labeled issue #4799:

:wave: Hey,

I had this thought overnight and I'm not sure if there are any hidden downsides.

Both cranelift-icache and clif-util bugpoint change the input functions. cranelift-icache is a bit more random in its approach and bugpoint has some constraints to preserve. However bugpoint is quite developed and can mutate functions in a bunch of ways that icache can't, we could try to use those mutators in icache and check if it breaks the cache or not.

cc: @bjorn3 @bnjbvr @jameysharp

Last updated: Dec 23 2024 at 12:05 UTC