Stream: git-wasmtime

Topic: wasmtime / issue #4781 cranelift: Allow testing trap code...

Wasmtime GitHub notifications bot (Aug 25 2022 at 09:40):

afonso360 opened issue #4781:

:wave: Hey,

Feature

This was brought up by @dheaton-arm on zulip. We should add a trap comparison operator to our CLIF test suite.

Benefit

As @cfallin mentions in the thread we already have reasonable coverage for traps via the wasm test suite.

However, this is still useful to improve the CLIF interpreter, to ensure that we are consistent in emitting the appropriate traps at the right time.

This could further be used in the fuzzer. The fuzzer currently stops executing any input, as soon as the interpreter traps.
This is overly restrictive since we could allow a DivByZero to execute in the backend and ensure that it comes up with the same trap. Other traps are also useful to check, but not all! We don't want to let a stack overflow keep executing since that can overwrite some memory and do terrible things.

I think this is also important if we want to start using the interpreter for const code execution, since we do need to ensure that we produce the same traps, and enabling it in the fuzzer is the best way I can think about it currently.

Implementation

We can add a trap handler either directly in cranelift-jit or the trampolines in the runtest suite. I'm not too familiar with how traps work in general to be able to expand on this.

Alternatives

So far I've been writing tests that trap in the interpreter test suite, and that is a viable option. But its not very scalable.

A very out of the box option would be to run wasmtime with the clif interpreter and use the wasm test suite and fuzzers to do this! I would love to do this, but right now its a bit too out there.

Wasmtime GitHub notifications bot (Aug 25 2022 at 09:41):

afonso360 edited issue #4781:

:wave: Hey,

Feature

This was brought up by @dheaton-arm on zulip. We should add a trap comparison operator to our CLIF test suite.

Benefit

As @cfallin mentions in the thread we already have reasonable coverage for traps via the wasm test suite.

However, this is still useful to improve the CLIF interpreter, to ensure that we are consistent in emitting the appropriate traps at the right time.

This could further be used in the clif differential fuzzer. The fuzzer currently stops executing any input, as soon as the interpreter traps.
This is overly restrictive since we could allow a DivByZero to execute in the backend and ensure that it comes up with the same trap. Other traps are also useful to check, but not all! We don't want to let a stack overflow keep executing since that can overwrite some memory and do terrible things.

I think this is also important if we want to start using the interpreter for const code execution, since we do need to ensure that we produce the same traps, and enabling it in the fuzzer is the best way I can think about it currently.

Implementation

We can add a trap handler either directly in cranelift-jit or the trampolines in the runtest suite. I'm not too familiar with how traps work in general to be able to expand on this.

Alternatives

So far I've been writing tests that trap in the interpreter test suite, and that is a viable option. But its not very scalable.

A very out of the box option would be to run wasmtime with the clif interpreter and use the wasm test suite and fuzzers to do this! I would love to do this, but right now its a bit too out there.

Wasmtime GitHub notifications bot (Sep 02 2022 at 15:40):

akirilov-arm labeled issue #4781:

:wave: Hey,

Feature

This was brought up by @dheaton-arm on zulip. We should add a trap comparison operator to our CLIF test suite.

Benefit

As @cfallin mentions in the thread we already have reasonable coverage for traps via the wasm test suite.

However, this is still useful to improve the CLIF interpreter, to ensure that we are consistent in emitting the appropriate traps at the right time.

This could further be used in the clif differential fuzzer. The fuzzer currently stops executing any input, as soon as the interpreter traps.
This is overly restrictive since we could allow a DivByZero to execute in the backend and ensure that it comes up with the same trap. Other traps are also useful to check, but not all! We don't want to let a stack overflow keep executing since that can overwrite some memory and do terrible things.

I think this is also important if we want to start using the interpreter for const code execution, since we do need to ensure that we produce the same traps, and enabling it in the fuzzer is the best way I can think about it currently.

Implementation

We can add a trap handler either directly in cranelift-jit or the trampolines in the runtest suite. I'm not too familiar with how traps work in general to be able to expand on this.

Alternatives

So far I've been writing tests that trap in the interpreter test suite, and that is a viable option. But its not very scalable.

A very out of the box option would be to run wasmtime with the clif interpreter and use the wasm test suite and fuzzers to do this! I would love to do this, but right now its a bit too out there.

Wasmtime GitHub notifications bot (Sep 02 2022 at 15:40):

akirilov-arm labeled issue #4781:

:wave: Hey,

Feature

This was brought up by @dheaton-arm on zulip. We should add a trap comparison operator to our CLIF test suite.

Benefit

As @cfallin mentions in the thread we already have reasonable coverage for traps via the wasm test suite.

However, this is still useful to improve the CLIF interpreter, to ensure that we are consistent in emitting the appropriate traps at the right time.

This could further be used in the clif differential fuzzer. The fuzzer currently stops executing any input, as soon as the interpreter traps.
This is overly restrictive since we could allow a DivByZero to execute in the backend and ensure that it comes up with the same trap. Other traps are also useful to check, but not all! We don't want to let a stack overflow keep executing since that can overwrite some memory and do terrible things.

I think this is also important if we want to start using the interpreter for const code execution, since we do need to ensure that we produce the same traps, and enabling it in the fuzzer is the best way I can think about it currently.

Implementation

We can add a trap handler either directly in cranelift-jit or the trampolines in the runtest suite. I'm not too familiar with how traps work in general to be able to expand on this.

Alternatives

So far I've been writing tests that trap in the interpreter test suite, and that is a viable option. But its not very scalable.

A very out of the box option would be to run wasmtime with the clif interpreter and use the wasm test suite and fuzzers to do this! I would love to do this, but right now its a bit too out there.

Wasmtime GitHub notifications bot (Sep 02 2022 at 15:40):

akirilov-arm labeled issue #4781:

:wave: Hey,

Feature

This was brought up by @dheaton-arm on zulip. We should add a trap comparison operator to our CLIF test suite.

Benefit

As @cfallin mentions in the thread we already have reasonable coverage for traps via the wasm test suite.

However, this is still useful to improve the CLIF interpreter, to ensure that we are consistent in emitting the appropriate traps at the right time.

This could further be used in the clif differential fuzzer. The fuzzer currently stops executing any input, as soon as the interpreter traps.
This is overly restrictive since we could allow a DivByZero to execute in the backend and ensure that it comes up with the same trap. Other traps are also useful to check, but not all! We don't want to let a stack overflow keep executing since that can overwrite some memory and do terrible things.

I think this is also important if we want to start using the interpreter for const code execution, since we do need to ensure that we produce the same traps, and enabling it in the fuzzer is the best way I can think about it currently.

Implementation

We can add a trap handler either directly in cranelift-jit or the trampolines in the runtest suite. I'm not too familiar with how traps work in general to be able to expand on this.

Alternatives

So far I've been writing tests that trap in the interpreter test suite, and that is a viable option. But its not very scalable.

A very out of the box option would be to run wasmtime with the clif interpreter and use the wasm test suite and fuzzers to do this! I would love to do this, but right now its a bit too out there.

Last updated: Dec 23 2024 at 12:05 UTC