Wasmtime GitHub notifications bot (Sep 02 2022 at 14:33): Wasmtime GitHub notifications bot (Sep 02 2022 at 14:33):alexcrichton labeled issue #4852:
Currently the spec interpreter is very rarely used when differential fuzzing due to the current constraints of the integration. Some new features are required (all of which I think require writing some OCaml) to support better integration:
- The instantiation and execution phases of a wasm instance need to be split apart. Right now instantiation always feigns success which means that wasm modules which fail to instantiate (with an out-of-bounds element segment for example) fail later than other engines. This is why the spec fuzzer disallows tables
- Currently the integration with the spec test fuzzing does not support more than one function because the current OCaml harness I believe executes the first function. Instead the function name should get threaded through here and executed.
- Getting the values of linear memory and/or wasm globals is currently unimplemented.
- Less critical but still nice-to-have, error messages are not validated to ensure that Wasmtime and the spec interpreter produce the same flavor of trap.
cc @conrad-watt IIRC you've been working with @abrown to make a more generalized harness and I think the intention was to land it in https://github.com/bytecodealliance/wasm-spec-interpreter but if you're willing I think it'd also be good to get the updates into this repository ahead of that if there are orthogonal issues there to tackle (I'm unsure of its status, we could also fill out that repo and then replace the in-tree copy of the repo here with the external dependency)
Wasmtime GitHub notifications bot (Sep 02 2022 at 14:33):alexcrichton opened issue #4852:
Currently the spec interpreter is very rarely used when differential fuzzing due to the current constraints of the integration. Some new features are required (all of which I think require writing some OCaml) to support better integration:
- The instantiation and execution phases of a wasm instance need to be split apart. Right now instantiation always feigns success which means that wasm modules which fail to instantiate (with an out-of-bounds element segment for example) fail later than other engines. This is why the spec fuzzer disallows tables
- Currently the integration with the spec test fuzzing does not support more than one function because the current OCaml harness I believe executes the first function. Instead the function name should get threaded through here and executed.
- Getting the values of linear memory and/or wasm globals is currently unimplemented.
- Less critical but still nice-to-have, error messages are not validated to ensure that Wasmtime and the spec interpreter produce the same flavor of trap.
cc @conrad-watt IIRC you've been working with @abrown to make a more generalized harness and I think the intention was to land it in https://github.com/bytecodealliance/wasm-spec-interpreter but if you're willing I think it'd also be good to get the updates into this repository ahead of that if there are orthogonal issues there to tackle (I'm unsure of its status, we could also fill out that repo and then replace the in-tree copy of the repo here with the external dependency)
Wasmtime GitHub notifications bot (Sep 02 2022 at 14:34):github-actions[bot] commented on issue #4852:
Subscribe to Label Action
cc @fitzgen
<details>
This issue or pull request has been labeled: "fuzzing"Thus the following users have been cc'd because of the following labels:
- fitzgen: fuzzing
To subscribe or unsubscribe from this label, edit the <code>.github/subscribe-to-label.json</code> configuration file.
Learn more.
</details>
Wasmtime GitHub notifications bot (Sep 02 2022 at 14:40):conrad-watt commented on issue #4852:
@abrown, I believe that the new interface I implemented for the interpreter here addresses the first three bullet points in the OP. How much effort would it be to merge this in?
Wasmtime GitHub notifications bot (Sep 02 2022 at 14:42):conrad-watt edited a comment on issue #4852:
@abrown, I believe that the new interface I implemented for the interpreter here addresses the first three bullet points in the OP. How much effort would it be to merge this in (edit: possibly with appropriate cleanup of the tests and legacy API)?
Wasmtime GitHub notifications bot (Sep 02 2022 at 15:24):abrown commented on issue #4852:
Sure, I like the idea of trying it out here first. I can work in the changes we made next week and submit a PR.
Wasmtime GitHub notifications bot (Sep 12 2022 at 23:38):conrad-watt commented on issue #4852:
With the merging of https://github.com/bytecodealliance/wasmtime/pull/4881, where are we on this now?
Wasmtime GitHub notifications bot (Sep 13 2022 at 02:31):abrown commented on issue #4852:
It looks like mostly done; maybe matching up the error messages to the Wasmtime trap equivalent is all that is left?
Wasmtime GitHub notifications bot (Sep 13 2022 at 14:24):alexcrichton closed issue #4852:
Currently the spec interpreter is very rarely used when differential fuzzing due to the current constraints of the integration. Some new features are required (all of which I think require writing some OCaml) to support better integration:
- The instantiation and execution phases of a wasm instance need to be split apart. Right now instantiation always feigns success which means that wasm modules which fail to instantiate (with an out-of-bounds element segment for example) fail later than other engines. This is why the spec fuzzer disallows tables
- Currently the integration with the spec test fuzzing does not support more than one function because the current OCaml harness I believe executes the first function. Instead the function name should get threaded through here and executed.
- Getting the values of linear memory and/or wasm globals is currently unimplemented.
- Less critical but still nice-to-have, error messages are not validated to ensure that Wasmtime and the spec interpreter produce the same flavor of trap.
cc @conrad-watt IIRC you've been working with @abrown to make a more generalized harness and I think the intention was to land it in https://github.com/bytecodealliance/wasm-spec-interpreter but if you're willing I think it'd also be good to get the updates into this repository ahead of that if there are orthogonal issues there to tackle (I'm unsure of its status, we could also fill out that repo and then replace the in-tree copy of the repo here with the external dependency)
Wasmtime GitHub notifications bot (Sep 13 2022 at 14:24):alexcrichton commented on issue #4852:
Indeed, yeah, I think that the spec interpreter could have something similar to v8 where it checks trap messages, but that's not critical since we already test against v8 and wasmi as well, so I think this can be closed.
Last updated: Dec 23 2024 at 13:07 UTC