Stream: wasmtime

Topic: assert_malformed and assert_invalid in test logs


view this post on Zulip Till Schneidereit (Feb 03 2020 at 13:31):

I just saw that the test logs contain lots of mentions of assert_malformed and assert_invalid with unexpected results, but tests marked as passing. See e.g. this log: https://github.com/bytecodealliance/wasmtime/runs/421975382?check_suite_focus=true#step:12:515

Is this something where our test harness just prints things that aren't important, or something where we actually have wrong results, but the harness somehow swallows them?

cc @Nick Fitzgerald, because I see a lot of multi-value in there.

Standalone JIT-style runtime for WebAssembly, using Cranelift - bytecodealliance/wasmtime

view this post on Zulip fitzgen (he/him) (Feb 03 2020 at 18:44):

My understanding was that these are tests that contain Wasm that's expected to fail (eg assert_malformed is asserting that the given Wasm is malformed, etc) and that we log the errors. We can probably stop logging them, since they're expected. The one complication, IIRC, is that the spec testsuite has specific error messages it is testing for, and we don't always match the message exactly. I'll take a peek again to confirm this.

cc @Yury Delendik -- I think you originally wrote the spec testsuite integration?

view this post on Zulip Till Schneidereit (Feb 03 2020 at 18:46):

I might misunderstand the logs, but it seems to me like the tests expect a specific error, but we're getting a different one

view this post on Zulip fitzgen (he/him) (Feb 03 2020 at 18:48):

assert_invalid: expected type mismatch, got stack item type does not match block item type (at offset 31)

this one, for example, looks like a disagreement for what the error message should be

view this post on Zulip fitzgen (he/him) (Feb 03 2020 at 18:48):

but there certainly could be others that are not so seemingly benign

view this post on Zulip fitzgen (he/him) (Feb 03 2020 at 19:05):

Looks like we have https://github.com/bytecodealliance/wasmtime/issues/492 on file

Repro steps cargo test --all --exclude lightbeam --exclude wasmtime-wasi-c --exclude wasmtime-py -- --nocapture Expected results The Wasm test suite tests should all pass. Actual results The Wasm t...

view this post on Zulip Till Schneidereit (Feb 03 2020 at 19:07):

ah, good, I hadn't seen that


Last updated: Dec 23 2024 at 14:03 UTC