Wasmtime GitHub notifications bot (Jun 03 2020 at 19:03): Wasmtime GitHub notifications bot (Jun 03 2020 at 19:03):cfallin opened Issue #1813:
After #1805, we're seeing test failures like the following in
debug::lldb::test_debug_dwarf_ptr:<details>
test debug::lldb::test_debug_dwarf_ptr ... FAILED failures: ---- debug::lldb::test_debug_dwarf_ptr stdout ---- thread 'main' panicked at 'didn't pass check > (lldb) target create "/home/runner/work/wasmtime/wasmtime/target/debug/wasmtime" > Current executable set to '/home/runner/work/wasmtime/wasmtime/target/debug/wasmtime' (x86_64). > (lldb) settings set -- target.run-args "-g" "--opt-level" "0" "tests/all/debug/testsuite/reverse-str.wasm" > (lldb) command source -s 0 '/tmp/.tmpmh81yA' > Executing commands in '/tmp/.tmpmh81yA'. > (lldb) b reverse-str.c:9 > Breakpoint 1: no locations (pending). ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Matched #0: \bBreakpoint 1: no locations \(pending\) > WARNING: Unable to resolve breakpoint to any actual locations. > (lldb) r > 1 location added to breakpoint 1 > Process 14667 stopped > * thread #1, name = 'wasmtime', stop reason = breakpoint 1.1 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~ Matched #1: \bstop reason = breakpoint 1\.1\b > frame #0: 0x00007ffff7fcf053 JIT(0x5555571bdd70)`reverse(s=(__ptr = 66562), len=13) at reverse-str.c:9:12 ^~~~~~~~ Matched #2: \bframe \#0\b ^~~~~~~~~~~~~~~~~~ Matched #3: \breverse\(s=\(__ptr = Missed #4: "Hello, world\." > > Process 14667 launched: '/home/runner/work/wasmtime/wasmtime/target/debug/wasmtime' (x86_64) > (lldb) p __vmctx->set(),&*s ', tests/all/debug/lldb.rs:52:5</details>
We suspect the version-bump of some dependency caused this, though we aren't yet sure why (and we cannot reproduce locally). We will disable the test for now but we need to investigate and re-enable.
Wasmtime GitHub notifications bot (Jun 03 2020 at 20:27):cfallin commented on Issue #1813:
We're temporarily avoiding CI failures by pinning versions in
Cargo.lockin #1811, but we should resolve the underlying issue and the update the lock file to the latest versions.
Wasmtime GitHub notifications bot (Jun 03 2020 at 20:28):cfallin labeled Issue #1813:
After #1805, we're seeing test failures like the following in
debug::lldb::test_debug_dwarf_ptr:<details>
test debug::lldb::test_debug_dwarf_ptr ... FAILED failures: ---- debug::lldb::test_debug_dwarf_ptr stdout ---- thread 'main' panicked at 'didn't pass check > (lldb) target create "/home/runner/work/wasmtime/wasmtime/target/debug/wasmtime" > Current executable set to '/home/runner/work/wasmtime/wasmtime/target/debug/wasmtime' (x86_64). > (lldb) settings set -- target.run-args "-g" "--opt-level" "0" "tests/all/debug/testsuite/reverse-str.wasm" > (lldb) command source -s 0 '/tmp/.tmpmh81yA' > Executing commands in '/tmp/.tmpmh81yA'. > (lldb) b reverse-str.c:9 > Breakpoint 1: no locations (pending). ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Matched #0: \bBreakpoint 1: no locations \(pending\) > WARNING: Unable to resolve breakpoint to any actual locations. > (lldb) r > 1 location added to breakpoint 1 > Process 14667 stopped > * thread #1, name = 'wasmtime', stop reason = breakpoint 1.1 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~ Matched #1: \bstop reason = breakpoint 1\.1\b > frame #0: 0x00007ffff7fcf053 JIT(0x5555571bdd70)`reverse(s=(__ptr = 66562), len=13) at reverse-str.c:9:12 ^~~~~~~~ Matched #2: \bframe \#0\b ^~~~~~~~~~~~~~~~~~ Matched #3: \breverse\(s=\(__ptr = Missed #4: "Hello, world\." > > Process 14667 launched: '/home/runner/work/wasmtime/wasmtime/target/debug/wasmtime' (x86_64) > (lldb) p __vmctx->set(),&*s ', tests/all/debug/lldb.rs:52:5</details>
We suspect the version-bump of some dependency caused this, though we aren't yet sure why (and we cannot reproduce locally). We will disable the test for now but we need to investigate and re-enable.
Wasmtime GitHub notifications bot (Jun 03 2020 at 20:28):cfallin commented on Issue #1813:
cc @yurydelendik
Wasmtime GitHub notifications bot (Jun 03 2020 at 21:16):alexcrichton commented on Issue #1813:
With a green master it's clear that
cargo update -p backtraceis causing the regression here. I can reproduce this locally, and the problem is that LLDB prints:error: Can't run the expression locally: Interpreter doesn't handle one of the expression's opcodeswhen we try to execute
p __vmctx->set(),&*sI'm not really sure how the backtrace crate is related to this. Even the gimli/addr2line/object/etc updates don't seem to affect wasmtime since they're all on versions that wasmtime itself isn't using, only the backtrace crate is using them.
@yurydelendik do you know where that error message may be coming from?
Wasmtime GitHub notifications bot (Jun 03 2020 at 21:28):yurydelendik commented on Issue #1813:
@alexcrichton "backtrace" now indirectly depends on "gimli 0.21", I'm thinking it is what is giving an issue. I'm working on upgrading our "gimli 0.20" atm, though not sure why it is giving "Interpreter doesn't handle one of the expression's opcodes".
Wasmtime GitHub notifications bot (Jun 03 2020 at 21:32):alexcrichton commented on Issue #1813:
I'm not sure why the gimli 0.21 dep would cause any issues though? That's exclusively used by the
backtracecrate (everything else in wasmtime is using 0.20), so is it like it's mere presence is causing issues?
Wasmtime GitHub notifications bot (Jun 03 2020 at 21:35):yurydelendik commented on Issue #1813:
I'm not sure why the gimli 0.21 dep would cause any issues though?
Me neither. But I would like to upgrade it anyway, but it is not easy. Too much API refactoring happened between 0.20 -> 0.21 .
Wasmtime GitHub notifications bot (Jun 03 2020 at 21:38):yurydelendik commented on Issue #1813:
do you know where that error message may be coming from?
It is likely LLDB -- it cannot decode generated by wasmtime DWARF expression
Wasmtime GitHub notifications bot (Jun 03 2020 at 21:41):yurydelendik edited a comment on Issue #1813:
do you know where that error message may be coming from?
It is likely LLDB -- it cannot decode generated by wasmtime DWARF expression
P.S. it is reason I'm connecting it with gimli
Wasmtime GitHub notifications bot (Jun 04 2020 at 19:54):yurydelendik commented on Issue #1813:
It is limited to Linux platforms. Bisect points for https://github.com/rust-lang/backtrace-rs/commit/5ad0b762eb84de27ee2917b61356786f3ee8260b
Wasmtime GitHub notifications bot (Jun 04 2020 at 20:58):alexcrichton commented on Issue #1813:
At this point it sounds like this is exposing a preexisting bug. The only real change in that commit which would affect this is that libbacktrace's C code is gone, and some new Rust code is pulled in. For these tests and LLDB I don't think any of that code is executed, but presumably the DWARF for wasmtime has changed, perhaps significantly, as a result.
Would it make sense to try to track this down not as a regression in the
backtracecrate but instead as a possible bug in wasmtime's emission of dwarf information? (or LLDB's interpretation of it?)
Wasmtime GitHub notifications bot (Jun 04 2020 at 21:04):yurydelendik commented on Issue #1813:
@alexcrichton the problem is related to LLDB only (works on GDB). Running any
p any_call()inside wasmtime debug session, without even JIT code, causes that too:(lldb) b wasmtime.rs:63 (lldb) r (lldb) p set_vmctx_memory(0)Which fails with the same error.
Maybe one of the new modules contains unsupported DWARF data?
Wasmtime GitHub notifications bot (Jun 04 2020 at 21:11):yurydelendik edited a comment on Issue #1813:
@alexcrichton the problem is related to LLDB only (works on GDB). Running any
p any_call()inside wasmtime debug session, without even JIT code, causes that too:(lldb) b wasmtime.rs:63 (lldb) r (lldb) p set_vmctx_memory(0)Which fails with the same error.
Maybe one of the new modules contains unsupported DWARF data?
P.S. also works in release mode, e.g.
cargo test --release test_debug_dwarf_ptr -- --ignored
Wasmtime GitHub notifications bot (Jun 04 2020 at 22:00):yurydelendik commented on Issue #1813:
Adding to Cargo.toml:
[profile.dev.package.backtrace] debug = 0allows test to past.
Tested with lldb-10 for Ubuntu -- it works there too. (It was working for lldb-9 on MacOSX, not on Ubunty)
If CI will start using lldb-10, the problem will go away.
Wasmtime GitHub notifications bot (Jun 04 2020 at 22:04):alexcrichton commented on Issue #1813:
Nice find! I'd be fine taking care of the CI bits if you want to check in that fix to Cargo.toml?
Wasmtime GitHub notifications bot (Jun 04 2020 at 22:07):yurydelendik commented on Issue #1813:
if you want to check in that fix to Cargo.toml?
Sorry, there are two solutions above: fix in Cargo.toml or lldb-10. Moving CI to lldb-10 will completely fix the issue.
Wasmtime GitHub notifications bot (Jun 04 2020 at 22:07):yurydelendik edited a comment on Issue #1813:
if you want to check in that fix to Cargo.toml?
Sorry, there are two solutions above: fix in Cargo.toml or lldb-10. Moving CI to lldb-10 will completely fix the issue (without Cargo.toml change).
Wasmtime GitHub notifications bot (Jun 04 2020 at 22:08):yurydelendik edited a comment on Issue #1813:
Adding to Cargo.toml:
[profile.dev.package.backtrace] debug = 0allows test to past.
Tested with lldb-10 for Ubuntu -- it works there too. (It was working for lldb-9 on MacOSX, not on Ubunty)
If CI will start using lldb-10, the problem will go away.
Wasmtime GitHub notifications bot (Jun 12 2020 at 17:52):alexcrichton commented on Issue #1813:
So I thought LLDB 10 would be pretty easy to install from https://github.com/llvm/llvm-project/releases/tag/llvmorg-10.0.0, but unfortunately those tarballs don't contain lldb at all. They did for LLVM 9 though (?)
Wasmtime GitHub notifications bot (Jun 15 2020 at 22:41):fitzgen commented on Issue #1813:
Note that https://github.com/bytecodealliance/wasmtime/pull/1832 is going to update
backtrace, but is adding# Cargo.toml [profile.dev.package.backtrace] debug = false # FIXME(#1813)to temporarily work around these failures.
We should remove that when properly fixing this bug.
Last updated: Dec 23 2024 at 12:05 UTC