Steve Williams (Sep 30 2021 at 16:26): Steve Williams (Sep 30 2021 at 16:26):<><> lldbg update :
https://github.com/adv-sw/lldbg
IDE coming along nicely, lldb won't step into wasm c++ or identify in scope globals. Unsure why.
Steve Williams (Sep 30 2021 at 16:40):Do you guys do anything with the name mangling which is different on Windows than from Linux ? Maybe it's that.
Steve Williams (Oct 01 2021 at 09:33):seems to be relative path issue. must start debugger from correct directory for some symbols to resolve. doh.
Steve Williams (Oct 01 2021 at 11:05):
figured previous out - was relative path issue. next up : wasm class instances are obscured.
note wrapper around App & its class members opaque behind __ptr.
can we do better ?
does rust have the same issue ?
Steve Williams (Oct 01 2021 at 12:09):seems related to this : https://github.com/bytecodealliance/wasmtime/issues/1896
Steve Williams (Oct 01 2021 at 14:30):from yury: https://github.com/bytecodealliance/wasmtime/pull/1482
Steve Williams (Oct 01 2021 at 14:31):once I get my head around this, as we have custom lldb front end, hopefully can hide all that away.
Steve Williams (Oct 01 2021 at 16:56):got it, I think ... next to see if that can be folded away. now.png
Steve Williams (Oct 01 2021 at 17:15):lldbg update : pp command used to dereference wasmtime pointers added.
Steve Williams (Oct 02 2021 at 10:20):next up automating this. provided alias expands as follows :
p __vmctx->set(), *thing
unsure what that means :)
you're calling that built in function with the sandbox pointer & it's returning diagnostics as to what's at that address, or its calculating a system memory address which lldb then uses to call up what's there ?
Steve Williams (Oct 02 2021 at 12:56):getting there with a little help from lldb guys
p __vmctx->set()
binds * operator to all wasmtime pointers in current stack frame
p *thing
calls that operator you just injected, returning regular memory pointer, which lldb then looks up ?
Steve Williams (Oct 02 2021 at 14:28):
Web Assembly pointer demangled & presented as expected in debugger.
Still to do - dereferencing of that.
Steve Williams (Oct 02 2021 at 17:41):almost sane now :)
added wasm filtering of things regular users won't need to see & demangle / lookup of wasmtime pointers to their actual equivalents.
basic testing thus far so might still be refinements required but fairly pleased with that.
the wasm filtering can be disabled/enabled as a compile define option.
Steve Williams (Oct 03 2021 at 10:07):
current issues :
__app static global not coming thru.
"this" not populated properly.
happy to do what I can to resolve as need this production quality.
so I suppose I need to take a look at what the dwarf generate thing is doing.
if anyone can point me at it, that'd be appreciated.
Steve Williams (Oct 04 2021 at 15:33):repro: http://advance-software.com/misc/test.zip
getting this out :
warning: `this' is not accessible (substituting 0)
unsure where that's coming from.
Steve Williams (Oct 04 2021 at 15:34):put breakpoint on m_i = i; in constructor in test.cpp
note "this" empty - either lldbg to observe directly or lldb pp *this after configuring the macro.
Steve Williams (Oct 04 2021 at 15:42):will need very recent wasmtime if reproducing on windows for jit debugging to work. trunk is fine, built this morning.
Steve Williams (Oct 04 2021 at 15:42):suspect we can repro on linux too, but haven't tested that yet.
Steve Williams (Oct 04 2021 at 15:45):oh yeah & need lldb patching for jit to work on windows too coz it's broken in their tree & they won't accept patch without a test I haven't got time to put together.
Steve Williams (Oct 04 2021 at 15:46):suspect this can be resolved linux to avoid lldb jit broken windows for those who can't be bothered patching & building their own llvm tree.
Steve Williams (Oct 04 2021 at 15:47):windows lldb patch here to fix up jit : https://reviews.llvm.org/D110066
Steve Williams (Oct 04 2021 at 16:13):if this all resolves, I'll put that lldb test together for them. grumble grumble :)
Steve Williams (Oct 04 2021 at 17:48):even simpler repro : http://advance-software.com/misc/test.zip
removed wasi requirement. test.cpp in there is as simple as it gets. breakpoint in constructor, this not populated as expected.
Steve Williams (Oct 04 2021 at 19:41):confirmed linux (ubuntu).
repro updated: http://advance-software.com/misc/test.zip
change debug.sh lldbg to lldb to skip gui if desired.
... but lldbg confirmed working linux. pain free build. binary needs to be in your path.
Steve Williams (Oct 04 2021 at 19:42):https://github.com/adv-sw/lldbg
pull
cmake .
make
Steve Williams (Oct 04 2021 at 19:44):set WASI_SDK in make_t.sh to where you have it. that's it. breakpoint in constructor. inspect this.
Steve Williams (Oct 05 2021 at 13:09):tested native compile on linux to verify its not lldbg gui or lldb API messing up. works as expected, this deferences properly. so its a wasmtime issue. hence will file in issues on github so we can track.
Steve Williams (Oct 05 2021 at 13:15):effectively a production stop c++ wasm issue. yeah you can get around not being able to inspect this in a debugger but its not something you'd want to take forward.
also had issue with static globals with double underscore being absent. that needs fixing too.
I'll do it once I figure out where the code is though rust noob & this is advanced work.
Steve Williams (Oct 05 2021 at 13:16):surprised by the silence - anyone able to reproduce ?
Steve Williams (Oct 05 2021 at 13:18):
linux repro.
Steve Williams (Oct 05 2021 at 13:38):so, this is what I need to get my head around ?
https://github.com/bytecodealliance/wasmtime/tree/main/crates/cranelift/src/debug/transform
Steve Williams (Oct 05 2021 at 13:39):when I said I'll fix it, that of course means input & guidance welcome as I've never touched this stuff before :)
Steve Williams (Oct 05 2021 at 13:57):that's some scary code. no specific references to "this"
Steve Williams (Oct 05 2021 at 14:07):think I file this in cranelift not wasmtime but happy to accept direction.
Steve Williams (Oct 05 2021 at 14:12):test updated: have make_native & make_wasm build scripts & dbg_native & debug_wasm run scripts.
essentially the same - just a tidy.
Steve Williams (Oct 05 2021 at 14:12):advance-software.com/misc/test.zip
Steve Williams (Oct 05 2021 at 14:21):https://github.com/bytecodealliance/wasmtime/issues/3419
Steve Williams (Oct 05 2021 at 15:42):seem to remember your custom clang has its own weird flag to turn optimizations off. can't seem to find that.
Steve Williams (Oct 05 2021 at 15:53):should I be using that or llvm trunk clang ? perhaps that's an irrelevant aside. don't understand problem yet.
Steve Williams (Oct 05 2021 at 15:55):what version of dwarf are you using plz
Steve Williams (Oct 05 2021 at 15:59):I'm starting at zero with this, generating debug info isn't something I've ever done before.
Steve Williams (Oct 05 2021 at 16:12):I ran a dwarfdump on native & wasm from ubuntu.
6407 test.d
3841 test.wasm.d
wasm dwarfdump half the size of native. have no idea if this is red herring or the issue, just learning my way around.
Steve Williams (Oct 05 2021 at 16:35):that's my first reading of a dwarfdump, so take this as noob input - both look correct. I can see this defined in constructor & the connectivity to its class definition. so the dwarf data knows what it is. what's weird is you don't seem to do anything special with 'this' so I wonder where its failing.
Steve Williams (Oct 05 2021 at 16:37):maybe lldb is blocking some reserved words resolving in their gdb/jit interface for some reason. might look there for clues. I can at least inspect what you're sending.
Steve Williams (Oct 06 2021 at 09:43):without gui in the way :
lldb -- wasmtime -g g:\dev\test\test.wasm
(lldb) target create "wasmtime"
Current executable set to 'wasmtime' (x86_64).
(lldb) settings set -- target.run-args "-g" "g:\\dev\\test\\test.wasm"
(lldb) b test.cpp:7
Breakpoint 1: no locations (pending).
WARNING: Unable to resolve breakpoint to any actual locations.
(lldb) r
Process 13564 launched: 'G:\dev\test\wasmtime.exe' (x86_64)
1 location added to breakpoint 1
Process 13564 stopped
Thing::Thing(this=(__ptr = 131088), i=5) at test.cpp:7:13
4 public:
5 Thing(int i)
6 {
-> 7 m_i = i;
8 }
9
10 int m_i;
(lldb) command regex pp 's/(.+)/p __vmctx->set(),%1/'
(lldb) pp *this
warning: this' is not accessible (substituting 0) Steve Williams (Oct 06 2021 at 10:21):tried gdb instead of lldb ... progress I think ... it lets us access "this" ... however doesn't seem to like __vmctx->set() so can't decode.
Thread 1 "wasmtime" hit Breakpoint 1, Thing::Thing (this=..., i=5) at ./test.cpp:8
8 m_i = i;
(gdb) info locals
__vmctx = 0x5555560e53c0
(gdb) p __vmctx->set()
Aborted (core dumped)
x@dev:~/dev/test$ gdb --args wasmtime -g ./test.wasm
Reading symbols from wasmtime...
(gdb) b test.cpp:7
No source file named test.cpp.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (test.cpp:7) pending.
(gdb) r
Starting program: /home/x/dev/tools/wasmtime -g ./test.wasm
Thread 1 "wasmtime" hit Breakpoint 1, Thing::Thing (this=..., i=5) at ./test.cpp:8
8 m_i = i;
(gdb) p this
$1 = {__ptr = 67216}
Steve Williams (Oct 06 2021 at 10:34):not ultimately wanting to end up at gdb so think I stop there. it shows us (I think) wasmtime is presenting "this" properly & hence inability to access is an lldb bug.
Steve Williams (Oct 06 2021 at 13:35):oh what fun :)
https://discord.com/channels/636084430946959380/636732809708306432
Steve Williams (Oct 07 2021 at 11:27):ummm ...
when I look at Thing *thing I get this :
(WebAssemblyPtrWrapper<Thing>) $0 = (__ptr = 131088)
when I look at this, I don't get :
(WebAssemblyPtrWrapper<this>) $0 = (__ptr = 131088)
perhaps we should for consistency. You're not this, you're a wrapped this. Debugger tweaks can hide that away but at the data sharing level, perhaps we should have consistency.
that said, still working thru bug with lldb engineer. unsure at this stage if that's necessary or wild speculation on my part.
Steve Williams (Oct 07 2021 at 11:32):mumbling to self. not sure we can. confused.com
Steve Williams (Oct 07 2021 at 11:38):could we perhaps rename this to ^this (or some other) on the fly to work around issue where some debuggers (lldb) assumes reserved word "this" is a direct pointer ?
Steve Williams (Oct 07 2021 at 11:48):would love an experimental s/this/__this in dwarf emit :)
discussion progressing here : https://discord.com/channels/636084430946959380/636732809708306432
getting into specifics beyond my knowledge, so feel free to dive in to correct/engage
Steve Williams (Oct 07 2021 at 12:46):suggestion __vmctx->set() injects a "this" that points to the real address. would complement the __this mod above which would be the wrapped pointer.
Steve Williams (Oct 07 2021 at 12:54):in event stack frame has a __this, keep adding underscores until you find an available identifier (rare).
Steve Williams (Oct 07 2021 at 13:12):simulate.rs generate_vars seems to be the emit, so lets see if I can fail my way thru a little rust :)
Steve Williams (Oct 07 2021 at 13:43):in generate_vars I want :
if (name_id == "this")
name_id = "__this"
.. but I don't rust. at all. not really wanting to as lots on & I don't feel the need other than to patch this.
Steve Williams (Oct 07 2021 at 17:03):given I don't rust, thinking I'll have a go from the other end - see if I can get lldb to accept this without assumptions.
wouldn't hurt to try both approaches then we can select the most robust.
Steve Williams (Oct 07 2021 at 17:38):still thinking of running with ^this to avoid potential symbol clash. managed c++ uses this syntax too. unclear that's an issue.
unclear whether debugger will accept so we'd have to try it.
Steve Williams (Oct 07 2021 at 17:48):& if debugger has issues, given I'm heading in there anyway, we can resolve them. hence we can close this as we please.
I'm liking ^thing dereferencing to thing actual pointer. Similarly ^this to this. When we send naked this, as its a genuine pointer, lldb won't barf without modifications being necessary.
Thoughts welcome.
Steve Williams (Oct 07 2021 at 18:58):The issue is lldb in its current form expects 'this' and 'self' to be standard pointers. if they're not, it barfs.
Steve Williams (Oct 07 2021 at 19:01):gdb doesn't have this restriction, however whatever version is ubuntu mainline barfs on p __vmctx->set()
hence its not a workaround option unless anyone has the mood to fix that if its still an issue gdb trunk.
Steve Williams (Oct 07 2021 at 19:17):10.1 trying trunk now.
Steve Williams (Oct 07 2021 at 19:21):.
Steve Williams (Oct 07 2021 at 20:06):gdb trunk fails : https://pastebin.com/ztrK5g0p
stopping there as have lldb solution preference.
Steve Williams (Oct 08 2021 at 10:20):though it might be possible to mod lldb to run with an 'invalid' this pointer, I'm not clear that's the way.
wasmtime dwarf can & should send a resolved this pointer.
sandboxed this pointer can be sent as a special that debugger front end can merge so interface is intuitive & you see the sandboxed pointer whilst presenting the naked pointer. hence I think wasmtime bug. in rust code :(
surprised at lack of assistance here guys, this is a core flaw. if you can't debug c++ effectively, you're a toy.
Steve Williams (Oct 08 2021 at 10:27):found following - likely an irrelevant aside, but anyway : https://reviews.llvm.org/D78801?id=261780
Steve Williams (Oct 08 2021 at 11:02):progress. weirdly, running
(lldb) fr v
first seems to resolve the matter.
(lldb) target create "wasmtime"
Current executable set to 'wasmtime' (x86_64).
(lldb) settings set -- target.run-args "-g" ".\\test.wasm"
(lldb) b test.cpp:7
Breakpoint 1: no locations (pending).
WARNING: Unable to resolve breakpoint to any actual locations.
(lldb) r
Process 12588 launched: 'Z:\dev\wasm\wasmtime\target\release\wasmtime.exe' (x86_64)
1 location added to breakpoint 1
Process 12588 stopped
Thing::Thing(this=(__ptr = 131088), i=5) at test.cpp:7:13
4 public:
5 Thing(int i)
6 {
-> 7 m_i = i;
8 }
9
10 int m_i;
(lldb) p this
warning: this' is not accessible (substituting 0). Couldn't load 'this' because its value couldn't be evaluated Steve Williams (Oct 08 2021 at 12:24):experiment :
Process 10565 stopped
Thing::Thing(this=(__ptr = 67216), i=5) at test.cpp:8:13
5 public:
6 Thing(int i)
7 {
-> 8 m_i = i;
9 }
10
11 int m_i;
(lldb) fr v this
(WebAssemblyPtrWrapper<Thing>) this = (__ptr = 67216)
(lldb) p __vmctx->set()
warning: this' is not accessible (substituting 0)we can get the sandbox pointer but not resolve it.
Steve Williams (Oct 08 2021 at 12:25):rust programmers be like not my problem :)
... but it is coz if you can only use wasmtime effectively from rust, it ain't going to go far.
Steve Williams (Oct 08 2021 at 12:36):if we had a function that could accept a sandbox pointer & return the real one, that'd maybe be enough. we can't evaluate on "this" without modding lldb as self/this are special cased. fr v works where p fails bcoz it doesn't evaluate, just passes the value (I think), but we need an evaluate with current implementation to resolve.
Steve Williams (Oct 08 2021 at 12:38):still the issue of lldb not liking "this" in unexpected format which could bite in different scenarios.
Steve Williams (Oct 08 2021 at 12:39):so back to ^this or __this for sandboxed pointer. seems most robust solution without ripping debugger apart.
Steve Williams (Oct 08 2021 at 15:20):eval "{auto tmp = this; *tmp; }"
or some such maybe. perhaps a workaround if we can avoid a direct eval of 'this'. better to fix properly, but whatever works.
Steve Williams (Oct 08 2021 at 15:26):& who's to say a new reserved word used to evaluate current this isn't properly. a few options.
Steve Williams (Oct 08 2021 at 15:28):p __vmctx->this()
?
evaluating to resolved this pointer. for use on debuggers that require it.
Steve Williams (Oct 08 2021 at 15:33):or, preferable :
this evaluates to resolved.
p __vmctx->this()
evaluates to sandbox pointer. that way, lldb gets a nice, lovely valid this pointer as it expects & can thus provide full functionality.
would be a change in behaviour so perhaps new -g:lldb flag to enable & SDK equivalent.
Steve Williams (Oct 08 2021 at 15:40):other workarounds might include querying previous stack frame for what object's method is being presented - that's implicitly current this.
unsure if we can grab that. just an idea right now.
Steve Williams (Oct 08 2021 at 15:52):however if we're looking on previous stack frame, your __vmctx->set() stack frame magic won't be set up properly so that one might not be possible, unless you were to maintain 2 stack frames. if we have to mod, not sure this is the way to go but an option. we'd only need one var of previous stack frame maintained.
Steve Williams (Oct 08 2021 at 15:53):... which amounts to __vmctx->this()
however we get it, that's what we need. a bypass around asking lldb to evaluate this coz it can't coz its not a valid pointer.
Steve Williams (Oct 08 2021 at 15:56):we could mod lldb to be more flexible with its handling of this however I suspect that's non-trivial work & no way of knowing if patch would be accepted so could end up with lldb divergence. hence I'm saying wasmtime issue. understood issue, resolvable.
I'd complete the engineering but rust does my head in. I don't want to go there :)
y'all crazy :)
Steve Williams (Oct 09 2021 at 11:43):have started getting following error :
p __vmctx->set()
error: Couldn't lookup symbols:
set_vmctx_memory
unsure why. currently running clean wasmtime trunk.
Steve Williams (Oct 09 2021 at 11:47):also running lldb trunk so issue could be anywhere. think I have lldb 12 & some old wasmtime working but that's no good for development work.
Steve Williams (Oct 09 2021 at 19:13):windows working too following a recompile :) lldb 12.0.1 tested.
Last updated: Dec 23 2024 at 12:05 UTC