Stream: cranelift

Topic: ✔ Segfault when JIT calling DLL function


view this post on Zulip Tim Ngo (Oct 03 2022 at 09:28):

hey! I'm running into a segfault when calling a function defined in a DLL compiled from C++. I can successfully find and call the function using dlopen:

let lib = Library::open("SimpleDLL.dll").unwrap();
let fn_ptr: *const u8 = unsafe { lib.symbol("DisplayHelloFromMyDLL") }.unwrap();
let display_hello: fn() -> () = unsafe { std::mem::transmute(fn_ptr) };
display_hello(); // This works.

But when I add display_hello to the JITBuilder's symbols, build a call to the function, and execute the finalized main function, I get a segmentation fault.

jit_builder.symbols(vec![
    // These functions defined in Rust directly work fine.
    ("Base.print_int", builtin::print_int as *const u8),
    ("Base.alloc", builtin::alloc as *const u8),
    ("Base.dealloc", builtin::dealloc as *const u8),
    // This doesn't.
    ("DisplayHelloFromMyDLL", fn_ptr),
]);
let callee = self
    .module
    .declare_function(&name, Linkage::Import, &sig)
    .unwrap();
let local_callee = self.module.declare_func_in_func(callee, c.b.func);
let call = c.b.ins().call(local_callee, &args);

Here's the source for the DLL.

#include <stdio.h>
extern "C"
{
  __declspec(dllexport) void DisplayHelloFromMyDLL()
  {
    printf("Hello.\n");
  }
}

view this post on Zulip bjorn3 (Oct 03 2022 at 10:40):

cg_clif (a project of mine) seems to be having the same problem. Could you check that fn_ptr is not a null pointer? And if it isn't get a backtrace at the point of the crash using a debugger?

Cranelift based backend for rustc. Contribute to bjorn3/rustc_codegen_cranelift development by creating an account on GitHub.

view this post on Zulip Tim Ngo (Oct 03 2022 at 17:47):

When printing fn_ptr after looking up the symbol, it's equal to 0x7ff8c57412e9. When stepping through the execution of main() this is the instruction that the debugger crashes at: screenshot

view this post on Zulip bjorn3 (Oct 03 2022 at 17:49):

Can you look if there is any memory mapped at 0x7ff8c57412e9 and if so with which permissions?

view this post on Zulip Tim Ngo (Oct 03 2022 at 17:54):

I'm a bit of a novice with systems stuff. Could you point me to a reference for how to do that?

view this post on Zulip bjorn3 (Oct 03 2022 at 18:19):

It depends on which debugger you use. In any case I don't use windows for development, so I don't have any experience with windows debuggers.

view this post on Zulip Tim Ngo (Oct 03 2022 at 18:57):

I'm currently using LLDB, if that's familiar. I'll do some more digging and try to find that info.

view this post on Zulip Tim Ngo (Oct 03 2022 at 19:18):

Oh here's the backtrace message:

* thread #1, stop reason = Exception 0xc0000005 encountered at address 0x7ff8c58912e9:
User-mode data execution prevention (DEP) violation at location 0x7ff8c58912e9
  * frame #0: 0x00007ff8c58912e9

So this address isn't executable?

view this post on Zulip bjorn3 (Oct 03 2022 at 19:39):

Didn't know lldb worked on windows. I thought you had to do eg windbg. Could you try disassembling the first couple of instructions at this address to see if there is executable code not marked as executable or if there is no code there in the first place? Something like disassemble --start-address 0xdeadbeef --count 20 works for disassembling 20 instructions at 0xdeadbeef.

view this post on Zulip bjorn3 (Oct 03 2022 at 19:41):

By the way feel free to @ mention me in case I don't respond. I don't continuously monitor this, but do have push notifications for mentions enabled.

view this post on Zulip Tim Ngo (Oct 03 2022 at 19:48):

Thanks, I'm in no rush.

> disassemble --start-address 0x7ff8c58912e9 --count 1
error: Only part of a ReadProcessMemory or WriteProcessMemory request was completed.
error: Failed to disassemble memory at 0x7ff8c58912e9.

I think there are windows os-level protections of some sort?

view this post on Zulip bjorn3 (Oct 03 2022 at 20:09):

Could you try image dump sections? Not sure if it works on Windows.

view this post on Zulip bjorn3 (Oct 03 2022 at 20:10):

I think there are windows os-level protections of some sort?

It seems like that memory isn't mapped at all, not just mapped as non-executable.

view this post on Zulip Tim Ngo (Oct 03 2022 at 20:13):

Here's the section for the DLL. I can paste the other sections if needed

Sections for 'C:\Users\Tim\Projects\hubbub\target\debug\SimpleDLL.dll' (x86_64):
  SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
  ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
  0xffffffffffffffff regular  [0x0000000180000000-0x0000000180000400)* r--  0x00000000 0x00000400 0x00000000 SimpleDLL.dll.PECOFF header
  0x00000001 code             [0x0000000180001000-0x0000000180011000)* rwx  0x00000000 0x00000000 0xe00000a0 SimpleDLL.dll..textbss
  0x00000002 code             [0x0000000180011000-0x0000000180018da1)* r-x  0x00000400 0x00007e00 0x60000020 SimpleDLL.dll..text
  0x00000003 data             [0x0000000180019000-0x000000018001bbe7)* r--  0x00008200 0x00002c00 0x40000040 SimpleDLL.dll..rdata
  0x00000004 data             [0x000000018001c000-0x000000018001c8e8)* rw-  0x0000ae00 0x00000200 0xc0000040 SimpleDLL.dll..data
  0x00000005 data             [0x000000018001d000-0x000000018001f148)* r--  0x0000b000 0x00002200 0x40000040 SimpleDLL.dll..pdata
  0x00000006 data             [0x0000000180020000-0x0000000180020d03)* r--  0x0000d200 0x00000e00 0x40000040 SimpleDLL.dll..idata
  0x00000007 data             [0x0000000180021000-0x000000018002110a)* rw-  0x0000e000 0x00000200 0xc0000040 SimpleDLL.dll..msvcjmc
  0x00000008 data             [0x0000000180022000-0x0000000180022175)* r--  0x0000e200 0x00000200 0x40000040 SimpleDLL.dll..00cfg
  0x00000009 data             [0x0000000180023000-0x000000018002343c)* r--  0x0000e400 0x00000600 0x40000040 SimpleDLL.dll..rsrc
  0x0000000a regular          [0x0000000180024000-0x0000000180024259)* r--  0x0000ea00 0x00000400 0x42000040 SimpleDLL.dll..reloc

view this post on Zulip bjorn3 (Oct 03 2022 at 20:14):

Is there also a range close to the address that causes the crash when being called?

view this post on Zulip bjorn3 (Oct 03 2022 at 20:18):

Also are you using libloading 0.7.3 (the latest version)?

view this post on Zulip bjorn3 (Oct 03 2022 at 20:20):

Just reread your original message. You are using the dlopen crate. I assumed that you used the libloading crate. My bad.

view this post on Zulip Tim Ngo (Oct 03 2022 at 20:21):

Listing all exes/libs, the main exe is the closest

> image list
[  0] 6571EF52-F94D-4FCA-8132-A4FC69AF8E87-00000BFB 0x00007ff71f4d0000 C:\Users\Tim\Projects\hubbub\target\debug\hubbub.exe
[  1] 51CC349D-A70E-6939-8B16-74F9D3A0C814-00000001 0x00007ff93e810000 C:\Windows\System32\ntdll.dll
[  2] FB221C21-913B-2EAD-8848-085FF757F870-00000001 0x00007ff93dad0000 C:\Windows\System32\kernel32.dll
...
[ 21] 19E22631-C79A-9F11-E31E-93E8BB518332-00000001 0x00007ff93e590000 C:\Windows\System32\imm32.dll
[ 22] 7896C01C-3A5D-6AA1-98D2-71F14C724849-00000001 0x00007ff93bff0000 C:\Windows\System32\bcryptprimitives.dll
[ 23] 4D7C44AD-7BE4-4CFA-908F-77D2AB12331F-00000001 SimpleDLL.dll[0x0000000180000000] C:\Users\Tim\Projects\hubbub\target\debug\SimpleDLL.dll
[ 24] 27AC0972-E525-84FE-1A88-B1FE70D1603B-00000002 ucrtbased.dll[0x0000000180000000] C:\Windows\System32\ucrtbased.dll
[ 25] 44BC4E56-BD3A-8A6E-4D6A-62E50576C750-00000001 vcruntime140d.dll[0x0000000180000000] C:\Windows\System32\vcruntime140d.dl

but it doesn't include the fn ptr in its range:

> image dump sections hubbub.exe
Sections for 'C:\Users\Tim\Projects\hubbub\target\debug\hubbub.exe' (x86_64):
  SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
  ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
  0xffffffffffffffff regular          [0x00007ff71f4d0000-0x00007ff71f4d0400)  r--  0x00000000 0x00000400 0x00000000 hubbub.exe.PECOFF header
  0x00000001 code             [0x00007ff71f4d1000-0x00007ff721a07270)  r-x  0x00000400 0x02536400 0x60000020 hubbub.exe..text
  0x00000002 data             [0x00007ff721a08000-0x00007ff723b718e0)  r--  0x02536800 0x02169a00 0x40000040 hubbub.exe..rdata
  0x00000003 data             [0x00007ff723b72000-0x00007ff723c3f098)  rw-  0x046a0200 0x00066800 0xc0000040 hubbub.exe..data
  0x00000004 data             [0x00007ff723c40000-0x00007ff723dc7254)  r--  0x04706a00 0x00187400 0x40000040 hubbub.exe..pdata
  0x00000005 data             [0x00007ff723dc8000-0x00007ff723dc815c)  r--  0x0488de00 0x00000200 0x40000040 hubbub.exe._RDATA
  0x00000006 regular          [0x00007ff723dc9000-0x00007ff723e350bc)  r--  0x0488e000 0x0006c200 0x42000040 hubbub.exe..reloc

view this post on Zulip bjorn3 (Oct 03 2022 at 20:27):

Weird. Just as a sanity check, you didn't drop the Library before running the cranelift function? Otherwise the library is probably unloaded.

view this post on Zulip Tim Ngo (Oct 03 2022 at 20:36):

Nope, no drop.

view this post on Zulip Tim Ngo (Oct 03 2022 at 21:00):

Wait, I had an error where I was opening the library in a second place and letting it implicitly drop there, which led to the first load being unloaded.

view this post on Zulip Tim Ngo (Oct 03 2022 at 21:01):

Thanks for walking me through this, it works now!

view this post on Zulip bjorn3 (Oct 04 2022 at 07:38):

:tada:

view this post on Zulip Notification Bot (Oct 10 2022 at 15:42):

Tim Ngo has marked this topic as resolved.


Last updated: Oct 23 2024 at 20:03 UTC