Kevin K. (May 31 2024 at 15:22): Kevin K. (May 31 2024 at 15:22):Hi everyone, currently experimenting with combining the Cranelift JIT capabilities with emulation.
For that, I've been trying to develop a separate Cranelift implementation for my GBA emulator which emulates the ARM7TDMI CPU and as some instructions need to read or write from and to the Bus, I have a Mcu trait declaring read{8, 16, 32} and write{8, 16, 32} methods in order to access the memory map.
When emulating via interpretation, I can just plainly call those read/write functions and everything works, for the Cranelift JIT I've tried to piece together the information I need to make those functions callable as FuncRefs.
I create a JITBuilder and declare the symbols:
let mut builder = JITBuilder::with_isa(isa, cranelift_module::default_libcall_names());
// Declare bus read functions.
builder.symbol("read8", Bus::read8 as *const u8);
builder.symbol("read16", Bus::read16 as *const u8);
builder.symbol("read32", Bus::read32 as *const u8);
// Declare bus write functions.
builder.symbol("write8", Bus::write8 as *const u8);
builder.symbol("write16", Bus::write16 as *const u8);
builder.symbol("write32", Bus::write32 as *const u8);
I then create a JITModule from the builderand declare the functions, here with a macro, and store the resulting FuncIDs for later use:
macro_rules! link_io_funcs {
($module:expr, read, $bits:expr) => {
paste! {{
let mut [<sigr $bits>] = $module.make_signature();
[<sigr $bits>].params.push(AbiParam::new(I32));
[<sigr $bits>].returns.push(AbiParam::new([<I $bits>]));
$module.declare_function(concat!("read", $bits), Linkage::Local, &[<sigr $bits>])
}}
};
($module:expr, write, $bits:expr) => {
paste! {{
let mut [<sigw $bits>] = $module.make_signature();
[<sigw $bits>].params.push(AbiParam::new(I32));
[<sigw $bits>].params.push(AbiParam::new([<I $bits>]));
$module.declare_function(concat!("write", $bits), Linkage::Local, &[<sigw $bits>])
}}
};
}
...
let r8 = link_io_funcs!(self.module, read, 8)?;
let r16 = link_io_funcs!(self.module, read, 16)?;
let r32 = link_io_funcs!(self.module, read, 32)?;
let w8 = link_io_funcs!(self.module, write, 8)?;
let w16 = link_io_funcs!(self.module, write, 16)?;
let w32 = link_io_funcs!(self.module, write, 32)?;
Inside the emulated instruction, where I generate the CLIR, I do the following to be able to call the function:
let r8_func = jit.module.declare_func_in_func(jit.io[0], clir.func);
...
let mem16 = clir.ins().call(r8_func, &[aligned_addr]);
When compiling in release mode, however, I get the STATUS_ACCESS_VIOLATION when trying to run the compiled code and in debug mode I don't think the generated call really does anything as the code runs but seemingly seems to skip over the call. The x86 disassembly from Cranelift itself only shows call User(userextname2) and IDA Pro spits out call $+2.
Kevin K. (May 31 2024 at 17:57):I've also tried declaring all the trait methods and its implementations as extern "C" just in case.
Chris Fallin (May 31 2024 at 17:59):It's hard to "debug remotely" via Zulip thread but the call $+2 is certainly suspicious: perhaps the relocation isn't happening for some reason? cc @bjorn3 for thoughts from the cranelift-jit side (I don't know that module well)
Chris Fallin (May 31 2024 at 17:59):most helpful would be to single-step at the assembly level and see where the crash actually happens
Kevin K. (May 31 2024 at 17:59):I can post the full repo but it is messy and I've tried to at least include an MRE here ^^
Chris Fallin (May 31 2024 at 18:00):i.e. if the call is going somewhere unexpected, or if control reaches your host functions but the ABI mismatches for some reason
bjorn3 (May 31 2024 at 18:00):Can you give a couple of instructions of context around the crash site?
bjorn3 (May 31 2024 at 18:00):And preferably including the raw bytes for the instructions.
Kevin K. (May 31 2024 at 18:00):I can, one sec
Chris Fallin (May 31 2024 at 18:00):(re: full repo, nah, personally at least I think it's unlikely folks are going to dig into a whole project, figure out how to run it, and debug it for you; better to give tips on what to look at)
Kevin K. (May 31 2024 at 18:01):https://gist.github.com/xkevio/056424c69f6158af2e9b2af7ffc76da8
this here includes the cranelift annotated x86 of a GBA rom i attempted to recompile as well as the bytes in the second attachment
Kevin K. (May 31 2024 at 18:03):ah, its call $+5 not +2 in IDA pro
bjorn3 (May 31 2024 at 18:03):For the raw bytes would you mind printing them with println!("{:#X}", bytes) instead of println!("{:X}", bytes)? That will allow me to copy-paste them into a rust source file without having to manually add 0x prefixes everywhere.
Chris Fallin (May 31 2024 at 18:04):
call $+5
that bit is certainly the issue then; call to the next instruction, so the call doesn't happen but a stray return address is pushed and the stack is misaligned. The key bit will be working out why the reloc didn't happen
Chris Fallin (May 31 2024 at 18:05):($+5 is the instruction after the call, since we emit the 5-byte form; the signed 32-bit offset is relative to the end of the inst, so that target indicates the field in the inst is all zeroes, i.e., hasn't been filled in)
bjorn3 (May 31 2024 at 18:05):Did you call jit_module.finalize_definitions() before calling the jitted function?
Kevin K. (May 31 2024 at 18:08):bjorn3 said:
For the raw bytes would you mind printing them with
println!("{:#X}", bytes)instead ofprintln!("{:X}", bytes)? That will allow me to copy-paste them into a rust source file without having to manually add0xprefixes everywhere.
I added it to the gist I already sent now
Kevin K. (May 31 2024 at 18:08):bjorn3 said:
Did you call
jit_module.finalize_definitions()before calling the jitted function?
I didn't, but I did just now and still the status access violation
bjorn3 (May 31 2024 at 18:20):Not sure what the issue is then.
Kevin K. (May 31 2024 at 18:21):any ideal place where I should be calling finalize_definitions? Just before using Context::compile, right after declare_func_in_func(...)?
bjorn3 (May 31 2024 at 18:22):It should be called after all define_function calls and before the first call of any jitted function.
Kevin K. (May 31 2024 at 18:26):oh uhm so I may or may not have forgotten to use define_function then
Kevin K. (May 31 2024 at 18:26):however, now I get a new error that wasnt there before
thread 'main' panicked at C:\Users\xkevi\.cargo\registry\src\index.crates.io-6f17d22bba15001f\cranelift-codegen-0.105.0\src\remove_constant_phis.rs:268:10:
remove_constant_phis: entry block unknown
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Do you have any empty functions?
Kevin K. (May 31 2024 at 18:29):the first block is always the same and shouldnt be affected by external functions, and thus should not ever be empty
bjorn3 (May 31 2024 at 18:37):Can you get a dump of the clif ir that is responsible for the crash?
Kevin K. (May 31 2024 at 21:24):bjorn3 said:
Can you get a dump of the clif ir that is responsible for the crash?
for the violation or the entry block unknown?
Kevin K. (May 31 2024 at 21:26):cause the entry block unknown is probably because define_function is trying to compile the function and put it into the context of all the other code i am translating. all the other external function examples also didnt need to use define_function
Kevin K. (Jun 03 2024 at 22:24):Okay so I've refactored my code a little bit to make use of all the JITModule functionality. When compiling/translating an ARM guest code block, after finalizing the builder, I declare the current jitted block function with Linkage::Export, I then define that function, use clear_context, finalize_definitions and get the fn pointer with get_finalized_function. My bus read/write functions are still declared after creating the JITModule and symbols are defined with JITBuilder::symbol. Before using the functions, I do declare_func_in_func still.
Now, however, when compiling the block that would include such a function call, I get an internal panic in cranelift_jit from a TryFromIntError() stemming from pcrel here.
Reloc::X86PCRel4 | Reloc::X86CallPCRel4 => {
let base = get_address(name);
let what = unsafe { base.offset(isize::try_from(addend).unwrap()) };
let pcrel = i32::try_from((what as isize) - (at as isize)).unwrap();
unsafe { write_unaligned(at as *mut i32, pcrel) };
}
(https://github.com/bytecodealliance/wasmtime/blob/main/cranelift/jit/src/compiled_blob.rs#L58)
Kevin K. (Jun 04 2024 at 17:42):@bjorn3
bjorn3 (Jun 04 2024 at 18:39):Try declaring your read and write functions as Linkage::Import.
bjorn3 (Jun 04 2024 at 18:41):That will ensure the colocated flag isn't set for them. On x86 colocated requires the target of a call to be within +/-2GB of the call site, which is pretty much guaranteed to not be the case for functions imported from the host program.
Kevin K. (Jun 04 2024 at 18:41):Then, we are back to the STATUS_ACCESS_VIOLATION
Kevin K. (Jun 04 2024 at 18:45):I hate to post the repo directly for the same reasons @Chris Fallin mentioned but just so we arent missing anything obvious. https://github.com/xkevio/kba/blob/cranelift/src/arm/jit/arm7tdmi.rs#L81-L89 for the main method finalization, https://github.com/xkevio/kba/blob/cranelift/src/arm/jit/mod.rs#L51-L82 for the read/write functions and https://github.com/xkevio/kba/blob/cranelift/src/arm/jit/arm7tdmi.rs#L257 for an example where they will be called
bjorn3 (Jun 04 2024 at 18:59):gba_bios.bin is https://archive.org/details/gba_bios_202206?
Kevin K. (Jun 04 2024 at 18:59):oh yea, forgot i still hard-include that
Kevin K. (Jun 04 2024 at 18:59):that one should suffice though
Kevin K. (Jun 04 2024 at 19:00):https://github.com/wheremyfoodat/Panda3DS/tree/cdn/docs/gba-demos
Kevin K. (Jun 04 2024 at 19:00):would also need first.gba, also named gang.gba to reproduce exactly
bjorn3 (Jun 04 2024 at 19:01):Managed to reproduce the SIGSEGV.
bjorn3 (Jun 04 2024 at 19:05):Thread 1 "kba" received signal SIGSEGV, Segmentation fault.
kba::mmu::bus::{impl#2}::write8 (self=0x617fa30, address=33554432, value=0) at src/mmu/bus.rs:227
227 0x02 => self.wram[address as usize % 0x0004_0000] = value,
(gdb) bt
#0 kba::mmu::bus::{impl#2}::write8 (self=0x617fa30, address=33554432, value=0) at src/mmu/bus.rs:227
#1 0x00005555555eb75a in kba::mmu::Mcu::write16<kba::mmu::bus::Bus> (self=0x617fa30, address=33554432, value=0) at src/mmu/mod.rs:67
#2 0x0000555556cc804e in ?? ()
write8 has a function signature of fn(&mut Bus, u32, u8), but in the clif IR you are calling with with a signature of fn(u32, u8), in other words, you are missing the &mut Bus argument. The crash happens because the u32 you give is interpreted as the &mut Bus pointer.
Kevin K. (Jun 04 2024 at 19:06):oh my god
Kevin K. (Jun 04 2024 at 19:07):&mut Bus would be isa::pointer_type in clif right?
bjorn3 (Jun 04 2024 at 19:07):I see two possible solutions for this:
Bus a singleton stored in a static.Bus through everywhere as argument. This is what Wasmtime effectively does for the VMContext which contains pointers to all wasm instance specific information. bjorn3 (Jun 04 2024 at 19:07):Kevin K. said:
&mut Buswould beisa::pointer_typein clif right?
Yes
Kevin K. (Jun 04 2024 at 19:28):it compiles now and works omg, thank you so much that was a much simpler error than i thought. cannot believe i managed to miss that
bjorn3 (Jun 04 2024 at 19:28):Great! Happy to help!
Notification Bot (Jun 04 2024 at 19:29):Kevin K. has marked this topic as resolved.
Last updated: Oct 23 2024 at 20:03 UTC