Hi everyone, currently experimenting with combining the Cranelift JIT capabilities with emulation.
For that, I've been trying to develop a separate Cranelift implementation for my GBA emulator which emulates the ARM7TDMI CPU and as some instructions need to read or write from and to the Bus
, I have a Mcu
trait declaring read{8, 16, 32}
and write{8, 16, 32}
methods in order to access the memory map.
When emulating via interpretation, I can just plainly call those read/write functions and everything works, for the Cranelift JIT I've tried to piece together the information I need to make those functions callable as FuncRefs
.
I create a JITBuilder
and declare the symbols:
let mut builder = JITBuilder::with_isa(isa, cranelift_module::default_libcall_names());
// Declare bus read functions.
builder.symbol("read8", Bus::read8 as *const u8);
builder.symbol("read16", Bus::read16 as *const u8);
builder.symbol("read32", Bus::read32 as *const u8);
// Declare bus write functions.
builder.symbol("write8", Bus::write8 as *const u8);
builder.symbol("write16", Bus::write16 as *const u8);
builder.symbol("write32", Bus::write32 as *const u8);
I then create a JITModule
from the builder
and declare the functions, here with a macro, and store the resulting FuncID
s for later use:
macro_rules! link_io_funcs {
($module:expr, read, $bits:expr) => {
paste! {{
let mut [<sigr $bits>] = $module.make_signature();
[<sigr $bits>].params.push(AbiParam::new(I32));
[<sigr $bits>].returns.push(AbiParam::new([<I $bits>]));
$module.declare_function(concat!("read", $bits), Linkage::Local, &[<sigr $bits>])
}}
};
($module:expr, write, $bits:expr) => {
paste! {{
let mut [<sigw $bits>] = $module.make_signature();
[<sigw $bits>].params.push(AbiParam::new(I32));
[<sigw $bits>].params.push(AbiParam::new([<I $bits>]));
$module.declare_function(concat!("write", $bits), Linkage::Local, &[<sigw $bits>])
}}
};
}
...
let r8 = link_io_funcs!(self.module, read, 8)?;
let r16 = link_io_funcs!(self.module, read, 16)?;
let r32 = link_io_funcs!(self.module, read, 32)?;
let w8 = link_io_funcs!(self.module, write, 8)?;
let w16 = link_io_funcs!(self.module, write, 16)?;
let w32 = link_io_funcs!(self.module, write, 32)?;
Inside the emulated instruction, where I generate the CLIR, I do the following to be able to call the function:
let r8_func = jit.module.declare_func_in_func(jit.io[0], clir.func);
...
let mem16 = clir.ins().call(r8_func, &[aligned_addr]);
When compiling in release mode, however, I get the STATUS_ACCESS_VIOLATION
when trying to run the compiled code and in debug mode I don't think the generated call
really does anything as the code runs but seemingly seems to skip over the call
. The x86 disassembly from Cranelift itself only shows call User(userextname2)
and IDA Pro spits out call $+2
.
I've also tried declaring all the trait methods and its implementations as extern "C"
just in case.
It's hard to "debug remotely" via Zulip thread but the call $+2
is certainly suspicious: perhaps the relocation isn't happening for some reason? cc @bjorn3 for thoughts from the cranelift-jit side (I don't know that module well)
most helpful would be to single-step at the assembly level and see where the crash actually happens
I can post the full repo but it is messy and I've tried to at least include an MRE here ^^
i.e. if the call is going somewhere unexpected, or if control reaches your host functions but the ABI mismatches for some reason
Can you give a couple of instructions of context around the crash site?
And preferably including the raw bytes for the instructions.
I can, one sec
(re: full repo, nah, personally at least I think it's unlikely folks are going to dig into a whole project, figure out how to run it, and debug it for you; better to give tips on what to look at)
https://gist.github.com/xkevio/056424c69f6158af2e9b2af7ffc76da8
this here includes the cranelift annotated x86 of a GBA rom i attempted to recompile as well as the bytes in the second attachment
ah, its call $+5
not +2 in IDA pro
For the raw bytes would you mind printing them with println!("{:#X}", bytes)
instead of println!("{:X}", bytes)
? That will allow me to copy-paste them into a rust source file without having to manually add 0x
prefixes everywhere.
call $+5
that bit is certainly the issue then; call to the next instruction, so the call doesn't happen but a stray return address is pushed and the stack is misaligned. The key bit will be working out why the reloc didn't happen
($+5
is the instruction after the call, since we emit the 5-byte form; the signed 32-bit offset is relative to the end of the inst, so that target indicates the field in the inst is all zeroes, i.e., hasn't been filled in)
Did you call jit_module.finalize_definitions()
before calling the jitted function?
bjorn3 said:
For the raw bytes would you mind printing them with
println!("{:#X}", bytes)
instead ofprintln!("{:X}", bytes)
? That will allow me to copy-paste them into a rust source file without having to manually add0x
prefixes everywhere.
I added it to the gist I already sent now
bjorn3 said:
Did you call
jit_module.finalize_definitions()
before calling the jitted function?
I didn't, but I did just now and still the status access violation
Not sure what the issue is then.
any ideal place where I should be calling finalize_definitions
? Just before using Context::compile
, right after declare_func_in_func(...)
?
It should be called after all define_function
calls and before the first call of any jitted function.
oh uhm so I may or may not have forgotten to use define_function
then
however, now I get a new error that wasnt there before
thread 'main' panicked at C:\Users\xkevi\.cargo\registry\src\index.crates.io-6f17d22bba15001f\cranelift-codegen-0.105.0\src\remove_constant_phis.rs:268:10:
remove_constant_phis: entry block unknown
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Do you have any empty functions?
the first block is always the same and shouldnt be affected by external functions, and thus should not ever be empty
Can you get a dump of the clif ir that is responsible for the crash?
bjorn3 said:
Can you get a dump of the clif ir that is responsible for the crash?
for the violation or the entry block unknown
?
cause the entry block unknown
is probably because define_function
is trying to compile the function and put it into the context of all the other code i am translating. all the other external function examples also didnt need to use define_function
Okay so I've refactored my code a little bit to make use of all the JITModule
functionality. When compiling/translating an ARM guest code block, after finalizing the builder, I declare the current jitted block function with Linkage::Export
, I then define that function, use clear_context
, finalize_definitions
and get the fn pointer with get_finalized_function
. My bus read/write functions are still declared after creating the JITModule
and symbols are defined with JITBuilder::symbol
. Before using the functions, I do declare_func_in_func
still.
Now, however, when compiling the block that would include such a function call, I get an internal panic in cranelift_jit
from a TryFromIntError()
stemming from pcrel
here.
Reloc::X86PCRel4 | Reloc::X86CallPCRel4 => {
let base = get_address(name);
let what = unsafe { base.offset(isize::try_from(addend).unwrap()) };
let pcrel = i32::try_from((what as isize) - (at as isize)).unwrap();
unsafe { write_unaligned(at as *mut i32, pcrel) };
}
(https://github.com/bytecodealliance/wasmtime/blob/main/cranelift/jit/src/compiled_blob.rs#L58)
@bjorn3
Try declaring your read and write functions as Linkage::Import.
That will ensure the colocated flag isn't set for them. On x86 colocated requires the target of a call to be within +/-2GB of the call site, which is pretty much guaranteed to not be the case for functions imported from the host program.
Then, we are back to the STATUS_ACCESS_VIOLATION
I hate to post the repo directly for the same reasons @Chris Fallin mentioned but just so we arent missing anything obvious. https://github.com/xkevio/kba/blob/cranelift/src/arm/jit/arm7tdmi.rs#L81-L89 for the main method finalization, https://github.com/xkevio/kba/blob/cranelift/src/arm/jit/mod.rs#L51-L82 for the read/write functions and https://github.com/xkevio/kba/blob/cranelift/src/arm/jit/arm7tdmi.rs#L257 for an example where they will be called
gba_bios.bin is https://archive.org/details/gba_bios_202206?
oh yea, forgot i still hard-include that
that one should suffice though
https://github.com/wheremyfoodat/Panda3DS/tree/cdn/docs/gba-demos
would also need first.gba
, also named gang.gba
to reproduce exactly
Managed to reproduce the SIGSEGV.
Thread 1 "kba" received signal SIGSEGV, Segmentation fault.
kba::mmu::bus::{impl#2}::write8 (self=0x617fa30, address=33554432, value=0) at src/mmu/bus.rs:227
227 0x02 => self.wram[address as usize % 0x0004_0000] = value,
(gdb) bt
#0 kba::mmu::bus::{impl#2}::write8 (self=0x617fa30, address=33554432, value=0) at src/mmu/bus.rs:227
#1 0x00005555555eb75a in kba::mmu::Mcu::write16<kba::mmu::bus::Bus> (self=0x617fa30, address=33554432, value=0) at src/mmu/mod.rs:67
#2 0x0000555556cc804e in ?? ()
write8
has a function signature of fn(&mut Bus, u32, u8)
, but in the clif IR you are calling with with a signature of fn(u32, u8)
, in other words, you are missing the &mut Bus
argument. The crash happens because the u32 you give is interpreted as the &mut Bus
pointer.
oh my god
&mut Bus
would be isa::pointer_type
in clif right?
I see two possible solutions for this:
Bus
a singleton stored in a static.Bus
through everywhere as argument. This is what Wasmtime effectively does for the VMContext
which contains pointers to all wasm instance specific information.Kevin K. said:
&mut Bus
would beisa::pointer_type
in clif right?
Yes
it compiles now and works omg, thank you so much that was a much simpler error than i thought. cannot believe i managed to miss that
Great! Happy to help!
Kevin K. has marked this topic as resolved.
Last updated: Dec 23 2024 at 12:05 UTC