Stream: cranelift

Topic: Call instruction causes generated code to segfault (aarch64)


view this post on Zulip Leslie Kerman (Nov 29 2023 at 03:13):

Screenshot-2023-11-29-at-03.11.50.png

The minimum reproducible case:

use std::fs::File;
use cranelift::prelude::{
    types::{self, Type},
    Value,
};
use cranelift_codegen::{
    ir::{
        AbiParam, Block, ExtFuncData, ExternalName, Function, InstBuilder, Signature,
        UserExternalName, UserFuncName,
    },
    isa::CallConv,
    settings::{self, Flags},
};
use cranelift_frontend::{FunctionBuilder, FunctionBuilderContext};
use cranelift_module::{Linkage, Module};
use cranelift_object::{ObjectBuilder, ObjectModule};

fn main() {
    // Cranelift setup.
    let flags = Flags::new(settings::builder());
    let mut object_module = {
        let isa = cranelift_native::builder()
            .expect("Error getting native ISA")
            .finish(flags)
            .unwrap();
        let obj_builder =
            ObjectBuilder::new(isa, "program", cranelift_module::default_libcall_names()).unwrap();
        ObjectModule::new(obj_builder)
    };
    let mut builder_ctx = FunctionBuilderContext::new();

    // Some repeating patterns used in both fn0 and fn1.
    fn make_sig(params: &[Type], returns: &[Type]) -> Signature {
        let mut sig = Signature::new(CallConv::SystemV);
        sig.params = params.iter().map(|&t| AbiParam::new(t)).collect();
        sig.returns = returns.iter().map(|&t| AbiParam::new(t)).collect();
        sig
    }
    fn init_entry_block(builder: &mut FunctionBuilder) -> Block {
        let block = builder.create_block();
        builder.append_block_params_for_function_params(block);
        builder.switch_to_block(block);
        builder.seal_block(block);
        block
    }

    // Function `succ(i32) -> i32`.
    // fn 0:0
    let sig0 = make_sig(&[types::I32], &[types::I32]);
    {
        let func_id = object_module
            .declare_function("succ", Linkage::Export, &sig0)
            .unwrap();
        let mut func = Function::with_name_signature(UserFuncName::user(0, 0), sig0.clone());
        let mut builder = FunctionBuilder::new(&mut func, &mut builder_ctx);

        let entry_block = init_entry_block(&mut builder);
        let arg0 = builder.block_params(entry_block)[0];
        let val = builder.ins().iadd_imm(arg0, 1);
        builder.ins().return_(&[val]);
        builder.finalize();
        println!("{}", func);
        object_module
            .define_function(func_id, &mut cranelift_codegen::Context::for_function(func))
            .unwrap();
        func_id
    };

    // Function `main() -> i32`.
    // fn 0:1
    let sig1 = make_sig(&[], &[types::I32]);
    {
        let func_id = object_module
            .declare_function("main", Linkage::Export, &sig1)
            .unwrap();
        let mut func = Function::with_name_signature(UserFuncName::user(0, 1), sig1);
        let mut builder = FunctionBuilder::new(&mut func, &mut builder_ctx);

        // Import fn0.
        let fn0_ref = {
            let sig_ref = builder.import_signature(sig0);
            let name_ref = builder
                .func
                .declare_imported_user_function(UserExternalName::new(0, 1));
            builder.import_function(ExtFuncData {
                name: ExternalName::user(name_ref),
                signature: sig_ref,
                colocated: false,
            })
        };

        init_entry_block(&mut builder);
        let x = builder.ins().iconst(types::I32, 0);
        // This call instruction causes the generated code to segfault:
        let call_inst = builder.ins().call(fn0_ref, &[x]);
        let &call_result: &Value = builder
            .inst_results(call_inst)
            .iter()
            .next()
            .unwrap()
            .into();
        builder.ins().return_(&[call_result]);
        // Commenting the above and uncomment this line does not segfault:
        // builder.ins().return_(&[x]);
        builder.finalize();

        println!("{}", func);
        object_module
            .define_function(func_id, &mut cranelift_codegen::Context::for_function(func))
            .unwrap();
        func_id
    };

    let obj_prod = object_module.finish();
    let bytes = obj_prod.emit().unwrap();
    write_bytes_to_path("program.o", &bytes).unwrap();
}

fn write_bytes_to_path(path: &str, buf: &[u8]) -> std::io::Result<()> {
    let mut file = File::create(path)?;
    std::io::Write::write_all(&mut file, buf)
}

Am I doing something wrong here or is this a bug?

view this post on Zulip Afonso Bordado (Nov 29 2023 at 12:41):

:wave: Hey,

This looks pretty much correct, I got this to run on my machine by changing the call instruction to call u0:0 instead of u0:1, I think it's just recursing into main until the stack overflows.

view this post on Zulip Leslie Kerman (Nov 29 2023 at 12:43):

Sorry, my mistake. Changing the call instruction to u0:0 still results in segfault.
Screenshot-2023-11-29-at-12.43.29.png

view this post on Zulip Afonso Bordado (Nov 29 2023 at 12:46):

Huh, weird, that works in my machine. :thinking: Can you try disabling pointer authentication by adding a

        isa.set("sign_return_address", "false").unwrap();

view this post on Zulip Afonso Bordado (Nov 29 2023 at 12:47):

when building the ISA?

view this post on Zulip Afonso Bordado (Nov 29 2023 at 12:47):

I don't know if pointer authentication needs someting special to work

view this post on Zulip Leslie Kerman (Nov 29 2023 at 12:53):

I'm not sure what you meant by isa.set(...), as there's no function set for TargetIsa. Could you clarify which line I'm supposed to add this in?

view this post on Zulip Afonso Bordado (Nov 29 2023 at 12:54):

Right, sorry, it's in the ISABuilder, so something along these lines:

        let mut isa = cranelift_native::builder()
            .expect("Error getting native ISA");
        isa.set("sign_return_address", "false").unwrap();
        let isa = isa
            .finish(flags)
            .unwrap();

view this post on Zulip Leslie Kerman (Nov 29 2023 at 12:56):

Screenshot-2023-11-29-at-12.55.49.png

It doesn't generate pointer auth anymore but it still segfaults. I suspect it's something wrong with my linking process.

view this post on Zulip Afonso Bordado (Nov 29 2023 at 12:57):

could you run it under lldb/gdb to check where it's crashing?

view this post on Zulip Afonso Bordado (Nov 29 2023 at 13:00):

Oh, also we could test with a PIE executable, not sure if that helps

view this post on Zulip Afonso Bordado (Nov 29 2023 at 13:01):

Try using these flags

    let mut flags_builder = settings::builder();
    flags_builder.enable("is_pic").unwrap();
    let flags = Flags::new(flags_builder);

view this post on Zulip Leslie Kerman (Nov 29 2023 at 13:05):

Screenshot-2023-11-29-at-13.04.35.png

Clang crashed on linking in this case

view this post on Zulip Afonso Bordado (Nov 29 2023 at 13:06):

That is weird! Could you send me that binary, or run objdump -Dr on it?

view this post on Zulip Leslie Kerman (Nov 29 2023 at 13:07):

Screenshot-2023-11-29-at-13.07.00.png

view this post on Zulip Afonso Bordado (Nov 29 2023 at 13:07):

(although i think that is probably unrelated to the main issue)

view this post on Zulip Leslie Kerman (Nov 29 2023 at 13:07):

program.o
the program

view this post on Zulip Leslie Kerman (Nov 29 2023 at 13:08):

program.o
Also this is the emitted object without enabling is_pic flag

view this post on Zulip Afonso Bordado (Nov 29 2023 at 13:30):

These all look okay to me. Could you try running the non PIC version under a debugger, and check where it crashes?

view this post on Zulip Afonso Bordado (Nov 29 2023 at 13:30):

I think the linker error under the PIC might be unrelated, although probably something to follow up after

view this post on Zulip Afonso Bordado (Nov 29 2023 at 13:31):

Unfortunateley I don't have a Mac to be able to run these examples

view this post on Zulip Leslie Kerman (Nov 29 2023 at 13:32):

It was fine on my x64 laptop, so it's probably a M-chip specific problem

view this post on Zulip Leslie Kerman (Nov 29 2023 at 13:33):

Screenshot-2023-11-29-at-13.33.27.png
this is the output on lldb

view this post on Zulip Afonso Bordado (Nov 29 2023 at 16:37):

I don't have much more time to look at this today, sorry! Would you be able to figure out where that invalid load is coming from? We have one load instruction in the main function, is it that one? Or does it happen before?

view this post on Zulip Leslie Kerman (Nov 30 2023 at 00:30):

That's ok. Thanks for your help

view this post on Zulip Leslie Kerman (Nov 30 2023 at 00:33):

Screenshot-2023-11-30-at-00.29.57.png
The executable seem to segfault on the instruction blr x2.

view this post on Zulip Afonso Bordado (Nov 30 2023 at 11:39):

Some interesting results. I borrowed a M1 Mac, it has clang 14.0.3 instead of 15.

Compiling with is_pic and linking with clang makes the executable work without issue.

Compiling without is_pic crashes in the dynamic linker before getting to main. I also tried to compile that binary with clang -fno-pie ./program.o, which led to the following warnings:

ld: warning: -no_pie is deprecated when targeting new OS versions
ld: warning: -no_pie ignored for arm64

So I'm not sure compiling without is_pic for Mac ARM machines is a valid configuration

view this post on Zulip Afonso Bordado (Nov 30 2023 at 11:39):

I'm going to try to install clang 15 to reproduce the issue you were having with is_pic

view this post on Zulip Afonso Bordado (Nov 30 2023 at 11:48):

Hmm, compiling with clang-15 worked here :thinking:

view this post on Zulip Leslie Kerman (Nov 30 2023 at 11:55):

Screenshot-2023-11-30-at-11.54.21.png
Can you try linking with ld? I think it's the problem of ld here not clang's invocation of it.

view this post on Zulip Afonso Bordado (Nov 30 2023 at 11:56):

It didn't seem to crash:

m1@ab850359-a1b4-4fa7-85ab-da6c63f0c56d clif-test % ld -arch arm64 -macos_version_min 14.0.0 program.o
ld: dynamic executables or dylibs must link with libSystem.dylib for architecture arm64

view this post on Zulip Afonso Bordado (Nov 30 2023 at 11:57):

Would you be able to try with an older xcode version?

view this post on Zulip Afonso Bordado (Nov 30 2023 at 11:57):

I wonder if this an issue only on newer linkers

view this post on Zulip Afonso Bordado (Nov 30 2023 at 12:00):

Also, do you have an alternate linker such as lld available? Using it also worked for me:

m1@ab850359-a1b4-4fa7-85ab-da6c63f0c56d clif-test % clang -fuse-ld=lld -v ./program.o
Homebrew clang version 15.0.7
Target: arm64-apple-darwin23.1.0
Thread model: posix
InstalledDir: /opt/homebrew/opt/llvm@15/bin
 "/opt/homebrew/opt/llvm@15/bin/ld64.lld" -demangle -dynamic -arch arm64 -platform_version macos 14.0.0 14.0.0 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -o a.out ./program.o -lSystem /opt/homebrew/Cellar/llvm@15/15.0.7/lib/clang/15.0.7/lib/darwin/libclang_rt.osx.a
m1@ab850359-a1b4-4fa7-85ab-da6c63f0c56d clif-test % ./a.out
m1@ab850359-a1b4-4fa7-85ab-da6c63f0c56d clif-test %

view this post on Zulip Leslie Kerman (Nov 30 2023 at 12:09):

Screenshot-2023-11-30-at-12.09.06.png
I have this error here with both LLVM15's lld and LLVM17's

view this post on Zulip Leslie Kerman (Nov 30 2023 at 12:13):

Ok using ld64.lld works

view this post on Zulip Leslie Kerman (Nov 30 2023 at 12:13):

Thanks a lot for the help

view this post on Zulip Afonso Bordado (Nov 30 2023 at 12:14):

hmm, i think that would point it to being an issue with the version that apple ships in the latest xcode. Or alternatively they no longer accepting something we are producing


Last updated: Oct 23 2024 at 20:03 UTC