What is the Preamble · cranelift

Stream: cranelift

Topic: What is the Preamble

Chris Clark (Oct 08 2023 at 15:56):

https://docs.rs/cranelift-codegen/latest/cranelift_codegen/ir/trait.InstBuilder.html#method.call

Direct function call.

Call a function which has been declared in the preamble. The argument types must match the function’s signature.

This is my first go at invoking another method. And I am not sure what this means. If I'm understanding this correctly, the other function must already be defined somewhere. I'm curious what this means for recursive functions, or functions that will eventually lead to calling the same function. I consider circular dependencies a code smell, and do want to avoid them. Just wondering how I can declare and configure this preamble

bjorn3 (Oct 08 2023 at 16:57):

The linked part here: https://github.com/bytecodealliance/wasmtime/blob/fef8a90f258483ac6c2e470d3de277a743d5d392/cranelift/filetests/filetests/isa/x64/call-conv.clif#L41-L42 is the preamble. Basically just the function local list of function signatures, function declarations and stack slot definitions.

bjorn3 (Oct 08 2023 at 16:58):

module.declare_func_in_func() adds a function declaration for a given module FuncId to the Function.

Chris Clark (Oct 08 2023 at 19:06):

Are there any examples that you know of that might help me clear this up?

"stack slot definition" is this a structs members?

Chris Fallin (Oct 08 2023 at 19:23):

@Chris Clark this isn't a circular dependency of any sort. A function just needs to declare other functions that it calls, as you see in the linked file that @bjorn3 provided. Think of it as providing the subset of all global prototypes that are needed for this function body; and we do need this information, because the signature is required to get the calling convention right, and the symbol is needed (for direct calls) to emit the relocation that the linker will fix up.

Some other examples of calls are here as well; and here you can see stackslot definitions with the explicit_slot directives. Note in all cases these are in the CLIF's preamble prior to the first block.

Chris Clark (Oct 08 2023 at 20:14):

Oh sorry, I meant examples of rust code that uses the ins builder for call. I'm still not at the level of being able to read the IR, and know where and how it got there. Particularly, getting the "FuncRef"

Afonso Bordado (Oct 08 2023 at 20:16):

We have a neat example in the cranelift-jit-demo here.

Chris Clark (Oct 08 2023 at 20:26):

ahh, seems we are using the module builder here.

let callee = self
            .module
            .declare_function(&name, Linkage::Import, &sig)
            .expect("problem declaring function");
        let local_callee = self.module.declare_func_in_func(callee, self.builder.func);
        ...
        let call = self.builder.ins().call(local_callee, &arg_values)

Is this not avoidable?

Afonso Bordado (Oct 08 2023 at 20:32):

You shouldn't need to use the module builder to declare a function in another function. All of the functions that declare_func_in_func uses are public.

However, it does a bunch of things that you probably will want to do anyway.

Chris Clark (Oct 08 2023 at 22:00):

This is why my monkey brain has so far, I also noticed from your link that values are returned by the recursed "translate" function. So this sounds like an optimization thing future me can do.

    pub fn handle_invoke(
        &mut self,
        op: &Invoke,
        builder: &mut FunctionBuilder,
    ) -> ResultFir<Variable> {
        let mut args: Vec<Value> = vec![];
        let mut call: Inst = Inst::from_u32(0);
        if op.args.is_some() {
            args = op
                .args
                .unwrap()
                .into_iter()
                .map(|x| {
                    return builder.use_var(self.recurse(&x, builder).unwrap());
                })
                .collect();
        }
        if op.prev.is_some() { // recurse on prev
        } else {
         // how to get a funcref here?
            call = builder.ins().call(FuncRef::from_u32(0), args.as_slice());
        }
        let result = self.add_var();
        builder.declare_var(result, I64);
        builder.def_var(result, builder.inst_results(call)[0]);
        Ok(result)
    }

The context of this function is in my IR generation stage, so I wouldn't have the module involved. I almost just want it to be a symbol like the UserFuncName::user name.

Chris Clark (Oct 11 2023 at 01:26):

I'm not seeing how I can do this, @Afonso Bordado . I can't seem to generate a FuncRef in any meaningful way from a Function

Afonso Bordado (Oct 11 2023 at 05:43):

You can use Function::import_function to declare a function inside another function. That should give you a FuncRef that you can use.

Chris Clark (Oct 11 2023 at 13:52):

The only function which has ExtFuncData as a return type, which is an argument to import_function is clone. I checked in frontend and codegen.

Chris Clark (Oct 11 2023 at 14:04):

I think I am barking up the wrong tree. Certainly Afonso alluded to that, but is there any reason why I need to bring Module into this? Module to me is the step to building object files, and dealing with tracking data. It seems to me like IR could be done without the need and just use symbols and symbol offsets. I might be vastly underestimating the IR, or misunderstanding Module as well.

Chris Fallin (Oct 11 2023 at 15:13):

@Chris Clark you don't need Module to define a function reference and call it within a function -- I think Afonso was bringing it up in a broader context of how to generate object files, but let's stick with the basic "compile a single function" for now. Several points that might help:

Cranelift's core (cranelift-codegen) is designed to compile functions independently. This allows for parallel compilation and in general is a nice functional property to have. (For example, it also allows for incremental compilation caching, where only changed function bodies are recompiled, because the whole input to a function's compilation is in that function's CLIF.)
Given that, all of a function's external references to other functions and symbols need to be locally declared. These are the things in the preamble: they are basically "interned" references, given a particular ID so we can use them in the function body.
Function is the function body type -- the actual definition of a function that we compile. If f calls g, then in the context of compiling f, we'll never build or have a Function for g -- we don't know its body, we're just calling it.
To get a FuncRef for g, we need to declare it in the preamble. That's what import_function does. It takes an ExtFuncData, and this is a struct with public fields, so you're meant to build an instance of it and pass it in.
The fields of ExtFuncData are an ExternalName, a signature, and a "colocated" flag (docs here are useful especially for the last one). The signature is needed because when calling the function we need to get the ABI right.
The ExternalName is basically passed through to the relocations you get with the compiled machine code. Because Cranelift is designed as a library, the core (cranelift-codegen) remains agnostic to the user's notion of symbols or references; basically, you say "symbol 23" (a UserExternalNameRef) and it's assumed that you have an external symbol table somewhere that tells you how to resolve the relocation at the end of the compilation. In other words, Cranelift will plumb through an arbitrary u32 index for you, and doesn't track symbols itself.

Does all that make sense? Happy to clarify more if needed!

Last updated: Oct 23 2024 at 20:03 UTC