wasmtime / issue #10766 Impressions from a first-time user · git-wasmtime

I tried using Cranelift for a new project JIT-compiling scripts for realtime audio processing. I've done similar work using Inkwell which wraps LLVM. I remain very interested in having a modern and competitive Rust-first JIT compiler, and I really want to see the Cranelift project succeed. While I was able to get get started quickly with Cranelift, there have been multiple separate and persistent issues which taken together have made me reconsider using Cranelift over Inkwell. I want to share my experiences and specific feedback because I think such first-time impressions are vital to the health of open source software and again, I love this project and want it to keep growing.

Apologies for putting so many things into a single issue. If it suits you, feel free to chop this up and quote verbatim or paraphrase into separate issues. For now, I'm moving on from Cranelift and returning to Inkwell and LLVM for my own productivity.

Multiple distracting TODO's in the JIT demo

My early rapid success interacting with Cranelift was from following and adapting the jit demo. Reading through example code though, there are several distracting TODO comments questioning Cranelift's own design and pointing out known pain points. As a first-time user, this gives me the impression that the API hasn't matured and is probably unstable, which makes me hesitant to want to commit to it. What I would love to see here instead is just a self-contained demo where each part of the library plays an obvious role. Possibly some of this commentary is out of date and the demo can be refined already.

Scattered documentation due to crate separation

I rely heavily on documentation hosted on docs.rs where it is easily navigable and rapidly searchable. Oftentimes I search for individual types, names, and keywords to quickly and serendipitously discover related functionality. The separation of Cranelift into multiple crates however means that oftentimes, the thing I'm looking for isn't found because it's in a separate crate. For example, FunctionBuilder is defined in cranelift-frontend, but all of the instruction-inserting methods I use it with are defined in the InstBuilder trait in the cranelift-codegen crate, where I would have to know to look first. This may be more of a problem for Rust's own documentation system than with Cranelift per se.

Difficulties with the first block of a function

I was trying to JIT-compile a function consisting of an entry block, multiple user-defined sequences which can't be known ahead of time, and an exit block. The first thing I tried was to create the 'entry' block first, then populate user-defined blocks, and finally switch to the 'entry' block and populate it with a jump instruction to the first user-defined block. This turned out to be very frustrating, and Cranelift would invariably error with "invalid reference to entry block". It turns out that Cranelift considers the entry block not to be the first created block (as I had assumed), but rather the first block that I pass to FunctionBuilder::switch_to_block, as explained in the docs for FunctionBuilder. This was confusing for me, since I wasn't ready to put anything in my entry block yet and so wouldn't have thought to switch to it first. Additionally, I'm not really convinced the documentation is correct here, and Cranelift actively prevents me from partially filling and later switching back to partly-filled blocks.

An explicit method such as FunctionBuilder::set_entry_block or having the first _created_ block become the function entry point would make this process a lot more straightforward. After all, the blocks themselves are merely nodes in a control flow graph, and everywhere else with jump and return instructions, their dependencies are explicitly stated, so it would make sense to explicitly state which is the entry block, rather than implying it through non-obvious workflows.

let entry_block = builder.create_block();
let exit_block = builder.create_block();

builder.switch_to_block(entry_block);
builder.ins().jump(exit_block, &[]);

builder.switch_to_block(exit_block);
builder.ins().return_(&[]);

let entry_block = builder.create_block();
let exit_block = builder.create_block();

builder.switch_to_block(exit_block);
builder.ins().return_(&[]);

builder.switch_to_block(entry_block);
builder.ins().jump(exit_block, &[]);

Compilation(Verifier(VerifierErrors([VerifierError { location: inst1, context: None, message: "invalid reference to entry block block1" }])))

Contrary to the documentation, adding a call to switch_to_block with the first block does not fix this:

let entry_block = builder.create_block();
let exit_block = builder.create_block();

builder.switch_to_block(entry_block);

builder.switch_to_block(exit_block);
builder.ins().return_(&[]);

builder.switch_to_block(entry_block);
builder.ins().jump(exit_block, &[]);

value: Compilation(Verifier(VerifierErrors([VerifierError { location: inst1, context: None, message: "invalid reference to entry block block1" }])))

If hypothesize that perhaps the first block that I switch to _and insert an instruction into_ becomes the first block, attempting to insert a nop to return to later just outright panics:

let entry_block = builder.create_block();
let exit_block = builder.create_block();

builder.switch_to_block(entry_block);
builder.ins().nop();

builder.switch_to_block(exit_block);
builder.ins().return_(&[]);

builder.switch_to_block(entry_block);
builder.ins().jump(exit_block, &[]);

thread panicked at cranelift-frontend-0.117.2/src/frontend.rs:361:9:
you have to fill your block before switching

Inflexible model of program synthesis

The above troubles point to a broader issue I have with Cranelift. In Inkwell and LLVM, generating JIT instructions and manipulating basic blocks feels very much like writing real source code: you can insert some instructions here, move the cursor and finish one idea, then append or prepend instructions where it makes sense for user-defined workloads as they're needed, and verify everything when one is ready. This enables writing JIT instructions just like how one would, for example, type out the loop variable and open and close brackets for a for-loop in one's editor before thinking about how to fill in the body. In Cranelift, the simple panic message you have to fill your block before switching tells me that this fine-grained but intuitive level of incremental edits is currently not possible.

I view this as a major shortcoming for JIT API, since if I already had a fully-ordered instruction-by-instruction understanding of what my user-defined program will do, I would be necessarily be repeating a huge amount of work that other JIT libraries like Inkwell and LLVM help me with. The restriction to fill a basic block all in one go may be appropriate for straightforward AST traversals with simple semantics, but it hugely complicates other workflows where single parts of user-defined code map to multiple instructions in multiple blocks. These troubles are compounded by the confusion around how to specify and populate the entry block as explained above.

I encourage removing these restrictions and supporting more dynamic workflows where blocks and instructions are created and revised on the fly, with the ability to insert, prepend and append instructions within blocks similarly to how a human would write them in an incremental, out-of-order manner. The mental model of JIT compilation that has served me well using Inkwell is one where JIT instructions and basic blocks are a fluid work-in-progress authoring format, and validation and rule enforcement come during finalization but no sooner.

Lack of familiar names in debug output

A key part of my JIT development workflow is printing out the IR when anything goes wrong and inspecting it. Inkwell's JIT API can do this quite nicely since everything ultimately models LLVM IR and most things need a name up front in the API. Here's what some sample LLVM IR might look like printed out as part of my development process:

define void @my_function(ptr %ptr_ptr_dst, i64 %dst_len, ...) {
entry:
  %arg_ptr_retv = call ptr @argument_pointer_wrapper(ptr %context_ptr, i64 527665334732589126)
  %ptr1 = getelementptr i64, ptr %arg_ptr_retv, i64 1
  --- snip ---
  %len_is_zero = icmp eq i64 %dst_len, 0

check_startover:                                  ; No predecessors!
  %init_flag = load i8, ptr %ptr_init_flag, align 1
  %was_init = icmp eq i8 %init_flag, 1
  br i1 %was_init, label %resume, label %startover

startover:                                        ; preds = %check_startover
  store i8 1, ptr %ptr_init_flag, align 1
  br label %pre_loop

resume:                                           ; preds = %check_startover
  br label %pre_loop

pre_loop:                                         ; preds = %resume, %startover
  br label %loop_begin

loop_begin:                                       ; preds = %loop_end, %pre_loop
  %loop_counter = phi i64 [ 0, %pre_loop ], [ %loop_counter_inc, %loop_end ]
  %loop_counter_inc = add i64 %loop_counter, 1
  %ptr_local_val = alloca float, align 4
  --- snip ---

In this snippet, I've forgotten a jump instruction from the entry block to check_startover and the IR printout even points this out with a helpful No predecessors! comment. The immediate usefulness of this stems largely from having given blocks and instructions human-readab
[message truncated]

Stream: git-wasmtime

Topic: wasmtime / issue #10766 Impressions from a first-time user

Wasmtime GitHub notifications bot (May 11 2025 at 15:34):

Multiple distracting TODO's in the JIT demo

Scattered documentation due to crate separation

Difficulties with the first block of a function

Inflexible model of program synthesis

Lack of familiar names in debug output