wasmtime / issue #14442 jit-icache-coherence: aarch64 cle... · git-wasmtime

Right, as Chris said, #12133 is where the aarch64 body was added. One thing my report got wrong: the cranelift-jit 0.124 failure was on a version where clear_cache did nothing at all on AArch64 (#3310), so it says nothing about dc cvau; the reproduction below does.

Reproduced on an AWS a1.medium (Graviton1, Cortex-A72), Debian 13, kernel 6.12.107+deb13-cloud-arm64, rustc 1.98.1. CTR_EL0 there is 0x8444c004: IDC=0, DIC=0, 64-byte lines.

The test writes a two-instruction function over one that has just run, publishes it the way cranelift-jit does (RW, write, clear_cache, RX, pipeline_flush_mt) and calls it. A call that returns the old value ran stale instructions. 200,000 iterations each:

flush	stale
none	197,935
`clear_cache` from 49.0.1	199,057
`dc cvau` by DminLine, `dsb ish`, `ic ivau` by IminLine, `dsb ish`, `isb`	0

The same loop in C on an RWX page, 1,000,000 iterations: ic ivau; dsb ish; isb gives 999,963 stale, 999,929 with a dsb ish in front, 0 with dc cvau first, 0 with __builtin___clear_cache.

Code in fresh pages doesn't hit this, since the kernel cleans a page when it is first mapped executable. Rewriting code in a page that has already executed does.

<details><summary><code>Cargo.toml</code> and <code>src/main.rs</code></summary>

[package]
name = "icache-reuse"
version = "0.1.0"
edition = "2024"

[dependencies]
wasmtime-internal-jit-icache-coherence = "=49.0.1"
libc = "0.2"

use std::ffi::c_void;
use wasmtime_internal_jit_icache_coherence::{clear_cache, pipeline_flush_mt};

// compiler-rt's __clear_cache sequence, for the control run
fn full_sync(p: *const u8, len: usize) {
    let ctr: u64;
    unsafe { core::arch::asm!("mrs {}, ctr_el0", out(reg) ctr) };
    let (s, e) = (p as usize, p as usize + len);
    if ctr & (1 << 28) == 0 {
        let l = 4usize << ((ctr >> 16) & 15);
        let mut a = s & !(l - 1);
        while a < e { unsafe { core::arch::asm!("dc cvau, {}", in(reg) a) }; a += l; }
    }
    unsafe { core::arch::asm!("dsb ish") };
    if ctr & (1 << 29) == 0 {
        let l = 4usize << (ctr & 15);
        let mut a = s & !(l - 1);
        while a < e { unsafe { core::arch::asm!("ic ivau, {}", in(reg) a) }; a += l; }
        unsafe { core::arch::asm!("dsb ish") };
    }
    unsafe { core::arch::asm!("isb") };
}

fn main() {
    let mode = std::env::args().nth(1).unwrap(); // none | upstream | full
    let iters: u32 = std::env::args().nth(2).unwrap().parse().unwrap();
    let page = 4096usize;
    let mem = unsafe {
        libc::mmap(std::ptr::null_mut(), page, libc::PROT_READ | libc::PROT_WRITE,
                   libc::MAP_PRIVATE | libc::MAP_ANONYMOUS, -1, 0)
    } as *mut u32;
    assert!(mem as isize != -1);
    let publish = |v: u32| unsafe {
        libc::mprotect(mem as *mut c_void, page, libc::PROT_READ | libc::PROT_WRITE);
        mem.write_volatile(0x5280_0000 | ((v & 0xffff) << 5)); // mov w0, #v
        mem.add(1).write_volatile(0xd65f_03c0); // ret
        match mode.as_str() {
            "upstream" => clear_cache(mem as *const c_void, 8).unwrap(),
            "full" => full_sync(mem as *const u8, 8),
            "none" => {}
            _ => panic!("mode"),
        }
        libc::mprotect(mem as *mut c_void, page, libc::PROT_READ | libc::PROT_EXEC);
        pipeline_flush_mt().unwrap();
    };
    let f: extern "C" fn() -> u32 = unsafe { std::mem::transmute(mem) };
    publish(0);
    let mut stale = 0u32;
    for i in 1..=iters {
        let v = i & 0xffff;
        f(); // the old function is now in the I-cache
        publish(v);
        if f() != v { stale += 1; }
    }
    println!("mode={mode} iters={iters} stale={stale}");
}

#14454 follows the compiler-rt sequence on non-Apple AArch64 and calls sys_icache_invalidate on Apple, since mrs ctr_el0 at EL0 gets SIGILL on macOS. With the PR's final commit on an a1.xlarge (Cortex-A72), 200,000 iterations: 49.0.1's clear_cache 146,886 stale, the PR's clear_cache 0 (the count varies between runs; 180,425 on an a1.medium with the previous revision).

Stream: git-wasmtime

Topic: wasmtime / issue #14442 jit-icache-coherence: aarch64 cle...

Wasmtime GitHub notifications bot (Sep 28 2026 at 19:46):

Wasmtime GitHub notifications bot (Sep 29 2026 at 21:28):

Wasmtime GitHub notifications bot (Sep 29 2026 at 22:08):

Wasmtime GitHub notifications bot (Sep 29 2026 at 22:23):

Wasmtime GitHub notifications bot (Sep 30 2026 at 12:53):

Wasmtime GitHub notifications bot (Oct 05 2026 at 17:06):