wasmtime / PR #13265 c-api: Defensively zero-extend kind ... · git-wasmtime

Stream: git-wasmtime

Topic: wasmtime / PR #13265 c-api: Defensively zero-extend kind ...

Wasmtime GitHub notifications bot (May 04 2026 at 16:38):

glima opened PR #13265 from glima:fix-wasm-valtype-new-abi to bytecodealliance:main:

Summary

The SysV AMD64 ABI does not require callers to zero- or sign-extend sub-int arguments. GCC, for example, emits mov $0xffffff81, %edi when passing WASM_FUNCREF (= 129) to wasm_valtype_new, leaving the upper bytes of %rdi holding the sign-extended representation.

Recent rustc/LLVM versions (observed with 1.95.0) compile the match over kind: u8 as a comparison against the full 32-bit %edi rather than %dil. Because 0xffffff81 != 0x81, the WASM_FUNCREF arm does not match and the function aborts with unexpected value type kind, even though the C caller supplied a perfectly valid kind.

This patch forces a memory round-trip via std::ptr::read_volatile so LLVM emits a movzbl load before the comparisons. Cost: one extra load per call. Benefit: robustness against any conformant C caller.

Reproduction

Observed in the wild: tree-sitter's ts_wasm_store_new calls wasm_valtype_new(WASM_FUNCREF) while embedded in the Zed editor. With rustc 1.95.0 + gcc 14 on Linux x86_64, every file open aborts the editor.

Disassembly of the unpatched function shows:

\`\`\`asm
mov %dil, 0x7(%rsp) ; stores low byte (correct)
cmp \$0x4, %edi ; :cross_mark: compares full 32-bit %edi
cmp \$0x80, %edi ; :cross_mark: also full register
cmp \$0x81, %edi ; :cross_mark: 0xffffff81 != 0x81 → falls through to abort
\`\`\`

After the patch:

\`\`\`asm
mov %dil, 0x6(%rsp)
movzbl 0x6(%rsp), %eax ; :check: zero-extending load
cmp \$0x80, %eax ; :check: now sees 0x81
cmp \$0x81, %eax ; :check: matches WASM_FUNCREF
\`\`\`

Notes

Related: rust-lang/rust#97463

Other extern "C" entry points in the C API that take small integer args and match on them may have the same latent bug; this PR only fixes the demonstrated case. Happy to extend the fix to other call sites if maintainers prefer.

Wasmtime GitHub notifications bot (May 04 2026 at 16:38):

glima requested cfallin for a review on PR #13265.

Wasmtime GitHub notifications bot (May 04 2026 at 16:38):

glima requested wasmtime-core-reviewers for a review on PR #13265.

Wasmtime GitHub notifications bot (May 04 2026 at 16:45):

:repeat: cfallin submitted PR review:

@glima I don't think this is the right fix: we should not be baking in workarounds with volatile loads and knowledge of the underlying ABI.

On the surface, this sounds like it could be a compiler bug. However, the comment in your PR links to a rust-lang issue that was closed/fixed in 2022, so is in all modern Rust versions.

Your PR description mentions Rust 1.95.0. Can you present evidence (e.g. Godbolt links) showing that Rust code taking an i8 actually relies on the upper bits of the register? And perhaps a compilation with an earlier Rust compiler version showing that 1.95.0 regresses? As-is, the evidence in your PR does not show anything like this.

Thanks!

Wasmtime GitHub notifications bot (May 04 2026 at 16:45):

cfallin commented on PR #13265:

(And cc @alexcrichton for ABI / C-API concerns)

Wasmtime GitHub notifications bot (May 04 2026 at 16:47):

cfallin commented on PR #13265:

(Put another way: if this is a compiler bug, let's see the recent upstream issue describing it, not a 4-year-old issue. And the right fix to work around it is probably not this; I would personally suggest we switch to i32 if i8 is unreliable wrt compiler bugs)

Wasmtime GitHub notifications bot (May 04 2026 at 16:54):

glima commented on PR #13265:

Thanks for the quick review @cfallin — and you're right to push back on the framing. Let me share what I dug up.

Not a recent rustc regression

I tested with the same minimal Rust function (mirroring wasm_valtype_new) across four compilers; codegen is identical, so the "1.95.0 regresses" framing in my PR description is wrong — apologies.

rustc match on kind: u8 codegen

1.92.0 cmpl $3, %edi / cmpl $128, %edi / cmpl $129, %edi

1.93.0 same

1.94.1 same

1.95.0 same

Minimal repro:
#![allow(non_camel_case_types)]
pub type wasm_valkind_t = u8;
#[no_mangle]
pub extern "C" fn classify(kind: wasm_valkind_t) -> u32 {
    match kind {
        0 => 1, 1 => 2, 2 => 3, 3 => 4,
        128 => 5,
        129 => 6,
        _ => panic!("unexpected kind: {}", kind),
    }
}
rustc -O --emit=asm (1.95.0):
classify:
        subq    $24, %rsp
        movb    %dil, 7(%rsp)
        cmpl    $3, %edi          ; <-- 32-bit compare on %edi, not %dil
        ja      .LBB0_1
        movl    %edi, %eax
        leaq    .LJTI0_0(%rip), %rcx
        movslq  (%rcx,%rax,4), %rax
        addq    %rcx, %rax
        jmpq    *%rax
.LBB0_1:
        cmpl    $128, %edi        ; <-- ditto
        je      .LBB0_10
        cmpl    $129, %edi        ; <-- ditto: 0xffffff81 != 0x81 → falls through
        jne     .LBB0_3
        ...
So this isn't a regression at all — it's longstanding behavior. The PR description was wrong on that point.

What GCC actually emits

Caller (gcc 16.0.1, -O2):
typedef uint8_t wasm_valkind_t;
#define WASM_FUNCREF 129
extern uint32_t classify(wasm_valkind_t kind);

uint32_t go(void)               { return classify(WASM_FUNCREF); }
uint32_t go_var(wasm_valkind_t k) { return classify(k); }
go:
        movl    $-127, %edi          ; = 0xffffff81 — sign-extended into %edi
        jmp     classify
go_var:
        movsbl  %dil, %edi           ; SIGN-extends %dil into %edi (!)
        jmp     classify
The constant-call form is especially telling: GCC has full visibility that the C type is uint8_t and the value is the positive 129, yet it still emits the sign-extended form. (WASM_FUNCREF lives in enum wasm_valkind_enum in wasm.h, but the implicit conversion to the uint8_t parameter type should have been 0x81, not 0xffffff81.) movsbl for the variable case is harder to defend.

End-to-end: linking the Rust cdylib to the C main aborts with unexpected kind: 129, exactly matching the in-the-wild Zed/tree-sitter crash.

So who's at fault?

The current x86-64 psABI (post-2018 clarification) does say the caller is responsible for sign/zero-extending sub-int integer arguments — which would put GCC in the wrong here. But the wording is contested, both compilers have shipped this behavior unchanged for years, and in practice this falls into the "ABI sharp edge" bucket: the only side that can actually fix the user-visible crash without coordinating two upstreams is wasmtime.

I think you're right that read_volatile is the wrong shape of fix — it advertises "compiler bug workaround" when really it's robustness against an ABI ambiguity.

On switching to i32 / u32

Just widening the parameter doesn't fix it on its own — I tested this:
#[no_mangle]
pub extern "C" fn classify(kind: u32) -> u32 {
    match kind { 0=>1, 1=>2, 2=>3, 3=>4, 128=>5, 129=>6, _ => panic!("unexpected kind: {}", kind) }
}
→ still aborts, this time with unexpected kind: 4294967169 (= 0xffffff81). Rust just sees the un-extended register as a u32.

What works is u32 plus an explicit truncation:
#[no_mangle]
pub extern "C" fn classify(kind: u32) -> u32 {
    match kind as u8 { 0=>1, 1=>2, 2=>3, 3=>4, 128=>5, 129=>6, _ => panic!(...) }
}
→ returns 6 as expected.

That is arguably cleaner than read_volatile (no unsafe, no volatile, no comment about ABI internals — just a normalize-on-entry idiom). Happy to respin the PR that way if you prefer. The downside is it changes the Rust signature away from the literal C declaration, which someone may notice in bindgen-style consumers; but for no_mangle extern "C" entry points it's equivalent on the wire.

Latent surface

Either way, there are other extern "C" entry points in c-api that take small-int args and match/index on them (wasm_mutability_t mutators, the various kind getters, etc.). Whichever fix shape we pick, I can sweep the rest in the same PR if you'd like.

Let me know which direction you'd like — I'll respin.

rustc	match on `kind: u8` codegen
1.92.0	`cmpl $3, %edi` / `cmpl $128, %edi` / `cmpl $129, %edi`
1.93.0	same
1.94.1	same
1.95.0	same

Wasmtime GitHub notifications bot (May 04 2026 at 17:02):

alexcrichton commented on PR #13265:

Is your gcc perhaps misconfigured and/or testing some unreleased version and/or using some special flag or something like that? In this godbolt example Rust defines an 8-bit u8 argument as i8 noundef zeroext %a which means that ABI-wise it's expected to be zero-extended. That's why Rust doesn't zero-extend.

However the GCC example above is using unsigned constants, not signed constants, and it's not reproducing the assembly that you're seeing. Godbolt has GCC 16.1 which differs from your 16.0.1, so I don't know what's going on there.

This is definitely an ABI mismatch problem and something that I think is worth fixing, but not via workaround on the Rust side IMO. We need to figure out why the ABIs are differing and fix that at the root, then consider temporary workarounds from there. For now I think the main open question is @glima why is your GCC producing different assembly than Godbolt?

Wasmtime GitHub notifications bot (May 04 2026 at 17:04):

alexcrichton commented on PR #13265:

(Also, as a side note, this is a lot of text to wade through which looks AI-generated, if you wouldn't mind keeping the AI tooling outside of human conversations that'd be appreciated, it's just quite a lot to sift through)

Wasmtime GitHub notifications bot (May 04 2026 at 17:06):

cfallin commented on PR #13265:

@glima a meta-question: are you using an LLM to generate this issue and replies? (Wall of text, Markdown formatting tells, "You're right to push back -- here's what I found")

If so, I will have to ask you to please follow Bytecode Alliance's AI tool use policy, which explicitly requires a human to be in the loop and to review output of LLMs. A wall of text is an "extractive contribution" as described in that policy. You need to do the work of understanding and summarizing in a more concise way, and talk as a human with your fellow humans rather than throwing your agent at us. Fine for debugging and fact-finding but you need to be the interface yourself. Thanks!

Wasmtime GitHub notifications bot (May 04 2026 at 17:07):

cfallin commented on PR #13265:

Also, re: i32/u32 -- one would need to switch both the Rust side and the C side. It looks like you experimented only with the type on the Rust side. Then if a plain u32 doesn't make it through a call from C to Rust, we have much much deeper problems and should absolutely address that upstream.

Wasmtime GitHub notifications bot (May 04 2026 at 17:56):

glima commented on PR #13265:

Hi @cfallin, @alexcrichton, @tomstuart et al. Ty for the prompt feedback!

Human-in-the-loop here. Yes, I could not work that fast without AI, it's obvious. Also in the middle of work here, just trying to be possibly useful upstream in the meantime.

So verdict is this is likely a nothing-burger at this level -> compiler issue for Fedora 44:

<back to AI>

So we have a tight bracket now:

┌─────────────────────────────────────────────────────┬─────────────────────┐
│ GCC │ Result │
├─────────────────────────────────────────────────────┼─────────────────────┤
│ 13.2.0 (other box) │ :check: correct │
├─────────────────────────────────────────────────────┼─────────────────────┤
│ 16.1 release (Godbolt) │ :check: correct │
├─────────────────────────────────────────────────────┼─────────────────────┤
│ 17 trunk 20260504 (Godbolt) │ :check: correct │
├─────────────────────────────────────────────────────┼─────────────────────┤
│ 16.0.1-0.10.fc44 snapshot 20260321 (your local) │ :cross_mark: sign-extends │
└─────────────────────────────────────────────────────┴─────────────────────┘

Only your specific Fedora 44 prerelease snapshot is affected. Verdict for the PR is solid: not a wasmtime issue, not a generic
GCC issue — a transient bug in one Fedora rawhide gcc snapshot, already gone by the 16.1 release. Worth a Fedora bugzilla note
but the PR should be withdrawn.

I'll close the PR now and suck up my local fix till the compiler gets fixed on my distro.

Ty again!

Wasmtime GitHub notifications bot (May 04 2026 at 17:56):

:cross_mark: glima closed without merge PR #13265.

Wasmtime GitHub notifications bot (May 05 2026 at 14:34):

glima commented on PR #13265:

~ ❯ gcc --version
gcc (GCC) 16.1.1 20260501 (Red Hat 16.1.1-1)
Copyright (C) 2026 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

just landed today. will give it a try soon

Last updated: Jun 01 2026 at 09:49 UTC

Stream: git-wasmtime

Topic: wasmtime / PR #13265 c-api: Defensively zero-extend kind ...

Wasmtime GitHub notifications bot (May 04 2026 at 16:38):

Summary

Reproduction

Notes

Wasmtime GitHub notifications bot (May 04 2026 at 16:38):

Wasmtime GitHub notifications bot (May 04 2026 at 16:38):

Wasmtime GitHub notifications bot (May 04 2026 at 16:45):

Wasmtime GitHub notifications bot (May 04 2026 at 16:45):

Wasmtime GitHub notifications bot (May 04 2026 at 16:47):

Wasmtime GitHub notifications bot (May 04 2026 at 16:54):

Not a recent rustc regression

What GCC actually emits

So who's at fault?

On switching to i32 / u32

Latent surface

Wasmtime GitHub notifications bot (May 04 2026 at 17:02):

Wasmtime GitHub notifications bot (May 04 2026 at 17:04):

Wasmtime GitHub notifications bot (May 04 2026 at 17:06):

Wasmtime GitHub notifications bot (May 04 2026 at 17:07):

Wasmtime GitHub notifications bot (May 04 2026 at 17:56):

Wasmtime GitHub notifications bot (May 04 2026 at 17:56):

Wasmtime GitHub notifications bot (May 05 2026 at 14:34):

On switching to `i32` / `u32`