glebpom edited PR #13975.
glebpom edited PR #13975:
Summary
Handle JIT functions and data allocated outside x86_64's signed 32-bit
PC-relative range.This change:
- Routes out-of-range direct calls and tail jumps through veneers.
- Uses far relocation forms when materializing data and function addresses.
Fixes #4000.
Motivation
A
JITMemoryProvidermay allocate generated code, data objects, and functions
independently. It does not guarantee that these allocations will be within
±2 GiB of one another.Both relevant x86_64 relocation forms use signed 32-bit PC-relative
displacements:
X86CallPCRel4performs a direct call or jump.X86PCRel4produces the address of a symbol.If the target is outside that range,
finalize_definitionspreviously panicked
when converting the displacement toi32.The problem was observed in Nervix while compiling Roto UDFs. The CI run
intermittently failed with bothTryFromIntError(NegOverflow)and
TryFromIntError(PosOverflow):https://github.com/nervix-io/nervix/actions/runs/30129378927/job/89600292950?pr=57#step:17:56674
After adding call veneers, a follow-up run showed that Roto could still fail
while materializing addresses of separately allocated data:https://github.com/nervix-io/nervix/actions/runs/30147477548/job/89651977365?pr=57
Implementation
Calls and tail jumps
- Reserve a worst-case veneer slot for each
X86CallPCRel4relocation.- Keep in-range calls and jumps direct.
Redirect out-of-range control transfers through a
jmp qword ptr [rip]veneer containing the absolute target address.Use an exact 16-byte veneer slot, sufficient for both AArch64 and x86_64.
- Create a separate, densely packed veneer for each out-of-range target.
Address materialization
A branch veneer cannot fix an address-producing instruction: it would produce
the address of the veneer rather than the requested symbol.Instead, before compiling a function in
cranelift-jit:
Mark symbolic global values used by
symbol_valueas non-colocated, causing
the backend to use a far address-materialization sequence.Give
func_addrinstructions separate non-colocated function references.- Preserve the original colocated function references used by direct calls, so
those calls remain direct and can use veneers when necessary.On x86_64, these far address operations use full-width
Abs8relocations and
therefore do not depend on the code and target being within ±2 GiB.Relocation handling
- Reserve
X86CallPCRel4for control transfers.Use
X86PCRel4for RIP-relative address materialization where proximity is
guaranteed, such as object emission.Teach the x86_64 text-section builder to resolve
X86PCRel4.- Fix the AArch64 veneer bounds assertion.
- Update x86_64 filetest and object-relocation expectations.
Testing
The JIT integration tests use a custom memory provider that deterministically
places allocations on opposite sides of a virtual address range larger than
2 GiB.Call and veneer coverage includes:
- A compiler-generated call with a negative out-of-range displacement.
- A tail jump with a positive out-of-range displacement.
- A mixture of one near and two far calls.
- Multiple densely packed veneers.
- Veneer instructions, absolute targets, and allocation sizing.
- Execution of the resulting JIT code.
Address-materialization coverage includes:
A read-only data object allocated more than 2 GiB from the code returning its
address.A function allocated more than 2 GiB from the code materializing its address.
A shared function reference used by both
callandfunc_addr, verifying
that the call remainsX86CallPCRel4while the address usesAbs8.Execution of the generated code and verification of the returned pointers.
Additional tests verify:
LoadExtNameuses the non-branchX86PCRel4relocation for near addresses.MachTextSectionBuilderresolves that relocation correctly.ELF, Mach-O, and COFF objects use non-branch relocations for address
materialization.Updated x86_64 filetest output.
Commands run:
cargo test -p cranelift-codegen --libcargo test -p cranelift-jitcargo test -p cranelift-objectcargo test -p cranelift-tools --test filetests -- --nocapturecargo clippy -p cranelift-codegen -p cranelift-jit -p cranelift-object --tests -- -D warningscargo fmt --all -- --checkAI-assisted development
Claude Code assisted with the initial call-veneer implementation.
OpenAI Codex assisted with auditing the fix, recovering and evaluating stashed
work, strengthening the tests, implementing the address-materialization fix,
and porting the changes to the current main branch.All AI-assisted code and text were reviewed by the human contributor, who
understands the changes and accepts full responsibility for the contribution.
glebpom edited PR #13975:
Summary
Handle JIT functions and data allocated outside x86_64's signed 32-bit PC-relative range.
This change:
- Routes out-of-range direct calls and tail jumps through veneers.
- Uses far relocation forms when materializing data and function addresses.
Fixes #4000.
Motivation
A
JITMemoryProvidermay allocate generated code, data objects, and functions independently. It does not guarantee that these allocations will be within ±2 GiB of one another.Both relevant x86_64 relocation forms use signed 32-bit PC-relative displacements:
X86CallPCRel4performs a direct call or jump.X86PCRel4produces the address of a symbol.If the target is outside that range,
finalize_definitionspreviously panicked when converting the displacement toi32.The problem was observed in Nervix while compiling Roto UDFs. The CI run intermittently failed with both
TryFromIntError(NegOverflow)andTryFromIntError(PosOverflow):https://github.com/nervix-io/nervix/actions/runs/30129378927/job/89600292950?pr=57#step:17:56674
After adding call veneers, a follow-up run showed that Roto could still fail while materializing addresses of separately allocated data:
https://github.com/nervix-io/nervix/actions/runs/30147477548/job/89651977365?pr=57
Implementation
Calls and tail jumps
- Reserve a worst-case veneer slot for each
X86CallPCRel4relocation.- Keep in-range calls and jumps direct.
Redirect out-of-range control transfers through a
jmp qword ptr [rip]veneer containing the absolute target address.Use an exact 16-byte veneer slot, sufficient for both AArch64 and x86_64.
- Create a separate, densely packed veneer for each out-of-range target.
Address materialization
A branch veneer cannot fix an address-producing instruction: it would produce the address of the veneer rather than the requested symbol.
Instead, before compiling a function in
cranelift-jit:
Mark symbolic global values used by
symbol_valueas non-colocated, causing
the backend to use a far address-materialization sequence.Give
func_addrinstructions separate non-colocated function references.- Preserve the original colocated function references used by direct calls, so
those calls remain direct and can use veneers when necessary.On x86_64, these far address operations use full-width
Abs8relocations and therefore do not depend on the code and target being within ±2 GiB.Relocation handling
- Reserve
X86CallPCRel4for control transfers.Use
X86PCRel4for RIP-relative address materialization where proximity is
guaranteed, such as object emission.Teach the x86_64 text-section builder to resolve
X86PCRel4.- Fix the AArch64 veneer bounds assertion.
- Update x86_64 filetest and object-relocation expectations.
Testing
The JIT integration tests use a custom memory provider that deterministically places allocations on opposite sides of a virtual address range larger than 2 GiB.
Call and veneer coverage includes:
- A compiler-generated call with a negative out-of-range displacement.
- A tail jump with a positive out-of-range displacement.
- A mixture of one near and two far calls.
- Multiple densely packed veneers.
- Veneer instructions, absolute targets, and allocation sizing.
- Execution of the resulting JIT code.
Address-materialization coverage includes:
- A read-only data object allocated more than 2 GiB from the code returning its address.
- A function allocated more than 2 GiB from the code materializing its address.
A shared function reference used by both
callandfunc_addr, verifying
that the call remainsX86CallPCRel4while the address usesAbs8.Execution of the generated code and verification of the returned pointers.
Additional tests verify:
LoadExtNameuses the non-branchX86PCRel4relocation for near addresses.MachTextSectionBuilderresolves that relocation correctly.ELF, Mach-O, and COFF objects use non-branch relocations for address
materialization.Updated x86_64 filetest output.
Commands run:
cargo test -p cranelift-codegen --libcargo test -p cranelift-jitcargo test -p cranelift-objectcargo test -p cranelift-tools --test filetests -- --nocapturecargo clippy -p cranelift-codegen -p cranelift-jit -p cranelift-object --tests -- -D warningscargo fmt --all -- --checkAI-assisted development
Claude Code assisted with the initial call-veneer implementation.
OpenAI Codex assisted with auditing the fix, recovering and evaluating stashed work, strengthening the tests, implementing the address-materialization fix, and porting the changes to the current main branch.
All AI-assisted code and text were reviewed by the human contributor, who understands the changes and accepts full responsibility for the contribution.
glebpom edited PR #13975:
Summary
Handle JIT functions and data allocated outside x86_64's signed 32-bit PC-relative range.
This change:
- Routes out-of-range direct calls and tail jumps through veneers.
- Uses far relocation forms when materializing data and function addresses.
Fixes #4000.
Motivation
A
JITMemoryProvidermay allocate generated code, data objects, and functions independently. It does not guarantee that these allocations will be within ±2 GiB of one another.Both relevant x86_64 relocation forms use signed 32-bit PC-relative displacements:
X86CallPCRel4performs a direct call or jump.X86PCRel4produces the address of a symbol.If the target is outside that range,
finalize_definitionspreviously panicked when converting the displacement toi32.The problem was observed in Nervix while compiling Roto UDFs. The CI run intermittently failed with both
TryFromIntError(NegOverflow)andTryFromIntError(PosOverflow):https://github.com/nervix-io/nervix/issues/60
After adding call veneers, a follow-up run showed that Roto could still fail while materializing addresses of separately allocated data:
https://github.com/nervix-io/nervix/actions/runs/30147477548/job/89651977365?pr=57
Implementation
Calls and tail jumps
- Reserve a worst-case veneer slot for each
X86CallPCRel4relocation.- Keep in-range calls and jumps direct.
Redirect out-of-range control transfers through a
jmp qword ptr [rip]veneer containing the absolute target address.Use an exact 16-byte veneer slot, sufficient for both AArch64 and x86_64.
- Create a separate, densely packed veneer for each out-of-range target.
Address materialization
A branch veneer cannot fix an address-producing instruction: it would produce the address of the veneer rather than the requested symbol.
Instead, before compiling a function in
cranelift-jit:
Mark symbolic global values used by
symbol_valueas non-colocated, causing
the backend to use a far address-materialization sequence.Give
func_addrinstructions separate non-colocated function references.- Preserve the original colocated function references used by direct calls, so
those calls remain direct and can use veneers when necessary.On x86_64, these far address operations use full-width
Abs8relocations and therefore do not depend on the code and target being within ±2 GiB.Relocation handling
- Reserve
X86CallPCRel4for control transfers.Use
X86PCRel4for RIP-relative address materialization where proximity is
guaranteed, such as object emission.Teach the x86_64 text-section builder to resolve
X86PCRel4.- Fix the AArch64 veneer bounds assertion.
- Update x86_64 filetest and object-relocation expectations.
Testing
The JIT integration tests use a custom memory provider that deterministically places allocations on opposite sides of a virtual address range larger than 2 GiB.
Call and veneer coverage includes:
- A compiler-generated call with a negative out-of-range displacement.
- A tail jump with a positive out-of-range displacement.
- A mixture of one near and two far calls.
- Multiple densely packed veneers.
- Veneer instructions, absolute targets, and allocation sizing.
- Execution of the resulting JIT code.
Address-materialization coverage includes:
- A read-only data object allocated more than 2 GiB from the code returning its address.
- A function allocated more than 2 GiB from the code materializing its address.
A shared function reference used by both
callandfunc_addr, verifying
that the call remainsX86CallPCRel4while the address usesAbs8.Execution of the generated code and verification of the returned pointers.
Additional tests verify:
LoadExtNameuses the non-branchX86PCRel4relocation for near addresses.MachTextSectionBuilderresolves that relocation correctly.ELF, Mach-O, and COFF objects use non-branch relocations for address
materialization.Updated x86_64 filetest output.
Commands run:
cargo test -p cranelift-codegen --libcargo test -p cranelift-jitcargo test -p cranelift-objectcargo test -p cranelift-tools --test filetests -- --nocapturecargo clippy -p cranelift-codegen -p cranelift-jit -p cranelift-object --tests -- -D warningscargo fmt --all -- --checkAI-assisted development
Claude Code assisted with the initial call-veneer implementation.
OpenAI Codex assisted with auditing the fix, recovering and evaluating stashed work, strengthening the tests, implementing the address-materialization fix, and porting the changes to the current main branch.
All AI-assisted code and text were reviewed by the human contributor, who understands the changes and accepts full responsibility for the contribution.
glebpom updated PR #13975.
glebpom edited PR #13975:
Summary
Handle JIT functions and data allocated outside x86_64's signed 32-bit PC-relative range.
This change:
- Routes out-of-range direct calls and tail jumps through veneers.
- Uses far relocation forms when materializing data and function addresses.
Fixes #4000.
Motivation
A
JITMemoryProvidermay allocate generated code, data objects, and functions independently. It does not guarantee that these allocations will be within ±2 GiB of one another.Both relevant x86_64 relocation forms use signed 32-bit PC-relative displacements:
X86CallPCRel4performs a direct call or jump.X86PCRel4produces the address of a symbol.If the target is outside that range,
finalize_definitionspreviously panicked when converting the displacement toi32.The problem was observed in Nervix while compiling Roto UDFs. The CI run intermittently failed with both
TryFromIntError(NegOverflow)andTryFromIntError(PosOverflow):https://github.com/nervix-io/nervix/issues/60
After adding call veneers, a follow-up run showed that Roto could still fail while materializing addresses of separately allocated data:
https://github.com/nervix-io/nervix/actions/runs/30147477548/job/89651977365?pr=57
Finally, after backporting 2334e132e2fcca79638d3051887153218dce054f the build succeeded https://github.com/nervix-io/nervix/actions/runs/30169535347/job/89708162970?pr=57
Implementation
Calls and tail jumps
- Reserve a worst-case veneer slot for each
X86CallPCRel4relocation.- Keep in-range calls and jumps direct.
Redirect out-of-range control transfers through a
jmp qword ptr [rip]veneer containing the absolute target address.Use an exact 16-byte veneer slot, sufficient for both AArch64 and x86_64.
- Create a separate, densely packed veneer for each out-of-range target.
Address materialization
A branch veneer cannot fix an address-producing instruction: it would produce the address of the veneer rather than the requested symbol.
Instead, before compiling a function in
cranelift-jit:
Mark symbolic global values used by
symbol_valueas non-colocated, causing
the backend to use a far address-materialization sequence.Give
func_addrinstructions separate non-colocated function references.- Preserve the original colocated function references used by direct calls, so
those calls remain direct and can use veneers when necessary.On x86_64, these far address operations use full-width
Abs8relocations and therefore do not depend on the code and target being within ±2 GiB.Relocation handling
- Reserve
X86CallPCRel4for control transfers.Use
X86PCRel4for RIP-relative address materialization where proximity is
guaranteed, such as object emission.Teach the x86_64 text-section builder to resolve
X86PCRel4.- Fix the AArch64 veneer bounds assertion.
- Update x86_64 filetest and object-relocation expectations.
Testing
The JIT integration tests use a custom memory provider that deterministically places allocations on opposite sides of a virtual address range larger than 2 GiB.
Call and veneer coverage includes:
- A compiler-generated call with a negative out-of-range displacement.
- A tail jump with a positive out-of-range displacement.
- A mixture of one near and two far calls.
- Multiple densely packed veneers.
- Veneer instructions, absolute targets, and allocation sizing.
- Execution of the resulting JIT code.
Address-materialization coverage includes:
- A read-only data object allocated more than 2 GiB from the code returning its address.
- A function allocated more than 2 GiB from the code materializing its address.
A shared function reference used by both
callandfunc_addr, verifying
that the call remainsX86CallPCRel4while the address usesAbs8.Execution of the generated code and verification of the returned pointers.
Additional tests verify:
LoadExtNameuses the non-branchX86PCRel4relocation for near addresses.MachTextSectionBuilderresolves that relocation correctly.ELF, Mach-O, and COFF objects use non-branch relocations for address
materialization.Updated x86_64 filetest output.
Commands run:
cargo test -p cranelift-codegen --libcargo test -p cranelift-jitcargo test -p cranelift-objectcargo test -p cranelift-tools --test filetests -- --nocapturecargo clippy -p cranelift-codegen -p cranelift-jit -p cranelift-object --tests -- -D warningscargo fmt --all -- --checkAI-assisted development
Claude Code assisted with the initial call-veneer implementation.
OpenAI Codex assisted with auditing the fix, recovering and evaluating stashed work, strengthening the tests, implementing the address-materialization fix, and porting the changes to the current main branch.
All AI-assisted code and text were reviewed by the human contributor, who understands the changes and accepts full responsibility for the contribution.
glebpom edited PR #13975:
Summary
Handle JIT functions and data allocated outside x86_64's signed 32-bit PC-relative range.
This change:
- Routes out-of-range direct calls and tail jumps through veneers.
- Uses far relocation forms when materializing data and function addresses.
Fixes #4000.
Motivation
A
JITMemoryProvidermay allocate generated code, data objects, and functions independently. It does not guarantee that these allocations will be within ±2 GiB of one another.Both relevant x86_64 relocation forms use signed 32-bit PC-relative displacements:
X86CallPCRel4performs a direct call or jump.X86PCRel4produces the address of a symbol.If the target is outside that range,
finalize_definitionspreviously panicked when converting the displacement toi32.The problem was observed in Nervix while compiling Roto UDFs. The CI run intermittently failed with both
TryFromIntError(NegOverflow)andTryFromIntError(PosOverflow):https://github.com/nervix-io/nervix/issues/60
After adding call veneers, a follow-up run showed that Roto could still fail while materializing addresses of separately allocated data:
https://github.com/nervix-io/nervix/actions/runs/30147477548/job/89651977365?pr=57
Finally, after backporting 2334e132e2fcca79638d3051887153218dce054f the build succeeded:
https://github.com/nervix-io/nervix/actions/runs/30169535347/job/89708162970?pr=57
Implementation
Calls and tail jumps
- Reserve a worst-case veneer slot for each
X86CallPCRel4relocation.- Keep in-range calls and jumps direct.
Redirect out-of-range control transfers through a
jmp qword ptr [rip]veneer containing the absolute target address.Use an exact 16-byte veneer slot, sufficient for both AArch64 and x86_64.
- Create a separate, densely packed veneer for each out-of-range target.
Address materialization
A branch veneer cannot fix an address-producing instruction: it would produce the address of the veneer rather than the requested symbol.
Instead, before compiling a function in
cranelift-jit:
Mark symbolic global values used by
symbol_valueas non-colocated, causing
the backend to use a far address-materialization sequence.Give
func_addrinstructions separate non-colocated function references.- Preserve the original colocated function references used by direct calls, so
those calls remain direct and can use veneers when necessary.On x86_64, these far address operations use full-width
Abs8relocations and therefore do not depend on the code and target being within ±2 GiB.Relocation handling
- Reserve
X86CallPCRel4for control transfers.Use
X86PCRel4for RIP-relative address materialization where proximity is
guaranteed, such as object emission.Teach the x86_64 text-section builder to resolve
X86PCRel4.- Fix the AArch64 veneer bounds assertion.
- Update x86_64 filetest and object-relocation expectations.
Testing
The JIT integration tests use a custom memory provider that deterministically places allocations on opposite sides of a virtual address range larger than 2 GiB.
Call and veneer coverage includes:
- A compiler-generated call with a negative out-of-range displacement.
- A tail jump with a positive out-of-range displacement.
- A mixture of one near and two far calls.
- Multiple densely packed veneers.
- Veneer instructions, absolute targets, and allocation sizing.
- Execution of the resulting JIT code.
Address-materialization coverage includes:
- A read-only data object allocated more than 2 GiB from the code returning its address.
- A function allocated more than 2 GiB from the code materializing its address.
A shared function reference used by both
callandfunc_addr, verifying
that the call remainsX86CallPCRel4while the address usesAbs8.Execution of the generated code and verification of the returned pointers.
Additional tests verify:
LoadExtNameuses the non-branchX86PCRel4relocation for near addresses.MachTextSectionBuilderresolves that relocation correctly.ELF, Mach-O, and COFF objects use non-branch relocations for address
materialization.Updated x86_64 filetest output.
Commands run:
cargo test -p cranelift-codegen --libcargo test -p cranelift-jitcargo test -p cranelift-objectcargo test -p cranelift-tools --test filetests -- --nocapturecargo clippy -p cranelift-codegen -p cranelift-jit -p cranelift-object --tests -- -D warningscargo fmt --all -- --checkAI-assisted development
Claude Code assisted with the initial call-veneer implementation.
OpenAI Codex assisted with auditing the fix, recovering and evaluating stashed work, strengthening the tests, implementing the address-materialization fix, and porting the changes to the current main branch.
All AI-assisted code and text were reviewed by the human contributor, who understands the changes and accepts full responsibility for the contribution.
glebpom edited PR #13975:
Summary
Handle JIT functions and data allocated outside x86_64's signed 32-bit PC-relative range.
This change:
- Routes out-of-range direct calls and tail jumps through veneers.
- Uses far relocation forms when materializing data and function addresses.
Fixes #4000.
Motivation
A
JITMemoryProvidermay allocate generated code, data objects, and functions independently. It does not guarantee that these allocations will be within ±2 GiB of one another.Both relevant x86_64 relocation forms use signed 32-bit PC-relative displacements:
X86CallPCRel4performs a direct call or jump.X86PCRel4produces the address of a symbol.If the target is outside that range,
finalize_definitionspreviously panicked when converting the displacement toi32.The problem was observed in Nervix while compiling Roto UDFs. The CI run intermittently failed with both
TryFromIntError(NegOverflow)andTryFromIntError(PosOverflow):https://github.com/nervix-io/nervix/issues/60
After adding call veneers, a follow-up run showed that Roto could still fail while materializing addresses of separately allocated data:
https://github.com/nervix-io/nervix/actions/runs/30147477548/job/89651977365?pr=57
Finally, after backporting 2334e132e2fcca79638d3051887153218dce054f the build succeeded:
https://github.com/nervix-io/nervix/actions/runs/30169535347/job/89708162970?pr=57
Implementation
Calls and tail jumps
- Reserve a worst-case veneer slot for each
X86CallPCRel4relocation.- Keep in-range calls and jumps direct.
Redirect out-of-range control transfers through a
jmp qword ptr [rip]veneer containing the absolute target address.Use an exact 16-byte veneer slot, sufficient for both AArch64 and x86_64.
- Create a separate, densely packed veneer for each out-of-range target.
Address materialization
A branch veneer cannot fix an address-producing instruction: it would produce the address of the veneer rather than the requested symbol.
Instead, before compiling a function in
cranelift-jit:
Mark symbolic global values used by
symbol_valueas non-colocated, causing
the backend to use a far address-materialization sequence.Give
func_addrinstructions separate non-colocated function references.- Preserve the original colocated function references used by direct calls, so
those calls remain direct and can use veneers when necessary.On x86_64, these far address operations use full-width
Abs8relocations and therefore do not depend on the code and target being within ±2 GiB.Relocation handling
- Reserve
X86CallPCRel4for control transfers.Use
X86PCRel4for RIP-relative address materialization where proximity is
guaranteed, such as object emission.Teach the x86_64 text-section builder to resolve
X86PCRel4.- Fix the AArch64 veneer bounds assertion.
- Update x86_64 filetest and object-relocation expectations.
Testing
The JIT integration tests use a custom memory provider that deterministically places allocations on opposite sides of a virtual address range larger than 2 GiB.
Call and veneer coverage includes:
- A compiler-generated call with a negative out-of-range displacement.
- A tail jump with a positive out-of-range displacement.
- A mixture of one near and two far calls.
- Multiple densely packed veneers.
- Veneer instructions, absolute targets, and allocation sizing.
- Execution of the resulting JIT code.
Address-materialization coverage includes:
- A read-only data object allocated more than 2 GiB from the code returning its address.
- A function allocated more than 2 GiB from the code materializing its address.
A shared function reference used by both
callandfunc_addr, verifying
that the call remainsX86CallPCRel4while the address usesAbs8.Execution of the generated code and verification of the returned pointers.
Additional tests verify:
LoadExtNameuses the non-branchX86PCRel4relocation for near addresses.MachTextSectionBuilderresolves that relocation correctly.ELF, Mach-O, and COFF objects use non-branch relocations for address
materialization.Updated x86_64 filetest output.
Commands run:
cargo test -p cranelift-codegen --libcargo test -p cranelift-jitcargo test -p cranelift-objectcargo test -p cranelift-tools --test filetests -- --nocapturecargo clippy -p cranelift-codegen -p cranelift-jit -p cranelift-object --tests -- -D warningscargo fmt --all -- --checkAI-assisted development
Claude Code assisted with the initial call-veneer implementation.
OpenAI Codex assisted with auditing the fix, recovering and evaluating stashed work, strengthening the tests, implementing the address-materialization fix, and porting the changes to the current main branch.
All AI-assisted code and text were reviewed by the human contributor, who understands the changes and accepts full responsibility for the contribution.
glebpom has marked PR #13975 as ready for review.
:memo: bjorn3 submitted PR review.
:speech_balloon: bjorn3 created PR review comment:
This feels like a hack to me. cranelift-jit shouldn't modify the function. A better solution is probably to just add support for PLT relocations and then tell Cranelift to emit PLT relocations.
glebpom updated PR #13975.
glebpom edited PR #13975:
Summary
Handle JIT functions and data allocated outside x86_64's signed 32-bit PC-relative range.
This change:
- Routes out-of-range direct calls and tail jumps through veneers.
- Materializes symbol addresses through GOT-relative loads that the JIT resolves itself, relaxing them to direct
leas when the symbol turns out to be in range.Fixes #4000.
Motivation
A
JITMemoryProvidermay allocate generated code, data objects, and functions independently. It does not guarantee that these allocations will be within ±2 GiB of one another.Both relevant x86_64 relocation forms use signed 32-bit PC-relative displacements:
X86CallPCRel4performs a direct call or jump.X86PCRel4produces the address of a symbol.If the target is outside that range,
finalize_definitionspreviously panicked when converting the displacement toi32.The problem was observed in Nervix while compiling Roto UDFs. The CI run intermittently failed with both
TryFromIntError(NegOverflow)andTryFromIntError(PosOverflow):https://github.com/nervix-io/nervix/actions/runs/30129378927/job/89600292950?pr=57#step:17:56674
After adding call veneers, a follow-up run showed that Roto could still fail while materializing addresses of separately allocated data:
https://github.com/nervix-io/nervix/actions/runs/30147477548/job/89651977365?pr=57
Implementation
The work is organized as two commits.
1. Calls and tail jumps: veneers
- Reserve a worst-case 16-byte veneer slot for each
X86CallPCRel4relocation.- Keep in-range calls and jumps direct.
- Redirect out-of-range control transfers through a
jmp qword ptr [rip]veneer containing the absolute target address, the same wayArm64Callis already handled.- Create a separate, densely packed veneer for each out-of-range transfer.
- Fix an off-by-one in the AArch64 veneer bounds assertion.
Redirecting through a veneer is only sound for control transfers, so
X86CallPCRel4is now reserved forcall/jmpdisplacements and this invariant is documented on the relocation. The RIP-relativeleaemitted byLoadExtNamefor near symbols now usesX86PCRel4instead;LabelUse::from_relocand the text-section builder learned to resolve it, and object files now get a PC-relative rather than a branch relocation for thelea, matching its semantics.2. Symbol addresses: a per-blob GOT
A branch veneer cannot fix an address-producing instruction: it would produce the address of the veneer rather than the requested symbol. Instead, as suggested by @bjorn3 in review, the JIT now uses GOT-based address materialization:
JITBuilderdefaults tois_pic=trueon x86_64 (andJITModulerejects PIC on other architectures), soLoadExtNameemitsmovq sym@GOTPCREL(%rip)loads withX86GOTPCRel4relocations.- The JIT reserves an 8-byte GOT entry per
X86GOTPCRel4relocation at the end of the blob's allocation ([code][veneers][GOT entries]), stores the absolute symbol address there at finalize time, and points the load's displacement at the entry, which is always in range.- When the symbol is within displacement range, the load is relaxed to a direct
lea(a one-bytemov→leaopcode rewrite), mirroring theR_X86_64_REX_GOTPCRELXrelaxation performed by linkers, so near symbol accesses don't pay for the indirection.X86CallPLTRel4, emitted for the ELF TLS model's__tls_get_addrcall, resolves likeX86CallPCRel4: direct when in range, through a veneer otherwise.Direct calls are unaffected: they keep their colocated references and use veneers when needed. GOT entries hold the symbol's real address, preserving pointer identity with the finalized definitions.
Testing
The JIT integration tests use a custom memory provider that deterministically places allocations on opposite sides of a virtual address range larger than 2 GiB.
Call and veneer coverage:
- A compiler-generated call with a negative out-of-range displacement.
- A tail jump with a positive out-of-range displacement.
- A mixture of one near and two far calls with distinct, densely packed veneers.
- Veneer instructions, absolute targets, and allocation sizing.
- Execution of the resulting JIT code.
Address-materialization coverage:
- A read-only data object and a function allocated more than 2 GiB from the code, verifying the
X86GOTPCRel4relocation, the GOT entry contents, pointer identity, and execution.- A shared function reference used by both
callandfunc_addr, verifying the call remainsX86CallPCRel4while the address goes through the GOT.- The
mov→learelaxation for a near symbol, verifying the rewritten instruction points directly at the data.LoadExtNamenear-symbol relocation kind,MachTextSectionBuilderresolution ofX86PCRel4, and non-branch address relocations for ELF, Mach-O, and COFF objects.The test ISA helper now enables
is_picon x86_64, resolving an old FIXME from when the JIT last supported PIC;libcall_functiontherefore exercises GOT resolution against real host symbols.Commands run:
cargo test -p cranelift-codegen --libcargo test -p cranelift-jitcargo test -p cranelift-objectcargo test -p cranelift-tools --test filetests -- --nocapturecargo clippy -p cranelift-codegen -p cranelift-jit -p cranelift-object --tests -- -D warningscargo fmt --all -- --checkNotes
AArch64 direct calls already use veneers, and its non-colocated materialization uses an inline literal, so it works at any distance. Its colocated
adrp+addsequence retains a pre-existing ±2 GiB assumption in theAarch64AdrPrelPgHi21handler; supportingis_picwith the AArch64 GOT relocations would be the analogous fix and is left as a follow-up.AI-assisted development
Claude Code assisted with the call-veneer implementation and with the GOT-based address materialization following the review suggestion. OpenAI Codex assisted with auditing the fix, recovering and evaluating stashed work, strengthening the tests, and porting the changes to the current main branch.
All AI-assisted code and text were reviewed by the human contributor, who understands the changes and accepts full responsibility for the contribution.
`
Last updated: Jul 29 2026 at 05:03 UTC