Alex Crichton (Mar 02 2022 at 15:45): Alex Crichton (Mar 02 2022 at 15:45):@Sam Parker or @Anton Kirilov I've got a question for y'all as our resident arm64-experts. When I build wasmtime's fuzzers myself on AArch64 I occasionally see linker errors that look like:
Alex Crichton (Mar 02 2022 at 15:45):er, sorry, early send
Alex Crichton (Mar 02 2022 at 15:45): compile_maybe_invalid.e9aef4b8-cgu.0:(.text.symbol_nmae+0x6c): relocation truncated to fit: R_AARCH64_CALL26 against symbol `__sanitizer_cov_trace_const_cmp4' defined in .text.__sanitizer_cov_trace_const_cmp4 section in /path/to/libfoo.rlib(FuzzerTracePC.o)
basically: relocation truncated to fit: R_AARCH64_CALL26
I was wondering, do y'all know how to get around this? Is there a linker option to say "please insert some sort of trampoline to get me all the way there" or something like that?
Alex Crichton (Mar 02 2022 at 15:46):note that this is the native linker, so not actually related to wasmtime itself, only the fact that our fuzz outputs are apparently quite large (I'm independently trying to go figure out why they're so large as well)
Sam Parker (Mar 02 2022 at 16:10):What optimisation level is being used and have you tried using an optimization level (-Os) to reduce code size? If it's a static build you can also try passing -mcmodel=large to the compiler.
Alex Crichton (Mar 02 2022 at 16:20):this is an optimized fuzzed build and I haven't tried -Copt-level=s yet (this is all mostly rust code)
Alex Crichton (Mar 02 2022 at 16:21):I tried -Ccode-model=large but it printed out lots of warnings from llvm about the tls model not being supported
Alex Crichton (Mar 02 2022 at 16:21):something about ELF TLS only being supported with a "small" code model or a "local-exec" tls model, but when forcing the tls model to "local-exec" it printed the same warnings
Alex Crichton (Mar 02 2022 at 16:22):specifically:
LLVM ERROR: ELF TLS only supported in small memory model or in local exec TLS model
and that's with:
RUSTFLAGS='-Ccode-model=large -Ztls-model=local-exec' CARGO_TARGET_DIR=large cargo +nightly fuzz build
I built binutils 2.38 and it appears to not work out-of-the-box at least
Alex Crichton (Mar 02 2022 at 16:24):oh also for the "large" code model it may not be supported in Rust for one reason or another (or I don't know what I'm doing), if I rustc foo.rs -C code-model=large for fn main() {} and nothing else it prints:
= note: /home/acrichto/root/bin/ld: foo.foo.938a9993-cgu.0.rcgu.o: relocation R_AARCH64_MOVW_UABS_G0_NC against `a local symbol' can not be used when making a shared object; recompile with -fPIC
/home/acrichto/root/bin/ld: foo.foo.938a9993-cgu.1.rcgu.o: relocation R_AARCH64_MOVW_UABS_G0_NC against `a local symbol' can not be used when making a shared object; recompile with -fPIC
foo.foo.938a9993-cgu.0.rcgu.o:(.data.rel.ro..L__unnamed_1+0x0): dangerous relocation: unsupported relocation
foo.foo.938a9993-cgu.0.rcgu.o:(.data.rel.ro..L__unnamed_1+0x18): dangerous relocation: unsupported relocation
foo.foo.938a9993-cgu.0.rcgu.o:(.data.rel.ro..L__unnamed_1+0x20): dangerous relocation: unsupported relocation
foo.foo.938a9993-cgu.0.rcgu.o:(.data.rel.ro..L__unnamed_1+0x28): dangerous relocation: unsupported relocation
collect2: error: ld returned 1 exit status
I'm trying to figure out what's so darned big but I can't figure out a good way to determine that
Alex Crichton (Mar 02 2022 at 16:26):this hasn't been a persistent problem in the past but now I can't build the fuzzers at all on aarch64 unfortunately
Alex Crichton (Mar 02 2022 at 16:30):ok looks like -Copt-level=s at least works for me for now, and wow the fuzzers are ~300mb apiece...
Alex Crichton (Mar 02 2022 at 17:00):https://github.com/bytecodealliance/wasmtime/pull/3872 should probably also help with the size of cranelift itself
Alex Crichton (Mar 02 2022 at 17:11):alas no still too large to link with default settings...
Alex Crichton (Mar 02 2022 at 18:54):Ok more investigation and thinking has revealed that cargo fuzz by default passes -Clink-dead-code to rustc. The cargo fuzz command also has --strip-dead-code as a parmeter, and when I pass that then all the fuzzers link locally because the binaries are ~100MB smaller.
It sounds, though, like there's no AArch64 linker option or thing like that to get this to "just work". I probably need to pass --strip-dead-code when I build the fuzzers myself.
@fitzgen (he/him) question for you as you ostensibly maintain cargo-fuzz -- back in https://github.com/rust-fuzz/cargo-fuzz/commit/86b4889e3e0f6faac8433659bcac08c4b32097c0 it was claimed that dead code caused fuzzers to print errors, but in https://github.com/rust-fuzz/cargo-fuzz/pull/260 an option was added to strip dead code and running locally that appears to work ok to me at least. Do you know if something about libfuzzer changed in the meantime, and if so if the "strip dead code" option could be turned on by default?
Alex Crichton (Mar 02 2022 at 18:56):one other thing that came up is that https://github.com/bytecodealliance/wasmtime/pull/3836 is the "cause" for why I can't build the fuzzers on aarch64 any more. AFAIK wasm-mutate is a pretty hefty library, especially when fuzzing, but it's only being linked because dead code is linked. Otherwise that should only affect the one fuzz target
fitzgen (he/him) (Mar 02 2022 at 19:50):Yeah I don't know the details. libFuzzer is always a mess of linker errors if you look at it the wrong way, unfortunately :(
Alex Crichton (Mar 02 2022 at 20:59):On a whim I attempted to use some different linkers here. I happened to be testing a cargo fuzz build --dev build which has actually never worked for me before on arm64, but the timings I get are:
I... may just switch to using LLD on this machine as a result
Sam Parker (Mar 03 2022 at 13:15):Wow, that's quite the difference!
Last updated: Oct 23 2024 at 20:03 UTC