I've recently started encountering a compilation with some of my components:
= note: rust-lld: error: duplicate symbol: cabi_realloc
>>> defined in /Volumes/workplace/smithy-rs/tools/target/wasm32-wasi/release/deps/libwit_bindgen-e0be62897f1f39ee.rlib(wit_bindgen-e0be62897f1f39ee.wit_bindgen.772b3d7561ccd03a-cgu.0.rcgu.o)
>>> defined in /Volumes/workplace/smithy-rs/tools/target/wasm32-wasi/release/deps/libwit_bindgen-51f815a7b68edcc4.rlib(wit_bindgen-51f815a7b68edcc4.wit_bindgen.d59611476daebe1f-cgu.0.rcgu.o)
Through testing this seems to happen whenever I change my local version of wit-bindgen
to anything other than 0.16.0
. I am also using the wasi::http
bindings from the wasi
crate, and those bindings seem to be generated with wit-bindgen
16 so I think that is likely the cause of the conflicting symbols whenever I change my version.
Is the guidance that I should pin my wit-bindgen
version to whatever the wasi
crate uses, or is there another way of solving the duplicated symbol issue?
Note: this is happening fairly deep in a repo I'm working on at the moment, I will try to pull out a minimal repro soon.
This looks like you might be including two versions of wit-bindgen in the final binary. That's probably something we should fix at the wit-bindgen layer to support that, but in the near-term the best fix is probably to ensure everything is using the same version of wit-bindgen
Gotcha, thankfully wasi
is my only other wit-bindgen
using dependency at the moment, so it is pretty easy to coordinate. I managed to get up a repro and created an issue to track this.
Thanks for the issues here.
The "best" way I can think of solving this is to switch the cabi_realloc
symbol to a weak symbol. That way it's ok to have multiple definitions and they'll all be largely the same so it doesn't matter which one the linker ends up picking. The bad part about this is that there's no way to define a weak symbol in stable Rust.
@Dan Gohman you wouldn't happen to have any tricks to defining a weak symbol in stable Rust would you? (or anyone else?)
This is definitely going to be something I want to solve because I don't want to have to update the wasi
crate every time wit-bindgen
updates
No, last I looked into it the weak symbol attributes were nightly-only.
one idea I had was to use global_asm!
perhaps with various directives there
but I'm not sure how stable that would be since I don't think LLVM provides strict stability guarantees about that layer of assembly
especially for a somewhat niche target like wasm
hm maybe I'll try going that route for now and see how it pans out, looks like the directive is .weak
We could make weak symbols using .weak
inside a global_asm!
. (edit: ah, you beat me too it)
bah global_asm!
isn't available on wasm targets
I suppose we could do the same object-file-hackery that the adapter used to do
(still does?)
oof that's a lot to hardcode though
How hard would it be to give cabi_realloc
a mangled name?
we could relatively easily do something like cabi_realloc_wit_bindgen_0_18_0
or something like that
another attempt would be to have cabi_realloc.c
in the repo and check in a precompiled libcabi_realloc.a
which is distributed to crates.io
but that would go straight to malloc
which bypasses Rust's allocator stuff which at least causes wit-bindgen tests to fail
since they rely on that to count allocations
we could also move all this logic into wasm-tools component new
and pick any function that starts with cabi_realloc
to be the allocator, but that doesn't feel great
If we could mangle the name though are you thinking we'd have C refer to that?
I think so? Like ,cabi_realloc_wit_bindgen_0_18_0
is manageable levels of ugly.
we'd have to recomiple the *.c
file per-version
which isn't the end of the world
I suppose with the shiny new release process we could even automate this
Is there a context where we have a C compiler that could compile cabi_realloc
? Because we can make weak symbols from C.
in CI for wit-bindgen
itself for sure, but not when users get the wit-bindgen
from crates.io
so I'm thinking we could add a *.c
file and verify in CI that there's a checked in *.a
file which "matches"
and then PRs to do a release automatically recompile and make a commit with the new version
so that way we use C to define a weak cabi_realloc
and that refers to a version-specific symbol
convoluted, but could work
Yeah
How evil would it be to smuggle a .weak
inside a non-global asm!
inside the body of cabi_realloc
?
even asm!
isn't reachable :(
Ah, I didn't realize we didn't have asm
oh I was using the wrong path, asm!
and global_asm!
are reachable but both flagged as unstable
I was doing core::arch::wasm32::asm!
instead of core::arch::asm!
Ok https://github.com/bytecodealliance/wit-bindgen/pull/851 is my attempt at all this
@Dan Gohman question for you, which I have no idea if you'll know the answer to this:
For wasi-sdk using the precompiled binaries from CI I'm seeing !1 = !{!"clang version 16.0.0"}
in LLVM IR. For wasi-sdk's precompiled binaries on macOS I'm seeing !1 = !{!"clang version 16.0.0 (https://github.com/llvm/llvm-project 08d094a0e457360ad8b94b017d2dc277e697ca76)"}
in the IR. This means that a macOS host and a Linux host produce a different "producers" section in the output. You wouldn't happen to know off the top of your head why that is would you?
Wow, no, I don't know. My guess is some build system logic to distinguish between "official builds" and custom builds, but I don't know what that logic is looking at.
@Andrew Brown might perhaps know?
I need to track that down, figure out how to get clang to not emit it, or strip it myself
Last updated: Dec 23 2024 at 13:07 UTC