Stream: general

Topic: ✔ to wasm and back again

Safa Sofuoglu (Aug 03 2021 at 13:54):

Hello! I'm experimenting with incorporating wasm into an established C-ffi system (specifically dart-lang). The use case is that C or binary files can not be distributed through the package system, but wasm potentially could. Then it would be up to the toolchain to recreate the intended native dynamic library locally.

What I'm experimenting with is a simple exported C function:
EXPORT int64_t add_five(int64_t input) { return input + 5; }

Compile it to wasm dynamic:
zig build-lib -dynamic shared.c -target wasm32-freestanding

Make it a .so:
wasmtime wasm2obj ../dart-wasm/zig/shared.wasm shared.o zig build-lib -dynamic shared.o

And then expect to use shared.so with the initially intended header and C-ffi.

Of course this fails on multiple levels. Hence the question:

• Is the approach fundamentally wrong?
• Can it be made somehow so that the C ABI and the API can be preserved through C->wasm->native compilation? If yes, what kind of development/modification is needed?

Till Schneidereit (Aug 03 2021 at 13:59):

Hi Safa! :wave: That use case sounds at least extremely closely related to the goals of the RLBox project. @Deian Stefan and @Shravan Narayan could say more about whether it might be good fit, though

Safa Sofuoglu (Aug 03 2021 at 14:09):

@Till Schneidereit thanks for the swift response. The use case does not involve sandboxing or security at the moment. My current objective is to figure out how to recreate (or wrap&expose) the C ABI/API surface after C->wasm->native compilation.

Till Schneidereit (Aug 03 2021 at 14:10):

ah, I see. This could potentially still give you a toolchain that does what you want-with improved security thrown in as a bonus :stuck_out_tongue:

Shravan Narayan (Aug 03 2021 at 19:28):

@Safa Sofuoglu Yup, RLBox definitely does cover this as a part of what it does. While our focus is security and sandboxing, we had to cover ABI compatibility issue as part of what we did.

We had address multiple challenges in preserving details across the C->Wasm->native boundary, specifically

• ABI --- C's types are not consistent in each platform. long means different things on different platforms/compilers
• Pointer swizzling --- pointers in Wasm are 32-bit offsets and not 64-bit globals (or whatever the platform pointer size is)
• Calling convention --- function names and their possible mangling by Wasm toolchains and some calling convention aspects such as how do functions that return structs operate may not be identical in every platform
• Data marshaling --- any shared data or buffers in use by the host application needs to be copied over to the Wasm heap, as the Wasm code would otherwise be unable to access it

For RLBox, we basically end up solving this through a combination of a C++ wrapper type system and template meta-programming to get things working transparently.

I think I broadly see 3 ways to address this

• Write a code generator that takes the C header and generates the ABI fixing code as wrappers around the generated Wasm code
• Modify RLBox to use just the ABI compat layer without the sandboxing bits. It should be doable, and I'd be happy to upstream/help with the changes.
• Implement an ABI compat layer in a C++ template meta-programming layer (like RLBox) but without the sandboxing bits

Hope this helps

Safa Sofuoglu (Aug 03 2021 at 19:56):

@Shravan Narayan your answer is immensely helpful, thanks! I'll start by inspecting RLBox source.

Notification Bot (Aug 03 2021 at 19:56):

Safa Sofuoglu has marked this topic as resolved.

Shravan Narayan (Aug 17 2021 at 19:35):

@Safa Sofuoglu Oh also re- a generator, if you want to use an existing generator you can also use witx (you have to manually write the witx interface file for libraries you care about). Then you can use witx bindgen tool
https://bytecodealliance.github.io/witx-bindgen/

Last updated: Nov 22 2024 at 16:03 UTC