Jonathan Halliday (Aug 10 2023 at 14:11): Jonathan Halliday (Aug 10 2023 at 14:11):@Joel Dice do you know if there is any standalone example or test of calling wasm exported functions from the wasmtime c-api? I can do it easily enough with functions that don't interact with memory, but I'm missing the magic that e.g. serializes string content into linear memory so the called function can read it. I assume that's spin's job in your world, but there must also be a generic way to accomplish it?
Lann Martin (Aug 10 2023 at 14:18):That magic was wit-bindgen <=0.2 prior to implementation of the component model, and components now.
Jonathan Halliday (Aug 10 2023 at 14:19):Hmm. So what part of the wasmtime c-api provides it, or has the api not caught up with components yet?
Joel Dice (Aug 10 2023 at 14:19):The Wasmtime C API does not yet expose Wasmtime's component support (i.e. it's only available in the Rust API), so there's no easy way to do it. Are you looking to instantiate and run a component, or just a module that uses the component canonical ABI (e.g. the output of TeaVM prior to running wasm-tools component new)? If the former, you'll probably need to use Rust. If the latter, you could probably hack something together that does the lifting and lowering according to https://github.com/WebAssembly/component-model/blob/main/design/mvp/CanonicalABI.md.
Joel Dice (Aug 10 2023 at 14:20):Component support in the Wasmtime C API has been a common request, but it's a pretty big lift, and nobody has volunteered to do it, yet.
Jonathan Halliday (Aug 10 2023 at 14:26):In the short term, I have a .wit "myfunc: func(in: string) -> string" and the corresponding .java 'deserialization' boilerplate that bindgen gives, plus the .wasm teavm transpiles from that. I have a c program that loads that .wasm, instantiates it and gives me the 'myfunc' function handle. That function doesn't take a string, it takes two ints that key into the linear memory. So now I need the boilerplate for the 'serializer' that will poke a string into linear memory so the function can find it. So there isn't any library assistance for that, I have to hand tool it at present, and likewise all the more complex versions of the same call pattern?
Joel Dice (Aug 10 2023 at 14:30):I'm afraid so, if you need to stick with C. If you can switch to Rust, it will all be taken care of for you.
Jonathan Halliday (Aug 10 2023 at 14:32):Eventually I need C, but I can live with Rust for now I think. Where can I find a Rust example of doing that to use as a starting point?
Joel Dice (Aug 10 2023 at 14:34):The runtime tests in the wit-bindgen repo (e.g. https://github.com/bytecodealliance/wit-bindgen/blob/main/tests/runtime/records.rs) are good examples.
Joel Dice (Aug 10 2023 at 14:35):They all amount to pointing wasmtime-wit-bindgen bindgen! macro at a WIT file and then using the API it produces.
Joel Dice (Aug 10 2023 at 14:36):Docs here: https://docs.rs/wasmtime/11.0.1/wasmtime/component/macro.bindgen.html
Joel Dice (Aug 10 2023 at 14:38):There's also a dynamic API which does not rely on build-time codegen (example here: https://github.com/bytecodealliance/wasmtime/blob/main/tests/all/component_model/dynamic.rs). That would be easier to wrap a C API around, I imagine.
Jonathan Halliday (Aug 10 2023 at 14:39):Hmm, so they use the rust bindgen in --import mode get a client ('serializer') and then call that? interesting. It should theoretically then be possible to generate, using that bindgen or another, a serializer function in wasm, then write a zero-args export function that calls it and which can be invoked from C. You'd basically put the serialization/call setup code itself in wasm instead of in C, since there is auto-generation for that but not for C.
Joel Dice (Aug 10 2023 at 14:43):I don't _think_ --import is a thing anymore in wit-bindgen now that we have WIT worlds, so I'm not quite following you on the first part. But yeah, you could put some extern "C" functions in your Rust code to make them available to C.
Jonathan Halliday (Aug 10 2023 at 14:45):Hmm. So given a .wit file, I want to codegen two things: an implementation, and an api for calling that implementation. You're saying I can only generate the implementation?
Lann Martin (Aug 10 2023 at 14:47):You may be interested in this thread as well :smile:
Joel Dice (Aug 10 2023 at 14:47):You can generate the guest APIs (i.e. the Wasm part) using wit-bindgen for Rust, C, Go, or Java (or Python or JS if you use the relevant tools). For the time being, you can only generate the host APIs using Rust via wasmtime-wit-bindgen.
Joel Dice (Aug 10 2023 at 14:49):Oh, and you can generate host APIs for Python, too, via wasmtime-py, assuming your component has no subcomponents.
Joel Dice (Aug 10 2023 at 14:49):I _think_ those are the only host API generators for the component model at the moment.
Joel Dice (Aug 10 2023 at 14:50):Oh, and jco for JS
Joel Dice (Aug 10 2023 at 14:51):And @Christof Petig is working on C++ support for both guest and host.
Jonathan Halliday (Aug 10 2023 at 14:57):Unless I misunderstand the terms, it's not a host/guest thing, it's a client/server or caller/callee thing. I can generate the receiving side for my function, now I want to generate the calling side for a guest language. Then the host's only job is to bootstrap that guest caller code. Apparently I can't put the calling logic directly in the host, my host is C and there isn't a codegen for that. But I should be able to put it in any guest language I want and then call that from C. My impression was the way to get bindgen to give me that calling code rather than the callee code was --import.
Jonathan Halliday (Aug 10 2023 at 15:10):Hang on, what stops me generating the guest caller API in C and then just copying all that bindgen boilerplate into my own C host code? The only thing it should need is a shim giving it a pointer to the linear memory.
Joel Dice (Aug 10 2023 at 15:29):That might work. I think you may need to edit the generated code to hook it up to Wasmtime, which will be a pain if you need to change the WIT and regenerate the code. I'd recommend starting with a simple example and see how it goes.
Christof Petig (Aug 10 2023 at 16:43):Jonathan Halliday said:
Hang on, what stops me generating the guest caller API in C and then just copying all that bindgen boilerplate into my own C host code? The only thing it should need is a shim giving it a pointer to the linear memory.
The canonical ABI is asymmetric, you can't use the same code on guest and host. This is due to exported linear memory from guest to host and shared nothing from host to guest. But due to the abi crate the effort to create host bindings is comparable to the effort to create guest bindings, with a good amount of code reuse.
Jonathan Halliday (Aug 10 2023 at 17:23):huh? Guest code doesn't know what it's talking to - could be the host, could be another guest. It's just a serialization protocol to a chunk of memory.
Lann Martin (Aug 10 2023 at 18:13):That's true. Host bindings differ from guest bindings because of how they need to interact with the wasmtime runtime.
Christof Petig (Aug 10 2023 at 21:14):Jonathan Halliday said:
huh? Guest code doesn't know what it's talking to - could be the host, could be another guest. It's just a serialization protocol to a chunk of memory.
A component/guest can't talk to another component/guest without the host (or another external instance) copying/converting the data, due to their shared nothing design.
Lann Martin (Aug 10 2023 at 21:32):Oh right, guest codegen is always from the perspective of the owner of the linear memory. Host codegen additionally needs to e.g. cabi_realloc to make space for new values.
Jonathan Halliday (Aug 10 2023 at 21:58):The guest is the owner of the linear memory only after the host creates it and hands it off to the guest. Between those two steps the host can presumably poke whatever it likes into it.
Lann Martin (Aug 10 2023 at 22:32):Well sure, but most guest environments will also have some kind of runtime (or at least static initialization) that expects control over at least parts of memory. As a general-purpose mechanism the canonical abi always asks the "inner" module for an allocation it can safely write to.
Last updated: Nov 22 2024 at 16:03 UTC