Mohannad Al-Awad (Feb 12 2024 at 19:01): Mohannad Al-Awad (Feb 12 2024 at 19:01):Hi everyone :wave: . Usually, compilers are rewritten in their new programming languages after the first version. Will I be able to embed Cranelift in such new language's rewritten compiler? In general, is it possible to embed Cranelift in a language other than Rust?
Chris Fallin (Feb 12 2024 at 20:06):Right now there is no C-compatible API but I suspect that'd be the easiest route: wrap the InstructionBuilder and related interfaces in an FFI shim and then call into that. (This question is I guess a bit more general: will your language have an FFI to Rust? If so then yes it can call Cranelift; if it instead has an FFI to language X, one will need a language-X wrapper around Cranelift, and C is a common choice for that)
Mohannad Al-Awad (Feb 13 2024 at 12:50):THx @Chris Fallin for passing by. Actually I did not know that I can FFI to Rust directly because I thought that Rust ABI is not stable. Is it documented somewhere besides its source code?
Chris Fallin (Feb 13 2024 at 15:50):No, it’s not; I can imagine other ways to do this if you’re developing your own language though. For example, you could generate shim code in Rust from “extern FFI” declarations in your language and build and link it against Cranelift.
Mohannad Al-Awad (Feb 13 2024 at 16:29):Are you saying that I can define extern "mylang" in Rust to expose Crainlift APIs to my language? Isn't this extending the Rust compiler?
Chris Fallin (Feb 13 2024 at 16:41):No, the other way around; "extern FFI declarations in your language" means something written in your language to describe the Rust interface
Mohannad Al-Awad (Feb 13 2024 at 16:51):Interesting. I still need to rely on the unstable Rust ABI though, right?
Chris Fallin (Feb 13 2024 at 16:52):No, in the proposed idea, you would generate Rust code that calls Cranelift directly, and exports a C-ABI interface that you can then call directly from your language's generated code
Chris Fallin (Feb 13 2024 at 16:52):I guess it's basically an automated approach to the "add a C API to Cranelift" option
Chris Fallin (Feb 13 2024 at 16:52):with the downside that it won't benefit other Cranelift users, but the upside that it gives you a general way to bind to Rust interfaces from your language -- both seem valid to me
Mohannad Al-Awad (Feb 13 2024 at 16:54):Got you! :+1:
Mohannad Al-Awad (Feb 13 2024 at 17:02):On a different approach. If I go with using Wasmtime after compiling my language to WASM.
Compared to using Cranelift IR, would I lose Cranelift optimizations that benefit from meta data passed down?
I guess another problem is that I cannot develop a standard runtime library that interfaces with the OS in my language if it targets WASM as WASI is still at early stage.
Chris Fallin (Feb 13 2024 at 17:04):targeting Wasm is actually a great option, IMHO
Chris Fallin (Feb 13 2024 at 17:05):if you do that, it would be worthwhile to look into the component model and wit-bindgen as well to give you a first-class way of doing FFI
Mohannad Al-Awad (Feb 13 2024 at 18:36):My understanding of WIT and CM is that they are for interoperability between WASM modules but not outside WASM (e.g. system ABIs).
How can I write a standard runtime library in my new language that actually interfaces with the host OS and compiles to WASM at the same time?
Chris Fallin (Feb 13 2024 at 18:38):that's their core purpose, yes, but in the wasm-component universe, additional WASI APIs are being defined: for example, there is now networking (http and I think raw sockets), and there is work on things like async IO, key-value stores, graphics APIs, and a bunch of other stuff
Mohannad Al-Awad (Feb 13 2024 at 18:41):I will need to extend WASI APIs in some areas and I want to write these extensions in the same new language.
Chris Fallin (Feb 13 2024 at 18:45):OK, perhaps Wasm with WASI isn't the best option then, if you think you'll need to extend APIs to make use of OS-specific behavior. In any case, lots of options here -- best of luck!
Mohannad Al-Awad (Feb 13 2024 at 18:53):Thank you for your time.
One more question if you allow me plz.
I guess what I want is to be able to link to external libraries. If I compile my language to WASM then AOT to binaries, will I be able to link against them (using my language compiler somehow) as host provided functions to my compiled WASM?
Chris Fallin (Feb 13 2024 at 18:56):that depends on the Wasm VM; it's not a builtin feature of Wasm. In Wasmtime, when you AOT-compile, you get an opaque artifact that still requires Wasmtime to load and run; the program that embeds Wasmtime (as a library) can provide arbitrary imports for the Wasm module implemented in native code. In other Wasm VMs, there may be other functionality for this (but it wouldn't be "standard").
Mohannad Al-Awad (Feb 13 2024 at 19:02):Oh! I thought AOT meant a final self-contained executable in the target ISA with no dependency on Wasmtime. I am wondering what I the role of Wasmtime in this case.
Chris Fallin (Feb 13 2024 at 19:04):The role of Wasmtime is still to provide the runtime support for the Wasm: for example, allocating the Wasm heaps, managing the Wasm tables, and eventually providing other Wasm features (GC, ...) as required. The AOT-compilation means that the Wasm bytecode is compiled to machine code ahead of time, but that machine code is still dependent on Wasmtime's runtime.
Mohannad Al-Awad (Feb 13 2024 at 19:08):Aha! I guess the AOT compiled code is still packaged as wasm then and needs a runtime for loading and linking which is the program embedding the Wasmtime. It would be nice if all can be packaged as standalone executable.
Mohannad Al-Awad (Feb 13 2024 at 19:17):Thank you once again for your generosity sir.
Chris Fallin (Feb 13 2024 at 19:19):Not quite packaged as Wasm; a .cwasm file (the output of wasmtime compile) is considered to be an opaque blob and Wasmtime doesn't make any promises about format. I think we have an issue on file to do a truly standalone compilation; it would involve basically linking the runtime part of Wasmtime against the AOT code, and a wrapper to start up the module
Chris Fallin (Feb 13 2024 at 19:19):if someone wants to build that out, it'd be neat! just not a priority at the moment
Mohannad Al-Awad (Feb 13 2024 at 19:21):Actually it might be also nice if AOT compiles on the first run then cache the result
Chris Fallin (Feb 13 2024 at 19:23):oh, wasmtime run does that already!
Chris Fallin (Feb 13 2024 at 19:24):(wasmtime run --disable-cache to see the speed difference)
Chris Fallin (Feb 13 2024 at 19:24):(or I think there's a new cmdline syntax for that now, I forget what it is)
Mohannad Al-Awad (Feb 13 2024 at 19:24):Nice! :+1:
Mohannad Al-Awad (Feb 13 2024 at 19:25):I will need to gather my thoughts after this fruitful discussion to decide on an approach to take with leveraging Wasmtime/Cranelift...
Last updated: Oct 23 2024 at 20:03 UTC