Stream: cranelift

Topic: ✔ Cranelift to Wasm

Frank Rehwinkel (Jun 22 2024 at 13:04):

Can the Cranelift compiler be compiled to WASM? I was wondering about getting something similar to Wasmtime written in Golang (or C) so it could be run on BSD platforms like FreeBSD, OpenBSD, and Dragonfly BSD, but the actual generation of machine code at runtime is not something to reinvent.

Alex Crichton (Jun 22 2024 at 13:49):

Yes, you can see this in Wasmtime with cargo build --target wasm32-wasip1 --features all-arch,compile --no-default-features which builds a wasm binary that can do ./wasmtime.wasm compile ... which is more-or-less cranelift-compiled to wasm

Notification Bot (Jun 22 2024 at 14:14):

Frank Rehwinkel has marked this topic as resolved.

Frank Rehwinkel (Jun 22 2024 at 14:27):

I misspoke when I said/thought the BSD program would be wasmtime-like. Now I appreciate more of what the wasmtime-cli brings to the table. So just a new embedder that could use the wasip1 build of cranelift is what I meant. And the debug and release versions of the command provided above built it cleanly. Thanks.

Frank Rehwinkel (Jun 22 2024 at 16:37):

Just a follow up. It works. Building the release version of the wasmtime as given above creates a 7MB wasm file (built it on a mac). On a Linux machine where I had built iwasm from wamr, I ran the cranelift compiler with iwasm and passed some arguments to the compiler, the filename and the target:

$ iwasm --dir=. /tmp/wasmtime.wasm compile ./hello.wasm --target x86_64-unknown-linux-gnu

which created a hello.cwasm file after 2 seconds.

Then to run the compiled binary, I resorted to wasmtime again because I didn't know anything else that runs compiled wasm yet:

$ wasmtime --allow-precompiled -W component-model=n -W threads=n hello.cwasm
Hello, world!

I think that's cool. Using a wasm version of cranelift to compile other wasm files into a target's compiled wasm. So far could only see it build and run with the x86_64-unknown-linux-gnu target and I could see the builds succeed for x86_64-unknown-{dragonfly, freebsd, openbsd} but don't have machines to try them on (and don't have an embedder on such machines yet either).

Chris Fallin (Jun 22 2024 at 18:05):

Separate from your actual question but to address a few other points you brought up with possibly-useful info:

so it could be run on BSD platforms like FreeBSD, OpenBSD, and Dragonfly BSD

Wasmtime should either build and work, or be very close to doing so, on FreeBSD (and patches welcome if not); OpenBSD is a little trickier because of crate support and need for rustc binaries (https://github.com/bytecodealliance/wasmtime/issues/6962 and you can see my PR that did have it working at one point at https://github.com/bytecodealliance/wasmtime/pull/2980); haven't tried on NetBSD or Dragonfly. We list more details in our docs under tiers (https://docs.wasmtime.dev/stability-tiers.html) and platforms (https://docs.wasmtime.dev/stability-platform-support.html)

I resorted to wasmtime again because I didn't know anything else that runs compiled wasm yet

.cwasm is Wasmtime's own format and we don't have a formal spec or even a stability guarantee between versions: the compiled code directly accesses data structures maintained by the Wasmtime runtime, so is very specific and unlikely to be easy to run in some other runtime. All that to say that "anything else that runs [.cwasm]" is unlikely to / shouldn't exist in the future.

Glad this worked out for you -- would be interesting to hear more about the use-case (just running on the above platforms or...? e.g. some folks run Cranelift compilation in Wasm because they want to sandbox against runaway compiler resource usage, etc)

Frank Rehwinkel (Jun 22 2024 at 22:26):

Oh, that's very interesting info. I was under the mistaken impression the compiled wasm could be runnable by something other than wasmtime. But that explains why the wasmtime version had to match what was found in the .cwasm file. And it makes perfect sense - there was no ABI for the first Wasm modules that embedders would all be trying to support.

My use case generally was to see what was possible to extend the use of wasm in environments where Wasmtime didn't play nicely, like OpenBSD in particular. I had seen posts earlier where because OpenBSD won't keep its ABI standard from release to release, it was considered too much of a moving target and there was no company behind wanting to keep up the necessary support. I see OpenBSD and Dragonfly BSD are listed in the Tier 3 rustc handbook, with check marks for the std library and the host tools; so I guess cargo can be built for them, not sure if rustup could be built for them.

OpenBSD has been of interest to me for a long time because of its emphasis on security first, before features and speed. And then I was looking at the Upspin project again, after having worked on it years ago, and thought it could be nice to replace some upspin binaries that have to be built for each platform (and they like to support Dragonfly and OpenBSD) with an embedder for each of those platforms but then the meat of the functionality (encryption of files and encryption of rpc and TCP connections with a name server and then the federated client server) would be served with Wasm modules.

Generally I thought, if someone is running an OpenBSD laptop or desktop, they probably have their privacy and security in mind first, and being able to offer a way to use Wasm modules, maybe Wasm component modules, what with the strictest sandboxing that implies, just falls right into their wheelhouse.

Upspin hasn't caught on with a lot of people yet. They admit their users so far have to be very software savvy. And their key pair story had made it hard to see how to put a client into a browser so (I think) their shared file system is pretty much only used on desktops (but that is based on info from years ago so maybe has changed).

Because GitHub has created an easy PassKey version for their 2FA, I started wondering about a mashup of Upspin, PassKeys and Wasm Components, but support on OpenBSD seemed like the first hurdle that would be easiest to understand. I guess there are C and C++, maybe even Golang, wasm embedders I could work from for OpenBSD and skip the idea of using Cranelift entirely, but they don't support Component modules so probably I won't let OpenBSD be a show stopper for now.

Thanks for the feedback above, the trail of wasmtime/openbsd attempts from 2001 and earlier is thought provoking.

Last updated: Oct 23 2024 at 20:03 UTC