Stream: wasmtime

Topic: wasm as compilation target

Amit Upadhyay (May 19 2023 at 13:59):

Hey everyone, we are trying to write a compiler in Rust and are trying to generate WASM out of it. I tried using binaryen_sys crate, but it is unsafe, and also not very friendly API for writing Rust code again. binaryen, the Rust wrapper's docs:

Binaryen bindings for Rust. They _used to provide bindings for IR-construction part_ of the API, but now this crate is more focused on tools provided by Binaryen ...

What are my options?

TIA,

Michael Ball (May 19 2023 at 14:47):

I too would be interested in hearing what are the happiest paths for building a compiler that targets wasm only, be it rust or some other language.

bjorn3 (May 19 2023 at 14:52):

https://github.com/cfallin/waffle and https://github.com/rustwasm/walrus are other options. Walrus closely mirrors wasm, while Waffle allows you to write code in SSA form with unstructured control flow and then converts it to stack manipulation and structured control flow when emitting the wasm module. I don't have any experience with either crates though.

Amit Upadhyay (May 19 2023 at 16:28):

Cool, waffle seems to be using https://github.com/bytecodealliance/wasm-tools/tree/main/crates/wasm-encoder, so that's another option.

Chris Fallin (May 19 2023 at 16:30):

@Amit Upadhyay indeed, I had exactly the same frustrations (no good Wasm backend in Rust) and so I wrote waffle; the use-case I had was a roundtrip transform (Wasm to IR back to Wasm) but it's totally usable for just IR->Wasm for codegen as well

Chris Fallin (May 19 2023 at 16:31):

wasm_encoder is much lower level, you'd have to reason about control flow and the value stack and such at the Wasm level; waffle provides a standard SSA IR

Chris Fallin (May 19 2023 at 16:31):

waffle is very much a "wrote it quickly to serve another project" kind of project, so the API isn't too polished yet, but I'm happy to have a collaborator on that front (e.g. creating a nice builder API on top of the raw IR) if you want :-)

Amit Upadhyay (May 19 2023 at 16:36):

Thanks @Chris Fallin. So far I have been prototyping what code in my language (fastn.com) would look like what WAT code, and I started naively taking a stab at creating my own wasm Ast and "printer" (https://github.com/fastn-stack/fastn/pull/927/files#diff-e12d760b0365562008fca782bd3bb0a4640321d58b3000ecc47e5d16ffaf6a6c).

Let me read some code. Do you have any quick sample code like https://github.com/rustwasm/walrus/blob/master/examples/build-wasm-from-scratch.rs for waffle?

Also is there a better resource than Wikipedia article if I want to understand what is SSA IR? I am very new to all this.

Chris Fallin (May 19 2023 at 16:37):

@Amit Upadhyay unfortunately no, I don't have any sample code, I haven't had the time to build out documentation of that sort

Chris Fallin (May 19 2023 at 16:38):

for IRs in general, this is a good reference: https://pfalcon.github.io/ssabook/latest/book.pdf

Chris Fallin (May 19 2023 at 16:38):

(thanks to @Jamey Sharp for linking that for me the other day, and he may have other thoughts too)

Chris Fallin (May 19 2023 at 16:39):

if you're already producing output at a Wasm level (as WATs or as an AST or otherwise) then something like Waffle isn't really necessary; but if you start to want to e.g. optimize the output in any way, or do transforms on it (CPS or asyncify in the future, or ...) then having a "real IR" is quite useful

Jamey Sharp (May 19 2023 at 19:38):

@Amit Upadhyay, do you have any experience with functional programming (e.g. Haskell, Ocaml, Clojure, etc)? it's not at all necessary, but if you do, that can be a helpful way to think of SSA. you're never allowed to re-assign a new value to the same variable name, so if you need to run some block of code multiple times with different values, you have to do something like a function call (or more precisely a tail call) to that block with each set of values as arguments. beyond that constraint, the other important thing to understand if you aren't already familiar with it is the "control-flow graph" (CFG) style of IR, because SSA form is a specialization of a CFG.

Amit Upadhyay (May 21 2023 at 19:02):

@Jamey Sharp I do, used Elm a lot. Used Scheme during by engineering days as well. I have not studied compilers academically, nor computer science for that matter. Am Mechanical Engineer by education. So this is all quite new to me.

We are working on fastn.com. Specifically we are trying to add native support. Currently fastn compiles to HTML/CSS/JS, we are thinking of instead compiling to WASM, and so we can render fastn powered UI both in browsers, and in terminal / native (without web rendering, using wgpu and custom rendering code).

We used to compile functions and components written in our language to JS, and now we are trying to experiment with compiling to WASM instead. The function part is relatively simpler, the function stuff we support is as yet quite rudimentary, no closures, no concurrent access, no async etc, so transpiling functions and component definitions, we have handle on.

The part where we are struggling with is memory management. Transliping to JS meant we do not have to worry about it, all our variables are in JS space, and can be garbage collected. But with wasm, we have no garbage collector. Which means we have to memory management, but we do not want to export Rust like explicit management. We want garbage collector benefits.

But building a full garbage collector is a bit too much for us to do right now. So we are experimenting with a reference counting based approach. Even reference counting means some of the aspects of memory management we have to expose to end users, which we do not want to do. So have some up with a sort of reference tracking like approach, which seems to allow circular references.

Currently we are in drawing board stage, creating excalidraw drawings etc to see if it works.

bjorn3 (May 21 2023 at 22:10):

There is a WebAssembly proposal for native garbage collection support which will probably be implemented in Wasmtime in the not too far future: https://github.com/WebAssembly/gc, https://github.com/bytecodealliance/rfcs/pull/31

Last updated: Nov 22 2024 at 16:03 UTC