Stream: general

Topic: Tarnik - a Rust macro for generating WASM GC code

Piotr Sarnacki (Nov 25 2024 at 23:29):

I couldn't find any good and ergonomic way to generate WASM GC code, and writing it by hand is kinda tedious, so I created a project to help with that: Tarnik. It's an early stage, but I like getting feedback early, so I decided to open source it already. The primary aim is to use it in jawsm and I think I will be able to do it soon. There is an example in the README and a few more in the tests. I plan to add more stuff in the next few weeks as I would really like to start using it in jawsm soon.

Let me know what you think!

Piotr Sarnacki (Dec 02 2024 at 09:25):

I figured out I might use this thread as a way to write about updates to the tool. It's too early to create any official releases I think, but recently I've added a few more things:

• defining and exporting memories
• exporting functions and types
• I implemented most of the binary and unary operations (I think the only left are ! and && + ||)

So this compiles successfully now:

let module = tarnik::wasm! {
  #[export("memory")]
  memory!("memory", 1);

  type String = [mut i8];

  #[export("_start")]
  fn run() {
    let foo: String = "Hello world";
    foo[1] = 'a';

    let sum: i32 = 0;
    for byte in foo {
      sum += byte;
    }
  }
};

println!("{module}");

Next thing to implement is memory access, which shouldn't be too hard, and then ref.cast + ref.test, which should be enough for me to actually start using it in Jawsm :fingers_crossed:

Piotr Sarnacki (Dec 04 2024 at 11:01):

Another small update. I implemented casting (like: i as i64), memory store/load support and imports, so now you can write fully functioning WASI program (using p1 for simplicity):

tarnik::wasm! {
    #[export("memory")]
    memory!("memory", 1);

    #[import("wasi_snapshot_preview1", "fd_write")]
    fn write(a1: i32, a2: i32, a3: i32, a4: i32) -> i32;

    type ImmutableString = [i8];

    #[export("_start")]
    fn run() {
        let str: ImmutableString = "Hello world!";
        let i: i32 = 100;
        for c in str {
            memory[i] = c;
            i += 1;
        }
        // store io vectors
        memory[0] = 100;
        memory[4] = i;

        // `let: foo`` is small hack, if a function returns a value it needs to be somehow consumed
        let foo: i32 = write(
            1, // stdout
            0, // io vectors start
            1, // number of io vectors
            50, // where to write the result
        );
    }
};

Victor Adossi (Dec 04 2024 at 15:36):

Just chiming in to say this is pretty awesome, enjoying reading along :)

Piotr Sarnacki (Dec 08 2024 at 19:21):

Thanks @Victor Adossi!

In the recent few days I've added quite a few things:

1. try/catch/catch_all support, along with passing argument to an exception example
2. ref.test with ref_test! macro example
3. len!() macro for getting length of an array
4. Inserting data strings with data!() macro, for example data!["foo"] will insert a data entry with the string "foo" and it will return an offset example (I still need to implement getting data entry length)

Piotr Sarnacki (Dec 11 2024 at 21:28):

Another update, and this one is exciting (at least for me). Till now I've been working on Tarnik without any real world usage, but the plan was always to use it in my other project: Jawsm compiler. I just finished rewriting about 3k lines of WAT code into about 1.5k of Rust-like code PR. I think this is a good stress test, that also uncovered a few issues, like for example problems when using expressions with casting (like -1 as i32 as i31ref) and a few other edge cases. I restructured the code to fix some of them, but I also fixed other issues so the code I already had just worked as it should. The list of latest changes:

• fix try/catch in context of an if statement and a while loop (it will still break for a for loop)
• implement negation - and not ! operators for numeric types
• implement && and || for numeric types (it still doesn't work for refs, but I plan to implement it at some point as a ref_test for null values)
• generalize struct fields access, so more forms are possible, like for example (anyobject as Number).value = 1 as f64. before only a_struct.field = value was possible
• implement float literal
• implement continue and break
• implement parenthesis, eg. (1 + 2) as i64
• fix accessing globals (it would previously break in some scenarios)
• implement calling func refs
• generalize ref_test!. It was only capable of checking a path like:ref_test!(number, Number), now it will accept any expression, like: ref_test!(numbers[1].value, Number)
• implement ability to set type for store/load with memory::<i8>[0]. The default is to use i32 store and load versions
• fix a few instructions that were wrongly handled (like issuing I32Add
  instead of I32Sub)

For now I will probably slow down development of new features as I will shift my focus a bit towards Jawsm, but I'll try to improve the crate whenever I have time.

Piotr Sarnacki (Dec 30 2024 at 11:04):

I don't really have any big updates as in the last weeks I've been mostly focusing on using Tarnik in JAWSM, but I have a few observations. One thing is that I feel like it speeds up a lot of the implementation, especially when I have to debug something. As code is much more concise than in WAT format, it's much easier to quickly skim the code and look for any obvious mistakes. I still need to read the WAT code from time to time, but I rely much more on the higher level code. In the recent weeks the size of the code using the wasm macro grew to almost 4k lines, which would be roughly an equivalent of 8-10k lines of WAT code and I really doubt I could work that fast in WAT (keep in mind that I only work on the project on my limited free time).

Another observation is that I'm outgrowing what macros are intended for in Rust, for a few reasons. I started with a macro, cause it was the easiest way to start, but there are a couple of drawbacks, with the biggest one being there is currently no way to split the code into smaller pieces. I think it might be fixable, but not entirely easy to do, cause Rust macros can't evaluate any code from outside the macro. rust-analyzer also starts to struggle with a few thousand lines in one macro :sweat_smile: At some point I might just go for implementing a full blown parser (probably based on one of the Rust parsers as the syntax is very similar) that will make it easier to do more advanced stuff. I hold off for now, though, as I'm not far from implementing the entire JS syntax, so I prefer to go with "do the thing that doesn't scale first".

I have also a few more ideas on how to simplify the code even more, but first I would like to write a bit more code with the macro, maybe even take stab at another language like Python or Ruby. Not a full implementation, cause of limited time, but just enough to see if there are any big differences in how I structure the code to implement semantics.

Regarding the features for simplifying the code there are a few smaller ones that I want to do soonish, like:

• allow to check for null using equality, like foo == null instead of ref_test!(foo, null). Still not sure what to do with ref.test in general. In theory I could allow foo == Function, but I don't like it too much as it conflates checking a value and checking the type. Maybe some custom operator like foo is Function?
• fix some small issues like for example: -1 as i31ref doesn't work properly, or you can't access array elements using expressions, so only foo[1] works, not object.foo[1] nor (object as Array)[1] etc.
• fix parenthesis, at the moment it doesn't really work as intended when you want to group some operators with parens

Regarding bigger features, I think the two biggest ones would be enums and some lightweight trait system. As WASM doesn't really have an enum type, it would have to be simulated, but I think it won't be hard to compile it to WASM code that doesn't have any overhead - the type checking would be done mostly on compile time. At the moment I have a lot of code like:

if ref_test!(value, Object) {
  let value_object = value as Object;
  // ...
} else if ref_test!(value, Function) {
  let value_function = value as Function;
  // ....
}

One problem here is that whenever I add a new type to be handled, I have to figure out which places I need to change. Another one is verbosity. I think it would be neat if all those ifs could be changed to a trait implementation and in places where I still have to enumerate some options, to use an enum. It would translate to a static if statement anyway, but the high level code would be much more concise. On a similar note, a match statement would then be nice too, so you can either match an enum or match on types, like:

match value {
  Function => { },
  Object => { },
}

maybe even with something like Rust's @ operator:

match value {
  v @ Function => {
    // v here would be already cast to function maybe?
  },
  v @ Object => { },
}

I don't think I will have time to do all of these anytime soon, but I'll be definitely pondering on these and keeping a list of stuff to do once I can get back to the library.

Last updated: Jan 24 2025 at 00:11 UTC