Stream: general

Topic: ✔ asking help for first time contribution


view this post on Zulip Jun Ryung Ju (Aug 25 2022 at 11:57):

I am looking for compiler repository that I can contribute. but it looks 'Easy' labeled issues on github are doesn't seem to be maintained. can I get some easy task for cranelift stuffs? I have experience with modifying CoreCLR(which is dotnet runtime) optimization passes, LLVM optimization, lowering passes.

view this post on Zulip Chris Fallin (Aug 25 2022 at 16:37):

@Jun Ryung Ju welcome and thank you very much for your interest and stopping by here!

We haven't done a great job of maintaining a list of easy issues, but some of the big ideas in many of them are still relevant (and actually always relevant), mainly:

On the third item above (testing infrastructure), @Afonso Bordado is currently working to build out and improve a "CLIF interpreter", which is an interpreter of our intermediate language (CLIF) that we can use as a reference to test the real compiler backends against. AFAIK it does not yet have implementations for all opcodes. @Afonso Bordado are there easy-to-separate pieces of work here that @Jun Ryung Ju might be interested in?

view this post on Zulip Afonso Bordado (Aug 25 2022 at 16:49):

:wave: Hey, Welcome to the project!

Yes, there are a bunch of opcodes not yet implemented in the interpreter. The two groups that come to mind are:

The reason these are important is that they are already implemented in the various backends, so when they are ready on the interpreter we can start fuzzing on them (which is the main driver for the work on the interpreter). Mostly its adding tests to our test suite and then working on a implementation on the interpreter.

Here's an example PR adding a few instructions

👋 Hey, Fuzzgen found that we were missing these as soon as I added float support

view this post on Zulip Afonso Bordado (Aug 25 2022 at 17:05):

If you'd like to work on something related to compiler passes, you might want to read this thread that has a bunch of information regarding the current state of the compiler:

https://github.com/bytecodealliance/wasmtime/issues/4712

Although I don't know how easy/hard any of those items are

What is Cranelift's job (in the context of Wasmtime)? To take Wasm that is produced by LLVM and already optimized 99% of the time and do the architecture-specific code generation that LLVM cann...

view this post on Zulip Jun Ryung Ju (Aug 25 2022 at 17:22):

Afonso Bordado 말함:

:wave: Hey, Welcome to the project!

Yes, there are a bunch of opcodes not yet implemented in the interpreter. The two groups that come to mind are:

The reason these are important is that they are already implemented in the various backends, so when they are ready on the interpreter we can start fuzzing on them (which is the main driver for the work on the interpreter). Mostly its adding tests to our test suite and then working on a implementation on the interpreter.

Here's an example PR adding a few instructions

Thanks for helping me out @Afonso Bordado

I think task that you suggested are good task to start contribute on cranelift. I'll take that :)

view this post on Zulip Afonso Bordado (Aug 25 2022 at 17:23):

If you need any help let me know

view this post on Zulip Jun Ryung Ju (Aug 29 2022 at 19:28):

Are there any resources I can refer to make a strict test? I have no idea what needs to be tested on "Bit ops on floats f32/f64" feature.

view this post on Zulip Jun Ryung Ju (Aug 29 2022 at 19:28):

I mean for the strict test. not simple test.

view this post on Zulip Afonso Bordado (Aug 29 2022 at 19:51):

I'm not sure what you mean by strict test?

view this post on Zulip Afonso Bordado (Aug 29 2022 at 19:52):

The most common tests we have are compile tests and run tests

view this post on Zulip Afonso Bordado (Aug 29 2022 at 19:52):

The interpreter can only run the latter

view this post on Zulip Afonso Bordado (Aug 29 2022 at 19:53):

so, "Bit ops on floats" is kind of a broad term, it means a bunch of bit wise instructions that can also be used on floats, such as:

etc...

view this post on Zulip Afonso Bordado (Aug 29 2022 at 19:55):

To build a run test you can do the following:

test interpret
test run
target x86_64
target aarch64
target s390x

function %band_f32(f32, f32) -> f32 {
block0(v0: f32, v1: f32):
    v2 = band v0, v1
    return v2
}
; run: %band_f32(0x0.5, 0x1.0) == 0x1.5

put that in a file in filetests/filetests/runtests/band-floats.clif and the you can run it by invoking cargo run -- test filetests/filetests/runtests/band-floats.clif from the cranelift directory

view this post on Zulip Afonso Bordado (Aug 29 2022 at 19:55):

(i haven't tested the output of that, Its probably wrong!)

view this post on Zulip Afonso Bordado (Aug 29 2022 at 19:57):

the first lines tell clif-util what kind of tests we want to run on this file. test interpret invokes the interpreter and checks if the conditions in the ; run comments pass. test run does the same, but compiles the file and runs it as a native binary

view this post on Zulip Afonso Bordado (Aug 29 2022 at 19:58):

What I usually do when I write these tests is copy some other file such as fadd.clif and modify it, it has a bunch of interesting f32 and f64 values that provide a good base to start from!

view this post on Zulip Afonso Bordado (Aug 29 2022 at 19:59):

Hope that helps!

view this post on Zulip Jun Ryung Ju (Aug 29 2022 at 20:01):

um I was talking about what to test for float bit ops. looks your test on fadd/fsub/fmul/fdiv covers almost all cases so. is there a reference to cover bits ops on float feature? like from llvm bit operation tests. (well, I just realized I can just copy and paste some llvm bit ops tests.)

view this post on Zulip Jun Ryung Ju (Aug 29 2022 at 20:02):

Afonso Bordado 말함:

To build a run test you can do the following:

test interpret
test run
target x86_64
target aarch64
target s390x

function %band_f32(f32, f32) -> f32 {
block0(v0: f32, v1: f32):
    v2 = band v0, v1
    return v2
}
; run: %band_f32(0x0.5, 0x1.0) == 0x1.5

put that in a file in filetests/filetests/runtests/band-floats.clif and the you can run it by invoking cargo run -- test filetests/filetests/runtests/band-floats.clif from the cranelift directory

by the way this also helped me a lot.

view this post on Zulip Jun Ryung Ju (Aug 29 2022 at 20:02):

thank you for the help!

view this post on Zulip Afonso Bordado (Aug 29 2022 at 20:02):

band should pretty much allow all floats with no exceptions (unlike for example NaN's for fadd) so as long as we have a few tests on each category of floats it should be ok

view this post on Zulip Afonso Bordado (Aug 29 2022 at 20:03):

i.e. for fadd we don't assert NaN's as strictly as we can with band

view this post on Zulip Jamey Sharp (Aug 29 2022 at 23:56):

@Afonso Bordado, that looks like a great start to some documentation on the filetests. Would you mind opening a PR just putting that kind of information in cranelift/filetests/README.md or something? I could have used that a few weeks ago. :laughing: You don't have to add any more detail, we can always expand it later.

view this post on Zulip Jun Ryung Ju (Sep 05 2022 at 10:35):

I started working on this. I think I can finish this before this week.

view this post on Zulip Jun Ryung Ju (Sep 05 2022 at 10:39):

The PR will be separate into two pieces. one is for basic ops(and, or, xor, not), and another one is shift ops(shl, shr, and so on...)

view this post on Zulip Jun Ryung Ju (Sep 05 2022 at 10:39):

I'll make PR ASAP when its done :) thanks for helping.

view this post on Zulip Afonso Bordado (Sep 05 2022 at 11:05):

No problem, and thanks for working on this!

view this post on Zulip Jun Ryung Ju (Sep 05 2022 at 12:16):

I don't know why but cranelift is trying to lower test BB into native instructions. (I am using aarch64-darwin on host which is apple M1/M2 system) also this makes my test fails. if I remove "target aarch64" line from test clif. test success.

 TRACE cranelift_codegen::machinst::lower      > about to lower function: function u0:0(f32, f32) -> f32 apple_aarch64 {
block0(v0: f32, v1: f32):
    v2 = band v0, v1
    return v2
}
FAIL filetests/filetests/runtests/f32-bitops.clif: panicked in worker #0: assertion failed: `(left == right)`
  left: `Int`,
 right: `Float`

can you help me to resolve this issue? or should I touch on instruction lowering section. (maybe missing i32 to f32 lowering stuffs? this should work fine because selected registers have same size. looks just type sanity check fails.)

view this post on Zulip Jun Ryung Ju (Sep 05 2022 at 12:17):

This is lowered aarch64 instruction tho.

VCode {
  Entry block: 0
  v130 := v132
Block 0:
    (original IR block: block0)
    (instruction range: 0 .. 6)
  Inst 0: fmov %v128, d0
  Inst 1: fmov %v129, d1
  Inst 2: and %v132, %v128, %v129
  Inst 3: fmov %v131, %v130
  Inst 4: fmov d0, %v131
  Inst 5: ret
}

view this post on Zulip Jun Ryung Ju (Sep 05 2022 at 12:18):

the panic is on this assertion check. /regalloc2-0.3.2/src/ion/liveranges.rs:526:25

view this post on Zulip Jun Ryung Ju (Sep 05 2022 at 12:19):

https://github.com/bytecodealliance/regalloc2/blob/be47ac39e80471250e010d7b526105b7a0dcd4c1/src/ion/liveranges.rs#L526

A new register allocator. Contribute to bytecodealliance/regalloc2 development by creating an account on GitHub.

view this post on Zulip Jun Ryung Ju (Sep 05 2022 at 12:29):

Yeah looks and instr returns integer type as return type. maybe need lower %v132into float register class type so sanity check passes.

view this post on Zulip Afonso Bordado (Sep 05 2022 at 13:11):

The good news is that if we are trying to run the code in the native target, it means that the interpreter tests passed!

But it looks like our native support for these operations is quite incomplete (I originally looked at some parts of the backend and thought that these operations were available).

I tried to run this on x86 and it fails in ISLE probably due to not being handled correctly and I also tried on aarch64 (linux) and got the same error as you. s390x also does not implement this.

maybe missing i32 to f32 lowering stuffs? this should work fine because selected registers have same size. looks just type sanity check fails.

Maybe, I'm not very familiar with how the register allocator works but I think we do track floats and integers as separate classes.

In any case, usually for operations that the backends don't implement yet we disable tests for them, in this case since none of the backends implement them we can remove the test run and all of the target's.

view this post on Zulip Afonso Bordado (Sep 05 2022 at 13:12):

@bjorn3 Is this something that would be useful for cg_clif? Otherwise I think it could be a good candidate for removing these ops?

view this post on Zulip bjorn3 (Sep 05 2022 at 13:13):

Which ops?

view this post on Zulip Afonso Bordado (Sep 05 2022 at 13:13):

bitwise ops on floats

view this post on Zulip Afonso Bordado (Sep 05 2022 at 13:13):

band/bor / etc..

view this post on Zulip bjorn3 (Sep 05 2022 at 13:13):

We even support that?

view this post on Zulip Afonso Bordado (Sep 05 2022 at 13:13):

well. Its technically legal, but unimplemented everywhere

view this post on Zulip bjorn3 (Sep 05 2022 at 13:14):

Rust doesn't have bitwise ops on floats. If it needs them, it does a transmute, which for float -> int is implemented as bitcast.

view this post on Zulip Afonso Bordado (Sep 05 2022 at 13:44):

@Jun Ryung Ju I've opened https://github.com/bytecodealliance/wasmtime/issues/4870 to track the status of these ops. I'm really sorry about asking you to work on this, I had the idea that they were implemented correctly!

Hopefully we can cleanup our opcodes instead which is also a unintended benefit!

👋 Hey, Jun Ryung Ju (sorry, I don't know the github user) on zulip discovered that bitwise operations on floats are unimplemented on AArch64. Upon further investigation they seem to be unimplem...

view this post on Zulip Jun Ryung Ju (Sep 05 2022 at 13:53):

can I try implement the lowering thing?

view this post on Zulip Afonso Bordado (Sep 05 2022 at 13:54):

Sure, but you might want to wait for a response on that issue, otherwise it might not be worth it if we are going to remove them

view this post on Zulip Jun Ryung Ju (Sep 06 2022 at 07:31):

I'll work on fcvt_to_int and all its variants feature instead of this while waiting :)

view this post on Zulip Jamey Sharp (Sep 07 2022 at 20:05):

We seem to have consensus that we should remove the bitwise operators on floats. Thank you for noticing this!

view this post on Zulip Jamey Sharp (Sep 07 2022 at 23:36):

When I tried removing the bitwise operators on floats, I discovered that Wasmtime actually uses them (on SIMD float vectors). So we're not removing them after all, and if you want to work on implementing them correctly for scalar floats, that'd be great!

view this post on Zulip Jun Ryung Ju (Sep 09 2022 at 10:58):

Jamey Sharp 말함:

When I tried removing the bitwise operators on floats, I discovered that Wasmtime actually uses them (on SIMD float vectors). So we're not removing them after all, and if you want to work on implementing them correctly for scalar floats, that'd be great!

Ok I'll try that. thanks for clarification.

view this post on Zulip Jun Ryung Ju (Sep 09 2022 at 11:00):

but I think seems this need change for SIMD lowering instead of implementing scalar float ops. (like using raw bit level casting)

view this post on Zulip Jun Ryung Ju (Sep 09 2022 at 11:01):

any decision for this?

view this post on Zulip Afonso Bordado (Sep 09 2022 at 14:56):

From what I understand I think we are going to go ahead with leaving the bit ops there for all types instead of switching to bitcast + int op.

I'm going to wait a bit to see if anyone else comments before opening issues against all backends

view this post on Zulip Afonso Bordado (Sep 09 2022 at 14:57):

You also might be interested in this: https://github.com/bytecodealliance/wasmtime/issues/4889

I finally had some time to do a proper roundup of what is or isn't working in the interpreter and combine it into a tracking issue.

Hey, I did a roundup of what is supported in the interpreter and what is tested or not. Now that we have multiple people working on the interpreter this is probably a better way to keep track of wh...

view this post on Zulip Jun Ryung Ju (Sep 18 2022 at 11:45):

Can I take task that lowers float bitops into x86/64 instruction after this?

view this post on Zulip Afonso Bordado (Sep 20 2022 at 08:51):

Sure, as far as I know no one else is working on that

view this post on Zulip Jamey Sharp (Sep 20 2022 at 19:19):

Yes, you're welcome to do that! It's easier to do once you've written some runtests for these bitops and ensured they pass on the interpreter, so I'll try to merge your PR https://github.com/bytecodealliance/wasmtime/pull/4920 as soon as you've added some tests and it passes CI. (Note that CI is currently failing because you need to run cargo fmt.)

Tests are not ready. I'll add it ASAP. @afonso360

view this post on Zulip Wojciech Niedźwiedź (Sep 21 2022 at 09:57):

howdy

view this post on Zulip Wojciech Niedźwiedź (Sep 21 2022 at 09:57):

I have this kinda strange idea

view this post on Zulip Wojciech Niedźwiedź (Sep 21 2022 at 09:58):

for those of you who know docker and docker compose - you probably heard this phrase that wasm kinda solves the same problems as docker (isolation, scoped permissions etc)

view this post on Zulip Wojciech Niedźwiedź (Sep 21 2022 at 09:58):

so I was thinking - why not create something like a wasmtime-compose.yml file where you put all the config for how to set up whole project

view this post on Zulip Wojciech Niedźwiedź (Sep 21 2022 at 09:59):

things that aren't really compiled to wasm like postgres could run on docker using a bridge, but stuff that is compilable to wasm would use wasmtime

view this post on Zulip Wojciech Niedźwiedź (Sep 21 2022 at 10:01):

so as an MVP it could be just a superset of docker-compose's file syntax that extends it with some wasm options, and over time as an ecosystem grows the docker bridge would become less and less important, until eventually you'd end up with purely wasm-deployed apps - just with a good old familiar syntax

view this post on Zulip Wojciech Niedźwiedź (Sep 21 2022 at 10:03):

does that sound feasible? as I said it's just a wild shower idea I had - think docker-compose for wasm-binaries

view this post on Zulip Peter Huene (Sep 21 2022 at 16:39):

@Wojciech Niedźwiedź I think tooling in this space makes a lot of sense and docker compose was some of the inspiration I had in making wasm-compose (https://github.com/bytecodealliance/wasm-tools/tree/main/crates/wasm-compose), where the idea is instead of composing containers together, it composes WebAssembly components (based on the component model proposal). The tool right now is pretty simple and I hope to evolve it over time with the component model itself. I'm happy to receive feedback about the tool or hear your ideas regarding what it means to you to compose WebAssembly as well (GitHub issues would probably be the best way to do that)!

Low level tooling for WebAssembly in Rust. Contribute to bytecodealliance/wasm-tools development by creating an account on GitHub.

view this post on Zulip Peter Huene (Sep 21 2022 at 16:41):

Note, though, that if you're inclined to kick the tires following the example, right now cargo component is out of date with respect to the current component model proposal which hopefully will be fixed shortly now that there's a PR to update one of its dependencies.

view this post on Zulip Peter Huene (Sep 21 2022 at 16:44):

Obviously the tool doesn't do a hybrid model of working with some containerized applications and some wasm; what you propose sounds quite interesting as a way to bring wasm-based services into an existing orchestration.

view this post on Zulip Wojciech Niedźwiedź (Sep 22 2022 at 09:22):

wow, I feel like a genious now xD I'm in no place to give any serious recommendation as I'm just a humble python developer who started writing rust just couple years ago for hobby projects - as I said just a shower idea. Feels really good to have the same idea as a smart person though

view this post on Zulip Wojciech Niedźwiedź (Sep 22 2022 at 09:25):

but what docker-compose does for me is I can define multiple services, give instructions on how to build them and close them down together in the same network - (plus simple deployments on a VPS using docker-compose up -d). for wasm you could also give permissions to the "service", like - can it access the internet etc. I can also just invite someone to collaborate and tell them to "just type docker-compose up and it'll work"

view this post on Zulip Notification Bot (Oct 10 2022 at 00:11):

Jun Ryung Ju has marked this topic as resolved.


Last updated: Dec 23 2024 at 12:05 UTC