Jun Ryung Ju (Aug 25 2022 at 11:57): Jun Ryung Ju (Aug 25 2022 at 11:57):I am looking for compiler repository that I can contribute. but it looks 'Easy' labeled issues on github are doesn't seem to be maintained. can I get some easy task for cranelift stuffs? I have experience with modifying CoreCLR(which is dotnet runtime) optimization passes, LLVM optimization, lowering passes.
Chris Fallin (Aug 25 2022 at 16:37):@Jun Ryung Ju welcome and thank you very much for your interest and stopping by here!
We haven't done a great job of maintaining a list of easy issues, but some of the big ideas in many of them are still relevant (and actually always relevant), mainly:
On the third item above (testing infrastructure), @Afonso Bordado is currently working to build out and improve a "CLIF interpreter", which is an interpreter of our intermediate language (CLIF) that we can use as a reference to test the real compiler backends against. AFAIK it does not yet have implementations for all opcodes. @Afonso Bordado are there easy-to-separate pieces of work here that @Jun Ryung Ju might be interested in?
Afonso Bordado (Aug 25 2022 at 16:49)::wave: Hey, Welcome to the project!
Yes, there are a bunch of opcodes not yet implemented in the interpreter. The two groups that come to mind are:
f32/f64fcvt_to_int and all its variantsThe reason these are important is that they are already implemented in the various backends, so when they are ready on the interpreter we can start fuzzing on them (which is the main driver for the work on the interpreter). Mostly its adding tests to our test suite and then working on a implementation on the interpreter.
Here's an example PR adding a few instructions
Afonso Bordado (Aug 25 2022 at 17:05):If you'd like to work on something related to compiler passes, you might want to read this thread that has a bunch of information regarding the current state of the compiler:
https://github.com/bytecodealliance/wasmtime/issues/4712
Although I don't know how easy/hard any of those items are
Jun Ryung Ju (Aug 25 2022 at 17:22):Afonso Bordado 말함:
:wave: Hey, Welcome to the project!
Yes, there are a bunch of opcodes not yet implemented in the interpreter. The two groups that come to mind are:
- Bit ops on floats
f32/f64fcvt_to_intand all its variantsThe reason these are important is that they are already implemented in the various backends, so when they are ready on the interpreter we can start fuzzing on them (which is the main driver for the work on the interpreter). Mostly its adding tests to our test suite and then working on a implementation on the interpreter.
Here's an example PR adding a few instructions
Thanks for helping me out @Afonso Bordado
I think task that you suggested are good task to start contribute on cranelift. I'll take that :)
Afonso Bordado (Aug 25 2022 at 17:23):If you need any help let me know
Jun Ryung Ju (Aug 29 2022 at 19:28):Are there any resources I can refer to make a strict test? I have no idea what needs to be tested on "Bit ops on floats f32/f64" feature.
Jun Ryung Ju (Aug 29 2022 at 19:28):I mean for the strict test. not simple test.
Afonso Bordado (Aug 29 2022 at 19:51):I'm not sure what you mean by strict test?
Afonso Bordado (Aug 29 2022 at 19:52):The most common tests we have are compile tests and run tests
Afonso Bordado (Aug 29 2022 at 19:52):The interpreter can only run the latter
Afonso Bordado (Aug 29 2022 at 19:53):so, "Bit ops on floats" is kind of a broad term, it means a bunch of bit wise instructions that can also be used on floats, such as:
etc...
Afonso Bordado (Aug 29 2022 at 19:55):To build a run test you can do the following:
test interpret
test run
target x86_64
target aarch64
target s390x
function %band_f32(f32, f32) -> f32 {
block0(v0: f32, v1: f32):
v2 = band v0, v1
return v2
}
; run: %band_f32(0x0.5, 0x1.0) == 0x1.5
put that in a file in filetests/filetests/runtests/band-floats.clif and the you can run it by invoking cargo run -- test filetests/filetests/runtests/band-floats.clif from the cranelift directory
Afonso Bordado (Aug 29 2022 at 19:55):(i haven't tested the output of that, Its probably wrong!)
Afonso Bordado (Aug 29 2022 at 19:57):the first lines tell clif-util what kind of tests we want to run on this file. test interpret invokes the interpreter and checks if the conditions in the ; run comments pass. test run does the same, but compiles the file and runs it as a native binary
Afonso Bordado (Aug 29 2022 at 19:58):What I usually do when I write these tests is copy some other file such as fadd.clif and modify it, it has a bunch of interesting f32 and f64 values that provide a good base to start from!
Afonso Bordado (Aug 29 2022 at 19:59):Hope that helps!
Jun Ryung Ju (Aug 29 2022 at 20:01):um I was talking about what to test for float bit ops. looks your test on fadd/fsub/fmul/fdiv covers almost all cases so. is there a reference to cover bits ops on float feature? like from llvm bit operation tests. (well, I just realized I can just copy and paste some llvm bit ops tests.)
Jun Ryung Ju (Aug 29 2022 at 20:02):Afonso Bordado 말함:
To build a run test you can do the following:
test interpret test run target x86_64 target aarch64 target s390x function %band_f32(f32, f32) -> f32 { block0(v0: f32, v1: f32): v2 = band v0, v1 return v2 } ; run: %band_f32(0x0.5, 0x1.0) == 0x1.5put that in a file in
filetests/filetests/runtests/band-floats.clifand the you can run it by invokingcargo run -- test filetests/filetests/runtests/band-floats.cliffrom the cranelift directory
by the way this also helped me a lot.
Jun Ryung Ju (Aug 29 2022 at 20:02):thank you for the help!
Afonso Bordado (Aug 29 2022 at 20:02):band should pretty much allow all floats with no exceptions (unlike for example NaN's for fadd) so as long as we have a few tests on each category of floats it should be ok
Afonso Bordado (Aug 29 2022 at 20:03):i.e. for fadd we don't assert NaN's as strictly as we can with band
Jamey Sharp (Aug 29 2022 at 23:56):@Afonso Bordado, that looks like a great start to some documentation on the filetests. Would you mind opening a PR just putting that kind of information in cranelift/filetests/README.md or something? I could have used that a few weeks ago. :laughing: You don't have to add any more detail, we can always expand it later.
Jun Ryung Ju (Sep 05 2022 at 10:35):I started working on this. I think I can finish this before this week.
Jun Ryung Ju (Sep 05 2022 at 10:39):The PR will be separate into two pieces. one is for basic ops(and, or, xor, not), and another one is shift ops(shl, shr, and so on...)
Jun Ryung Ju (Sep 05 2022 at 10:39):I'll make PR ASAP when its done :) thanks for helping.
Afonso Bordado (Sep 05 2022 at 11:05):No problem, and thanks for working on this!
Jun Ryung Ju (Sep 05 2022 at 12:16):I don't know why but cranelift is trying to lower test BB into native instructions. (I am using aarch64-darwin on host which is apple M1/M2 system) also this makes my test fails. if I remove "target aarch64" line from test clif. test success.
TRACE cranelift_codegen::machinst::lower > about to lower function: function u0:0(f32, f32) -> f32 apple_aarch64 {
block0(v0: f32, v1: f32):
v2 = band v0, v1
return v2
}
FAIL filetests/filetests/runtests/f32-bitops.clif: panicked in worker #0: assertion failed: `(left == right)`
left: `Int`,
right: `Float`
can you help me to resolve this issue? or should I touch on instruction lowering section. (maybe missing i32 to f32 lowering stuffs? this should work fine because selected registers have same size. looks just type sanity check fails.)
Jun Ryung Ju (Sep 05 2022 at 12:17):This is lowered aarch64 instruction tho.
VCode {
Entry block: 0
v130 := v132
Block 0:
(original IR block: block0)
(instruction range: 0 .. 6)
Inst 0: fmov %v128, d0
Inst 1: fmov %v129, d1
Inst 2: and %v132, %v128, %v129
Inst 3: fmov %v131, %v130
Inst 4: fmov d0, %v131
Inst 5: ret
}
the panic is on this assertion check. /regalloc2-0.3.2/src/ion/liveranges.rs:526:25
Jun Ryung Ju (Sep 05 2022 at 12:19): Jun Ryung Ju (Sep 05 2022 at 12:29):Yeah looks and instr returns integer type as return type. maybe need lower %v132into float register class type so sanity check passes.
Afonso Bordado (Sep 05 2022 at 13:11):The good news is that if we are trying to run the code in the native target, it means that the interpreter tests passed!
But it looks like our native support for these operations is quite incomplete (I originally looked at some parts of the backend and thought that these operations were available).
I tried to run this on x86 and it fails in ISLE probably due to not being handled correctly and I also tried on aarch64 (linux) and got the same error as you. s390x also does not implement this.
maybe missing i32 to f32 lowering stuffs? this should work fine because selected registers have same size. looks just type sanity check fails.
Maybe, I'm not very familiar with how the register allocator works but I think we do track floats and integers as separate classes.
In any case, usually for operations that the backends don't implement yet we disable tests for them, in this case since none of the backends implement them we can remove the test run and all of the target's.
Afonso Bordado (Sep 05 2022 at 13:12):@bjorn3 Is this something that would be useful for cg_clif? Otherwise I think it could be a good candidate for removing these ops?
bjorn3 (Sep 05 2022 at 13:13):Which ops?
Afonso Bordado (Sep 05 2022 at 13:13):bitwise ops on floats
Afonso Bordado (Sep 05 2022 at 13:13):band/bor / etc..
bjorn3 (Sep 05 2022 at 13:13):We even support that?
Afonso Bordado (Sep 05 2022 at 13:13):well. Its technically legal, but unimplemented everywhere
bjorn3 (Sep 05 2022 at 13:14):Rust doesn't have bitwise ops on floats. If it needs them, it does a transmute, which for float -> int is implemented as bitcast.
Afonso Bordado (Sep 05 2022 at 13:44):@Jun Ryung Ju I've opened https://github.com/bytecodealliance/wasmtime/issues/4870 to track the status of these ops. I'm really sorry about asking you to work on this, I had the idea that they were implemented correctly!
Hopefully we can cleanup our opcodes instead which is also a unintended benefit!
Jun Ryung Ju (Sep 05 2022 at 13:53):can I try implement the lowering thing?
Afonso Bordado (Sep 05 2022 at 13:54):Sure, but you might want to wait for a response on that issue, otherwise it might not be worth it if we are going to remove them
Jun Ryung Ju (Sep 06 2022 at 07:31):I'll work on fcvt_to_int and all its variants feature instead of this while waiting :)
Jamey Sharp (Sep 07 2022 at 20:05):We seem to have consensus that we should remove the bitwise operators on floats. Thank you for noticing this!
Jamey Sharp (Sep 07 2022 at 23:36):When I tried removing the bitwise operators on floats, I discovered that Wasmtime actually uses them (on SIMD float vectors). So we're not removing them after all, and if you want to work on implementing them correctly for scalar floats, that'd be great!
Jun Ryung Ju (Sep 09 2022 at 10:58):Jamey Sharp 말함:
When I tried removing the bitwise operators on floats, I discovered that Wasmtime actually uses them (on SIMD float vectors). So we're not removing them after all, and if you want to work on implementing them correctly for scalar floats, that'd be great!
Ok I'll try that. thanks for clarification.
Jun Ryung Ju (Sep 09 2022 at 11:00):but I think seems this need change for SIMD lowering instead of implementing scalar float ops. (like using raw bit level casting)
Jun Ryung Ju (Sep 09 2022 at 11:01):any decision for this?
Afonso Bordado (Sep 09 2022 at 14:56):From what I understand I think we are going to go ahead with leaving the bit ops there for all types instead of switching to bitcast + int op.
I'm going to wait a bit to see if anyone else comments before opening issues against all backends
Afonso Bordado (Sep 09 2022 at 14:57):You also might be interested in this: https://github.com/bytecodealliance/wasmtime/issues/4889
I finally had some time to do a proper roundup of what is or isn't working in the interpreter and combine it into a tracking issue.
Jun Ryung Ju (Sep 18 2022 at 11:45):Can I take task that lowers float bitops into x86/64 instruction after this?
Afonso Bordado (Sep 20 2022 at 08:51):Sure, as far as I know no one else is working on that
Jamey Sharp (Sep 20 2022 at 19:19):Yes, you're welcome to do that! It's easier to do once you've written some runtests for these bitops and ensured they pass on the interpreter, so I'll try to merge your PR https://github.com/bytecodealliance/wasmtime/pull/4920 as soon as you've added some tests and it passes CI. (Note that CI is currently failing because you need to run cargo fmt.)
Wojciech Niedźwiedź (Sep 21 2022 at 09:57):howdy
Wojciech Niedźwiedź (Sep 21 2022 at 09:57):I have this kinda strange idea
Wojciech Niedźwiedź (Sep 21 2022 at 09:58):for those of you who know docker and docker compose - you probably heard this phrase that wasm kinda solves the same problems as docker (isolation, scoped permissions etc)
Wojciech Niedźwiedź (Sep 21 2022 at 09:58):so I was thinking - why not create something like a wasmtime-compose.yml file where you put all the config for how to set up whole project
Wojciech Niedźwiedź (Sep 21 2022 at 09:59):things that aren't really compiled to wasm like postgres could run on docker using a bridge, but stuff that is compilable to wasm would use wasmtime
Wojciech Niedźwiedź (Sep 21 2022 at 10:01):so as an MVP it could be just a superset of docker-compose's file syntax that extends it with some wasm options, and over time as an ecosystem grows the docker bridge would become less and less important, until eventually you'd end up with purely wasm-deployed apps - just with a good old familiar syntax
Wojciech Niedźwiedź (Sep 21 2022 at 10:03):does that sound feasible? as I said it's just a wild shower idea I had - think docker-compose for wasm-binaries
Peter Huene (Sep 21 2022 at 16:39):@Wojciech Niedźwiedź I think tooling in this space makes a lot of sense and docker compose was some of the inspiration I had in making wasm-compose (https://github.com/bytecodealliance/wasm-tools/tree/main/crates/wasm-compose), where the idea is instead of composing containers together, it composes WebAssembly components (based on the component model proposal). The tool right now is pretty simple and I hope to evolve it over time with the component model itself. I'm happy to receive feedback about the tool or hear your ideas regarding what it means to you to compose WebAssembly as well (GitHub issues would probably be the best way to do that)!
Peter Huene (Sep 21 2022 at 16:41):Note, though, that if you're inclined to kick the tires following the example, right now cargo component is out of date with respect to the current component model proposal which hopefully will be fixed shortly now that there's a PR to update one of its dependencies.
Peter Huene (Sep 21 2022 at 16:44):Obviously the tool doesn't do a hybrid model of working with some containerized applications and some wasm; what you propose sounds quite interesting as a way to bring wasm-based services into an existing orchestration.
Wojciech Niedźwiedź (Sep 22 2022 at 09:22):wow, I feel like a genious now xD I'm in no place to give any serious recommendation as I'm just a humble python developer who started writing rust just couple years ago for hobby projects - as I said just a shower idea. Feels really good to have the same idea as a smart person though
Wojciech Niedźwiedź (Sep 22 2022 at 09:25):but what docker-compose does for me is I can define multiple services, give instructions on how to build them and close them down together in the same network - (plus simple deployments on a VPS using docker-compose up -d). for wasm you could also give permissions to the "service", like - can it access the internet etc. I can also just invite someone to collaborate and tell them to "just type docker-compose up and it'll work"
Notification Bot (Oct 10 2022 at 00:11):Jun Ryung Ju has marked this topic as resolved.
Last updated: Oct 23 2024 at 20:03 UTC