wasmtime / issue #5499 FreeBSD arm64/aarch64 support · git-wasmtime

Stream: git-wasmtime

Topic: wasmtime / issue #5499 FreeBSD arm64/aarch64 support

Wasmtime GitHub notifications bot (Dec 30 2022 at 11:36):

nunotexbsd opened issue #5499:

Feature

Add FreeBSD support on arm64/aarch64

Implementation

Add to https://github.com/bytecodealliance/wasmtime/blob/main/crates/runtime/src/traphandlers/unix.rs
 } else if #[cfg(all(target_os = "freebsd", target_arch = "aarch64"))] {
                    pc = context.uc_mcontext.mc_gpregs.gp_elr as usize;
                    sp = context.uc_mcontext.mc_gpregs.gp_sp as usize;
I take this code from https://github.com/wasmerio/wasmer/blob/dcfdea76999a/lib/vm/src/trap/traphandlers.rs#L302-L304 but don't know if some change needs to be made.

0.2.5 build log:
https://pkg-status.freebsd.org/ampere3/data/131arm64-default/b02e58aec16f/logs/lapce-0.2.5_1.log

Wasmtime GitHub notifications bot (Jan 03 2023 at 16:47):

alexcrichton labeled issue #5499:

Feature

Add FreeBSD support on arm64/aarch64

Implementation

Add to https://github.com/bytecodealliance/wasmtime/blob/main/crates/runtime/src/traphandlers/unix.rs
 } else if #[cfg(all(target_os = "freebsd", target_arch = "aarch64"))] {
                    pc = context.uc_mcontext.mc_gpregs.gp_elr as usize;
                    sp = context.uc_mcontext.mc_gpregs.gp_sp as usize;
I take this code from https://github.com/wasmerio/wasmer/blob/dcfdea76999a/lib/vm/src/trap/traphandlers.rs#L302-L304 but don't know if some change needs to be made.

0.2.5 build log:
https://pkg-status.freebsd.org/ampere3/data/131arm64-default/b02e58aec16f/logs/lapce-0.2.5_1.log

Wasmtime GitHub notifications bot (Jan 03 2023 at 16:47):

alexcrichton commented on issue #5499:

This is not currently a supported platform but if you'd like to send a PR to update the code it would be appreciated!

Wasmtime GitHub notifications bot (Jan 03 2023 at 20:25):

nunotexbsd commented on issue #5499:

@alexcrichton

Nice I will do that. Soon I will have arm64 hardware to further tests instead on relying on freebsd build servers.

Thanks

Wasmtime GitHub notifications bot (Jan 07 2023 at 11:08):

afonso360 commented on issue #5499:

We probably also need to update the wasmtime-jit-icache-coherence crate.

I looked into this a little bit in https://github.com/bytecodealliance/wasmtime/pull/5323 but it looks like FreeBSD doesn't yet support the membarrier interface. But perhaps there is something else that we can use?

I had a look at what other JIT's are doing:

V8 emits its own icache maintenance instructions.

Spidermonkey uses membarriers

Mono also emits their own icache maintenance instructions

Maybe FreeBSD has some other syscall that we can use? I'm not too familiar with it.

CC: #3310

Wasmtime GitHub notifications bot (Jan 13 2023 at 10:30):

sec commented on issue #5499:

Just to add something, for latest relase (6.0.0), small change is need to make it compile:
} else if #[cfg(all(target_os = "freebsd", target_arch = "aarch64"))] {
    let cx = &*(cx as *const libc::mcontext_t);
    (
        cx.mc_gpregs.gp_elr as *const u8,
        cx.mc_gpregs.gp_x[29] as usize,
    )
around https://github.com/bytecodealliance/wasmtime/blob/main/crates/runtime/src/traphandlers/unix.rs#L220

This make it compile, but running tests is not possible becuase of https://github.com/denoland/rusty_v8/issues/1094

Wasmtime GitHub notifications bot (Jan 13 2023 at 12:53):

nunotexbsd commented on issue #5499:

@sec
Thanks for testing it. I still waiting for arm64 hardware so I can't test it.
I will apply your patch and try to find someone that build and run test for me.

Wasmtime GitHub notifications bot (Jan 13 2023 at 12:56):

nunotexbsd edited a comment on issue #5499:

@sec
Thanks for testing it. I still waiting for arm64 hardware so I can't test it.
I will apply your patch and try to find someone that build and run test for me.

https://github.com/lapce/lapce , FreeBSD port, will be tested on arm64.aarch64

Wasmtime GitHub notifications bot (Jan 13 2023 at 18:08):

afonso360 commented on issue #5499:

:wave: Hey, I've tested the above patch in FreeBSD 14.0-CURRENT

freebsd@generic:~ % uname -a
FreeBSD generic 14.0-CURRENT FreeBSD 14.0-CURRENT #0 main-n259967-11b5b9e8a520: Sat Jan  7 18:30:37 UTC 2023     root@releng1.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC arm64

cargo build and cargo run work.

freebsd@generic:~/git/wasmtime % cargo run ./examples/gcd.wat --invoke gcd 8 8
    Finished dev [unoptimized + debuginfo] target(s) in 2.21s
     Running `target/debug/wasmtime ./examples/gcd.wat --invoke gcd 8 8`
warning: using `--invoke` with a function that takes arguments is experimental and may break in the future
warning: using `--invoke` with a function that returns values is experimental and may break in the future
8

I was able to run cargo test, but the tests fail with illegal instruction

     Running unittests src/lib.rs (target/debug/deps/wasmtime_cli-f9e37a0a7433af84)

running 2 tests
test commands::compile::test::test_aarch64_flags_compile ... ok
test commands::compile::test::test_successful_compile ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.57s

     Running unittests src/bin/wasmtime.rs (target/debug/deps/wasmtime-27a7f646c50db093)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/all/main.rs (target/debug/deps/all-619db9ec6a9e4647)

running 819 tests
test async_functions::async_with_pooling_stacks ... ok
test async_functions::cancel_during_run ... ok
test async_functions::async_host_func_with_pooling_stacks ... ok
test async_functions::fuel_eventually_finishes ... ok
error: test failed, to rerun pass `--test all`

Caused by:
  process didn't exit successfully: `/home/freebsd/git/wasmtime/target/debug/deps/all-619db9ec6a9e4647` (signal: 4, SIGILL: illegal instruction)

Wasmtime GitHub notifications bot (Jan 13 2023 at 18:10):

afonso360 edited a comment on issue #5499:

:wave: Hey, I've tested the above patch in FreeBSD 14.0-CURRENT

freebsd@generic:~ % uname -a
FreeBSD generic 14.0-CURRENT FreeBSD 14.0-CURRENT #0 main-n259967-11b5b9e8a520: Sat Jan  7 18:30:37 UTC 2023     root@releng1.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC arm64

cargo build and cargo run work.

freebsd@generic:~/git/wasmtime % cargo run ./examples/gcd.wat --invoke gcd 8 8
    Finished dev [unoptimized + debuginfo] target(s) in 2.21s
     Running `target/debug/wasmtime ./examples/gcd.wat --invoke gcd 8 8`
warning: using `--invoke` with a function that takes arguments is experimental and may break in the future
warning: using `--invoke` with a function that returns values is experimental and may break in the future
8

I was able to run cargo test, but the tests fail with illegal instruction

     Running unittests src/lib.rs (target/debug/deps/wasmtime_cli-f9e37a0a7433af84)

running 2 tests
test commands::compile::test::test_aarch64_flags_compile ... ok
test commands::compile::test::test_successful_compile ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.57s

     Running unittests src/bin/wasmtime.rs (target/debug/deps/wasmtime-27a7f646c50db093)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/all/main.rs (target/debug/deps/all-619db9ec6a9e4647)

running 819 tests
test async_functions::async_with_pooling_stacks ... ok
test async_functions::cancel_during_run ... ok
test async_functions::async_host_func_with_pooling_stacks ... ok
test async_functions::fuel_eventually_finishes ... ok
error: test failed, to rerun pass `--test all`

Caused by:
  process didn't exit successfully: `/home/freebsd/git/wasmtime/target/debug/deps/all-619db9ec6a9e4647` (signal: 4, SIGILL: illegal instruction)

I would guess this is due to the missing icache code. The symptoms are fairly similar to what was happening in #4997 for Windows AArch64.

Wasmtime GitHub notifications bot (Jan 13 2023 at 18:28):

afonso360 edited a comment on issue #5499:

:wave: Hey, I've tested the above patch in FreeBSD 14.0-CURRENT

freebsd@generic:~ % uname -a
FreeBSD generic 14.0-CURRENT FreeBSD 14.0-CURRENT #0 main-n259967-11b5b9e8a520: Sat Jan  7 18:30:37 UTC 2023     root@releng1.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC arm64

cargo build and cargo run work.

freebsd@generic:~/git/wasmtime % cargo run ./examples/gcd.wat --invoke gcd 8 8
    Finished dev [unoptimized + debuginfo] target(s) in 2.21s
     Running `target/debug/wasmtime ./examples/gcd.wat --invoke gcd 8 8`
warning: using `--invoke` with a function that takes arguments is experimental and may break in the future
warning: using `--invoke` with a function that returns values is experimental and may break in the future
8

I was able to run cargo test, but the tests fail with illegal instruction

     Running unittests src/lib.rs (target/debug/deps/wasmtime_cli-f9e37a0a7433af84)

running 2 tests
test commands::compile::test::test_aarch64_flags_compile ... ok
test commands::compile::test::test_successful_compile ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.57s

     Running unittests src/bin/wasmtime.rs (target/debug/deps/wasmtime-27a7f646c50db093)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/all/main.rs (target/debug/deps/all-619db9ec6a9e4647)

running 819 tests
test async_functions::async_with_pooling_stacks ... ok
test async_functions::cancel_during_run ... ok
test async_functions::async_host_func_with_pooling_stacks ... ok
test async_functions::fuel_eventually_finishes ... ok
error: test failed, to rerun pass `--test all`

Caused by:
  process didn't exit successfully: `/home/freebsd/git/wasmtime/target/debug/deps/all-619db9ec6a9e4647` (signal: 4, SIGILL: illegal instruction)

I would guess this is due to the missing icache code. The symptoms are fairly similar to what was happening in #4997 for Windows AArch64.

Edit: The cranelift test suite passes, which I wouldn't have expected :thinking: Maybe something else is going on here.

Wasmtime GitHub notifications bot (Jan 13 2023 at 22:24):

nunotexbsd commented on issue #5499:

@afonso360
Thanks for do further tests.
I'm working on https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=268929 and if it succeeds, then I can start testing other ports that depends on this crate.

Wasmtime GitHub notifications bot (Jan 15 2023 at 13:59):

nunotexbsd commented on issue #5499:

Just to add something, for latest relase (6.0.0), small change is need to make it compile:

} else if #[cfg(all(target_os = "freebsd", target_arch = "aarch64"))] { let cx = &*(cx as *const libc::mcontext_t); ( cx.mc_gpregs.gp_elr as *const u8, cx.mc_gpregs.gp_x[29] as usize, )

around https://github.com/bytecodealliance/wasmtime/blob/main/crates/runtime/src/traphandlers/unix.rs#L220

This make it compile, but running tests is not possible becuase of denoland/rusty_v8#1094

Hello @sec,

I have results that it builds on aarch64 and I committed your patch at https://cgit.freebsd.org/ports/commit/?id=75a1d429216aa977d1bf6b1f4a9c5eb4eb474ac8

Should a upstream change be made with this patch?

( Build fails on armv7 FreeBSD 13.1:
error[E0308]: mismatched types
   --> /wrkdirs/usr/ports/editors/lapce/work/lapce-0.2.5/cargo-crates/rustix-0.35.10/src/backend/libc/net/syscalls.rs:565:21
    |
565 |                     tv_sec,
    |                     ^^^^^^ expected `i64`, found `i32`
    |                                                                                                                              help: you can convert an `i32` to an `i64`
    |
565 |                     tv_sec: tv_sec.into(),
    |                     +++++++       +++++++

[num-traits 0.2.15] cargo:rustc-cfg=has_leading_trailing_ones
[num-traits 0.2.15] cargo:rustc-cfg=has_int_assignop_ref
For more information about this error, try `rustc --explain E0308`.
error: could not compile `rustix` due to previous error
This is due to time_t not being a long on armv7. )

Wasmtime GitHub notifications bot (Jan 17 2023 at 18:13):

sunfishcode commented on issue #5499:

I have a fix for that armv7 FreeBSD bug here; I'll do a point release of rustix and update Wasmtime.

Wasmtime GitHub notifications bot (Jan 17 2023 at 21:38):

afonso360 commented on issue #5499:

Looked into this a bit more, this branch also does the icache clearing. However I'm still getting the same error.

I'll try to track this down when I get a chance.

Last updated: Dec 23 2024 at 12:05 UTC