wasmtime / PR #720 Js-polyfill using wasi-common crate · git-wasmtime

There is a couple of issues with this PR, so I'm gonna mark it as a draft PR until all (or at least a vast majority is satisfactorily resolved) before moving forward with it.

Rustc regression wrt Emscripten in beta/nightly channels

It seems there might be a compiler regression wrt wasm32-unknown-emscripten target on beta and nightly channels. I'm still to add a CI job for building the introduced js-polyfill crate, but if you try and build with the latest Emscripten upstream LLVM backend, you will be presented with cryptic compilation failure of the num-integer crate:

Called function must be a pointer!
  call addrspace(7667714) void
error: could not compile `num-integer`.

Caused by:
  process didn't exit successfully: `rustc --crate-name num_integer /Users/kubkon/.cargo/registry/src/github.com-1ecc6299db9ec823/num-integer-0.1.41/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C debuginfo=2 -C metadata=4f0bdfeb6f9e43f4 -C extra-filename=-4f0bdfeb6f9e43f4 --out-dir /Users/kubkon/dev/wasmtime/crates/wasi-common/js-polyfill/target/wasm32-unknown-emscripten/debug/deps --target wasm32-unknown-emscripten -L dependency=/Users/kubkon/dev/wasmtime/crates/wasi-common/js-polyfill/target/wasm32-unknown-emscripten/debug/deps -L dependency=/Users/kubkon/dev/wasmtime/crates/wasi-common/js-polyfill/target/debug/deps --extern num_traits=/Users/kubkon/dev/wasmtime/crates/wasi-common/js-polyfill/target/wasm32-unknown-emscripten/debug/deps/libnum_traits-7aaa72299f995a0d.rmeta --cap-lints allow -C 'link-args=--js-library assets/load-files.js --shell-file assets/shell.html --pre-js assets/wasi.js -s EXPORTED_FUNCTIONS=['\''_main'\'','\''_get_wasi_context'\'','\''_handleFiles'\'','\''_old_wasi_common_args_get'\'','\''_old_wasi_common_args_sizes_get'\'','\''_old_wasi_common_clock_res_get'\'','\''_old_wasi_common_clock_time_get'\'','\''_old_wasi_common_environ_get'\'','\''_old_wasi_common_environ_sizes_get'\'','\''_old_wasi_common_fd_advise'\'','\''_old_wasi_common_fd_allocate'\'','\''_old_wasi_common_fd_close'\'','\''_old_wasi_common_fd_datasync'\'','\''_old_wasi_common_fd_fdstat_get'\'','\''_old_wasi_common_fd_fdstat_set_flags'\'','\''_old_wasi_common_fd_fdstat_set_rights'\'','\''_old_wasi_common_fd_filestat_get'\'','\''_old_wasi_common_fd_filestat_set_size'\'','\''_old_wasi_common_fd_filestat_set_times'\'','\''_old_wasi_common_fd_pread'\'','\''_old_wasi_common_fd_prestat_dir_name'\'','\''_old_wasi_common_fd_prestat_get'\'','\''_old_wasi_common_fd_pwrite'\'','\''_old_wasi_common_fd_read'\'','\''_old_wasi_common_fd_readdir'\'','\''_old_wasi_common_fd_renumber'\'','\''_old_wasi_common_fd_seek'\'','\''_old_wasi_common_fd_sync'\'','\''_old_wasi_common_fd_tell'\'','\''_old_wasi_common_fd_write'\'','\''_old_wasi_common_path_create_directory'\'','\''_old_wasi_common_path_filestat_get'\'','\''_old_wasi_common_path_filestat_set_times'\'','\''_old_wasi_common_path_link'\'','\''_old_wasi_common_path_open'\'','\''_old_wasi_common_path_readlink'\'','\''_old_wasi_common_path_remove_directory'\'','\''_old_wasi_common_path_rename'\'','\''_old_wasi_common_path_symlink'\'','\''_old_wasi_common_path_unlink_file'\'','\''_old_wasi_common_poll_oneoff'\'','\''_old_wasi_common_proc_exit'\'','\''_old_wasi_common_proc_raise'\'','\''_old_wasi_common_random_get'\'','\''_old_wasi_common_sched_yield'\'','\''_old_wasi_common_sock_recv'\'','\''_old_wasi_common_sock_send'\'','\''_old_wasi_common_sock_shutdown'\''] -o assets/polyfill.html' --cfg has_i128` (signal: 11, SIGSEGV: invalid memory reference)

If you decide to use the fastcomp and Rust stable channel instead, you will be greated by linker errors to do with our usage of u128, etc. Either way, this needs further investigation and potential bug filing in Rust itself.

Clean up of .cargo/config (or an alternative?)

I'm currently using .cargo/config to specify wasm32-unknown-emscripten as the default target, plus pass all the necessary link-args to emcc compiler. This is super messy, so if anyone comes up with a better alternative, please shout out! Another caveat of this approach here, is that the JS + Wasm artifacts land in the assets/ folder.

Dealing with Wasm memory on the WASI syscall/polyfill boundary

The way we have wasi-common designed currently, is that (most of) syscalls accept a combination of &mut WasiCtx and &mut [u8] where the latter is a mutable view at Wasm memory which we use to decode/encode the passed in pointers to and from. The generated C bindings using wasi-common-cbindgen crate the require the syscalls to accept *mut WasiCtx and *mut u8 + usize (memory's address and its length). In order to avoid a lot of changes in the original [wasi.js] glue-code, I'm simply passing in the _entire_ Emscripten memory (HEAP8) to every wasi-common syscall, which certainly carries some overhead. This should be cleaned up one way or another. If you've got any ideas for this, please do shout out! Even if we don't necessarily fix it in this PR, it'll be good to have some discussion about the best approach for later.

Anyhow, my main idea for this is to rewrite translation routines on the JS side to always alloc a contiguous _and_ aligned memory chunks that are big enough to fit the contents located in the WASI Guest heap and required by the syscall to operate on. The tricky bit here is ensuring contiguity and alignment at the same time. I noticed a bit of speed up when I’ve repacked ciovec’s manually this way into a contiguous (compressed if you will) chunk of memory allocated in Emscripten’s heap with only a single _malloc call.

Dealing with Wasi context on the WASI syscall/polyfill boundary

Currently, in this PR, we generate the WasiCtx struct as a thread-local struct in src/main.rs much like it was done in the original [polyfill.c]. In order to pass it as an arg in every WASI syscall in JS, I've exposed an unsafe "getter" fn get_wasi_ctx() -> *mut WasiCtx but I'm wondering if there could be a better way to handle this. One thing I've had in mind was to feature-gate the generated syscalls in wasi-common so that if compiled with js-polyfill feature on (for instance), WasiCtx would be accessed statically. All thoughts on this are much appreciated!

Stream: git-wasmtime

Topic: wasmtime / PR #720 Js-polyfill using wasi-common crate

Wasmtime GitHub notifications bot (Jun 25 2020 at 18:45):

Rustc regression wrt Emscripten in beta/nightly channels

Clean up of .cargo/config (or an alternative?)

Dealing with Wasm memory on the WASI syscall/polyfill boundary

Dealing with Wasi context on the WASI syscall/polyfill boundary

Wasmtime GitHub notifications bot (Feb 22 2024 at 22:27):

Wasmtime GitHub notifications bot (Feb 22 2024 at 22:27):