wasmtime / PR #9744 Support executing Pulley in Wasmtime · git-wasmtime

Stream: git-wasmtime

Topic: wasmtime / PR #9744 Support executing Pulley in Wasmtime

Wasmtime GitHub notifications bot (Dec 05 2024 at 19:54):

alexcrichton opened PR #9744 from alexcrichton:pulley-in-wasmtime to bytecodealliance:main:

This commit is the initial implementation of executing the Pulley
interpreter from the wasmtime crate. This gives access to all of the
wasmtime crate's runtime APIs backed by execution of bytecode in
Pulley. This builds on the previous PRs I've been making for support in
Pulley to culminate in testing on CI in this PR. This PR handles some
final tidbits related to producing a runnable image that can be
interpreted by the wasmtime crate such as:

Pulley compilation artifacts are no longer marked as natively
executable, just read-only.

Pulley compilation artifacts include wasm-to-array trampolines like
normal platforms (removes a pulley special-case).

Dispatch of host calls from Pulley to the Wasmtime runtime are
implemented.

Pulley's list of panicking wasm features is slimmed down as most are
covered by "this lowering isn't supported" errors.

Execution of wasm code now has an if to see whether Pulley is
enabled within a Store or not.

Traps and basic "unwinding" of the pulley stack are implemented (e.g.
a "pulley version" of setjmp and longjmp, sort of)

Halting the interpreter has been refactored to help shrink the size of
ControlFlow<Done> and handle metadata with each done state.

Some minor refactorings are also included here and there along with a
few fixes here and there necessary to get tests passing.

The next major part of this commit is updates to our wast test suite
and executing all *.wast files. Pulley is now executed by default for
all files as a new execution engine. This means that all platforms in CI
will start executing Pulley tests. At this time almost all tests are
flagged as "expected to fail" but there are a small handful of
allow-listed tests which are expected to pass. This exact list will
change over time as CLIF lowerings are implemented and the interpreter
is extended.

Follow-up PRs will extend the testing strategy further such as:

Extending #[wasmtime_test] to include Pulley in addition to Winch.

Getting testing set up on CI for 32-bit platforms.

Wasmtime GitHub notifications bot (Dec 05 2024 at 19:55):

alexcrichton requested fitzgen for a review on PR #9744.

Wasmtime GitHub notifications bot (Dec 05 2024 at 19:55):

alexcrichton commented on PR #9744:

I'll note that this is in draft form as it currently is based on https://github.com/bytecodealliance/wasmtime/pull/9743. I'm going ahead and opening this up to see what CI has to say about this.

Wasmtime GitHub notifications bot (Dec 05 2024 at 20:08):

alexcrichton updated PR #9744.

Wasmtime GitHub notifications bot (Dec 05 2024 at 20:13):

alexcrichton updated PR #9744.

Wasmtime GitHub notifications bot (Dec 05 2024 at 20:33):

alexcrichton updated PR #9744.

Wasmtime GitHub notifications bot (Dec 05 2024 at 21:44):

github-actions[bot] commented on PR #9744:

Label Messager: wasmtime:config

It looks like you are changing Wasmtime's configuration options. Make sure to
complete this check list:

[ ] If you added a new Config method, you wrote extensive documentation for
it.

<details>

Our documentation should be of the following form:

```text
Short, simple summary sentence.

More details. These details can be multiple paragraphs. There should be
information about not just the method, but its parameters and results as
well.

Is this method fallible? If so, when can it return an error?

Can this method panic? If so, when does it panic?

Example

Optional example here.
```

</details>

[ ] If you added a new Config method, or modified an existing one, you
ensured that this configuration is exercised by the fuzz targets.

<details>

For example, if you expose a new strategy for allocating the next instance
slot inside the pooling allocator, you should ensure that at least one of our
fuzz targets exercises that new strategy.

Often, all that is required of you is to ensure that there is a knob for this
configuration option in [wasmtime_fuzzing::Config][fuzzing-config] (or one
of its nested structs).

Rarely, this may require authoring a new fuzz target to specifically test this
configuration. See [our docs on fuzzing][fuzzing-docs] for more details.

</details>

[ ] If you are enabling a configuration option by default, make sure that it
has been fuzzed for at least two weeks before turning it on by default.

[fuzzing-config]: https://github.com/bytecodealliance/wasmtime/blob/ca0e8d0a1d8cefc0496dba2f77a670571d8fdcab/crates/fuzzing/src/generators.rs#L182-L194
[fuzzing-docs]: https://docs.wasmtime.dev/contributing-fuzzing.html

<details>

To modify this label's message, edit the <code>.github/label-messager/wasmtime-config.md</code> file.

To add new label messages or remove existing label messages, edit the
<code>.github/label-messager.json</code> configuration file.

Learn more.

</details>

Wasmtime GitHub notifications bot (Dec 05 2024 at 22:24):

alexcrichton updated PR #9744.

Wasmtime GitHub notifications bot (Dec 05 2024 at 22:24):

alexcrichton has marked PR #9744 as ready for review.

Wasmtime GitHub notifications bot (Dec 05 2024 at 22:24):

alexcrichton requested wasmtime-core-reviewers for a review on PR #9744.

Wasmtime GitHub notifications bot (Dec 05 2024 at 22:24):

alexcrichton requested wasmtime-default-reviewers for a review on PR #9744.

Wasmtime GitHub notifications bot (Dec 05 2024 at 22:24):

alexcrichton requested wasmtime-compiler-reviewers for a review on PR #9744.

Wasmtime GitHub notifications bot (Dec 05 2024 at 22:24):

alexcrichton updated PR #9744.

Wasmtime GitHub notifications bot (Dec 05 2024 at 22:25):

alexcrichton commented on PR #9744:

@fitzgen ok should be rebased and updated, I believe s390x CI is failing but I think most of the rest should be passing. Once results come in I'll figure out how to best ignore s390x for now.

Wasmtime GitHub notifications bot (Dec 05 2024 at 22:54):

alexcrichton updated PR #9744.

Wasmtime GitHub notifications bot (Dec 05 2024 at 23:00):

alexcrichton updated PR #9744.

Wasmtime GitHub notifications bot (Dec 05 2024 at 23:07):

alexcrichton submitted PR review.

Wasmtime GitHub notifications bot (Dec 05 2024 at 23:07):

alexcrichton created PR review comment:

I've added this to https://github.com/bytecodealliance/wasmtime/issues/9747

Wasmtime GitHub notifications bot (Dec 05 2024 at 23:22):

alexcrichton updated PR #9744.

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:33):

fitzgen submitted PR review:

Very nice! Super excited to see this progress!

A few nitpicks and suggestions below, but nothing major

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:33):

fitzgen created PR review comment:

Curious: why was this change needed?

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:33):

fitzgen created PR review comment:

You could use crate::runtime::Uninhabited instead of defining it again :shrug:

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:33):

fitzgen created PR review comment:

Can we #[cfg(feature = "pulley")] this?

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:33):

fitzgen created PR review comment:

Ah okay, nvm about the last comment.

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:33):

fitzgen created PR review comment:

Minor comment addition since this is tricky code and all the other conditions have comments:
        // If the fault was at a location that was not marked as
        // potentially trapping, then its not our problem.
        let Some(trap) = code.lookup_trap_code(text_offset) else {

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:33):

fitzgen created PR review comment:

Nice.

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:33):

fitzgen created PR review comment:

I think this is slightly out-of-sync now that there is a TrapTest enum?

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:33):

fitzgen created PR review comment:

    /// must all be valid for this wasm function call to proceed. For example

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:33):

fitzgen created PR review comment:

        // If this target is Pulley then flag the text section as not needing the

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:33):

fitzgen created PR review comment:

    /// provided in registers according to Pulley's ABI.
    ///
    /// # Unsafety
    ///
    /// All the same unsafety as `call` and additiionally, you must
    /// invoke `call_run` and then `call_end` after calling `call_start`.
    /// If you don't want to wrangle these invocations, use `call` instead
    /// of `call_{start,run,end}`.
    pub unsafe fn call_start<'a>(&'a mut self, args: &[Val]) {

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:33):

fitzgen created PR review comment:

Similarly, nitpick-y regrouping of code and comment so that the comment covers the whole okay-this-really-is-a-wasm-trap path:
        // If all that passed then this is indeed a wasm trap, so return the
        // `jmp_buf` passed to `wasmtime_longjmp` to resume.
        self.set_jit_trap(regs, faulting_addr, trap);
        TrapTest::Trap {

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:33):

fitzgen created PR review comment:

Possible bikeshed: Maybe rename Cranelift to CraneliftNative and Pulley to CraneliftPulley? In addition to clarifying the native code execution bits, this would future proof for a possible WinchPulley in the future.

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:33):

fitzgen created PR review comment:

    /// executed.
    ///
    /// # Unsafety
    ///
    /// In addition to all the invariants documented for `call`, you
    /// may only invoke `call_run` after invoking `call_start` to
    /// initialize this call's arguments.
    pub unsafe fn call_run(&mut self, pc: NonNull<u8>) -> DoneReason<()> {

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:33):

fitzgen created PR review comment:

Maybe we should

impl TestConfit {
    fn custom_page_sizes(&self) -> bool {
        self.custom_page_sizes.unwrap_or(false)
    }

    // etc...
}

if we are doing this dance every time we access these fields

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:33):

fitzgen created PR review comment:

            id: u8,

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:33):

fitzgen created PR review comment:

    /// Peforms the tail end of [`Vm::call`] by returning the values as
    /// determined by `rets` according to Pulley's ABI.
    ///
    /// # Unsafety
    ///
    /// In addition to the invariants documented for `call`, this may
    /// only be called after `call_run`.
    pub unsafe fn call_end<'a>(

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:46):

alexcrichton submitted PR review.

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:46):

alexcrichton created PR review comment:

Ah this is due to some other rule only accepting 32 or 64-bit inputs which causes ISLE to panic as something not a partial constructor. Basically tests panic unless this is here, so a future refactoring will need to flesh out more lowering rules for 8/16-bit types.

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:47):

alexcrichton submitted PR review.

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:47):

alexcrichton created PR review comment:

Thanks I always forget what we have, where it's defined, and what it's called heh.

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:58):

alexcrichton updated PR #9744.

Wasmtime GitHub notifications bot (Dec 06 2024 at 19:58):

alexcrichton has enabled auto merge for PR #9744.

Wasmtime GitHub notifications bot (Dec 06 2024 at 20:06):

alexcrichton requested wasmtime-fuzz-reviewers for a review on PR #9744.

Wasmtime GitHub notifications bot (Dec 06 2024 at 20:06):

alexcrichton updated PR #9744.

Wasmtime GitHub notifications bot (Dec 06 2024 at 20:40):

alexcrichton merged PR #9744.

Last updated: Dec 23 2024 at 12:05 UTC