Stream: git-wasmtime

Topic: wasmtime / issue #6370 Land WASI Preview 2 support into W...

view this post on Zulip Wasmtime GitHub notifications bot (May 11 2023 at 00:33):

pchickey opened issue #6370:

We have been working on a prototype of what WASI Preview 2 support will look like in Wasmtime for 7 months now! https://github.com/bytecodealliance/preview2-prototyping/

The work is not yet totally complete, but I believe that it is mature enough that we should start landing it into this repository, and keep iterating on it here. It was included in the March 28 tooling compatibility matrix, but it turns out we skipped compatibility with the wasmtime 8.0 release due to some bindgen changes we needed. I'd like to get everything landed and part of the regular Wasmtime release cycle for 10.0, which will cut at the beginning of June.

Here is my rough plan for how we will land changes. These plans are open to changes so please leave your feedback!

  1. [ ] Move the adapter (found at the root of the p2p repo) to crates/wasi-preview1-component-adapter. We will bring its child proc-macro crate byte-array, and the test harness validate, in as well underneath that directory. Add building, validating, and publishing the adapter binary (for both commands and reactors) to the wasmtime repository's CI. Runtime (integration) tests will be added in the subsequent steps.
  2. [ ] Restructure the test-programs crate to support building both modules and components (using the adapter from step 1). Keep all the existing tests of wasi-common (which covers both the cap-std-sync and tokio backends) intact, but refactor like we did in p2p to make the structure of how tests are run driven by hand-written code, rather than packing a ton of magic into the macros. The components will be built, but not actually executed yet.
  3. [ ] Land the contents of preview2-prototyping's wasi-common crate under wasmtime-wasi in the pub mod preview2 space, and behind the (enabled by default) preview2 cargo feature. Leave this crate's wasi-common, wasi-cap-std-sync, and wasi-tokio crates untouched. They will continue to provide all preview 1 functionality for the moment. The preview 2 functionality will be passing all tests that it does in p2p through the test-programs (at the moment this means poll-oneoff and some trailing slashes behavior is broken), and this PR will port in the other tests from the p2p repo as well.

At this point we can say the wasmtime project has preview 2 support! It is, however, behind a different WasiCtx api (with some changes to the builder, and the way to add it to a linker), and is totally separate from the preview 1 support.

The next steps are:

  1. [ ] Finish the development work I have planned to straighten out the poll-oneoff / streams / async vs sync embedding design in preview 2, which is currently not passing all its tests. At this point, we can just ignore some trailing slash behavior in the test suite, but the stuff everyone really cares about ought to be done.
  2. [ ] Land @rvolosatovs 's work on a host-side preview 2 wit-bindgen to preview 1 wiggle adapter. Hook this up to the test suite, passing all the same tests as the legacy implementation, without using the wasi-preview1-component-adapter or any component-model features in wasmtime.
  3. [ ] Land preview2-prototyping's wasmtime-wasi-sockets crate into this repository. This crate is separate from, but depends upon, what will land in wasmtime-wasi::preview2. It need to be integrated with the poll oneoff design fixes, and it needs a test suite, in order to land.
  4. [ ] Port the wasmtime-wasi-http crate to use the poll-oneoff fixes above, making it compatible with components instead of just modules

Once the poll design changes and host-side adapter are landed, we can start working on transitioning the legacy implementation out of wasmtime:

  1. [ ] Move all of wasmtime-wasi's existing preview 1 support behind pub mod preview1_legacy and the off-by-default preview1_legacy cargo feature. Move the preview2 module out to the root. The new host-side adapter interface to provides preview 1 support to all users, except those who have regressions and need to use the legacy support for a limited time.
  2. [ ] The known regressions for preview 2 support include some trailing slash filesystem behaviors, and the support for preview 1's late-added sock_* functions. Presently, the sock functions have no tests in the tree, or anywhere else I can find, so I don't have a way to port those to preview 2 without taking a wild guess at what semantics they need. The only known user of these functions was Enarx, who are no longer in business. If any other users need these sock functions to keep working in either the host or wasm adapter for preview 1, we'll need them to contribute to that support- all of the Bytecode Alliance contributors I have spoke to so far don't have a business case for supporting preview 1's sockets, and instead are all in on preview 2 sockets.
  3. [ ] After 2 releases in off-by-default legacy mode, we will delete the legacy preview 1 code from the repository.

view this post on Zulip Wasmtime GitHub notifications bot (May 16 2023 at 00:05):

pchickey edited issue #6370:

We have been working on a prototype of what WASI Preview 2 support will look like in Wasmtime for 7 months now! https://github.com/bytecodealliance/preview2-prototyping/

The work is not yet totally complete, but I believe that it is mature enough that we should start landing it into this repository, and keep iterating on it here. It was included in the March 28 tooling compatibility matrix, but it turns out we skipped compatibility with the wasmtime 8.0 release due to some bindgen changes we needed. I'd like to get everything landed and part of the regular Wasmtime release cycle for 10.0, which will cut at the beginning of June.

Here is my rough plan for how we will land changes. These plans are open to changes so please leave your feedback!

  1. [x] Move the adapter (found at the root of the p2p repo) to crates/wasi-preview1-component-adapter. We will bring its child proc-macro crate byte-array, and the test harness validate, in as well underneath that directory. Add building, validating, and publishing the adapter binary (for both commands and reactors) to the wasmtime repository's CI. Runtime (integration) tests will be added in the subsequent steps.
  2. [ ] Restructure the test-programs crate to support building both modules and components (using the adapter from step 1). Keep all the existing tests of wasi-common (which covers both the cap-std-sync and tokio backends) intact, but refactor like we did in p2p to make the structure of how tests are run driven by hand-written code, rather than packing a ton of magic into the macros. The components will be built, but not actually executed yet.
  3. [ ] Land the contents of preview2-prototyping's wasi-common crate under wasmtime-wasi in the pub mod preview2 space, and behind the (enabled by default) preview2 cargo feature. Leave this crate's wasi-common, wasi-cap-std-sync, and wasi-tokio crates untouched. They will continue to provide all preview 1 functionality for the moment. The preview 2 functionality will be passing all tests that it does in p2p through the test-programs (at the moment this means poll-oneoff and some trailing slashes behavior is broken), and this PR will port in the other tests from the p2p repo as well.

At this point we can say the wasmtime project has preview 2 support! It is, however, behind a different WasiCtx api (with some changes to the builder, and the way to add it to a linker), and is totally separate from the preview 1 support.

The next steps are:

  1. [ ] Finish the development work I have planned to straighten out the poll-oneoff / streams / async vs sync embedding design in preview 2, which is currently not passing all its tests. At this point, we can just ignore some trailing slash behavior in the test suite, but the stuff everyone really cares about ought to be done.
  2. [ ] Land @rvolosatovs 's work on a host-side preview 2 wit-bindgen to preview 1 wiggle adapter. Hook this up to the test suite, passing all the same tests as the legacy implementation, without using the wasi-preview1-component-adapter or any component-model features in wasmtime.
  3. [ ] Land preview2-prototyping's wasmtime-wasi-sockets crate into this repository. This crate is separate from, but depends upon, what will land in wasmtime-wasi::preview2. It need to be integrated with the poll oneoff design fixes, and it needs a test suite, in order to land.
  4. [ ] Port the wasmtime-wasi-http crate to use the poll-oneoff fixes above, making it compatible with components instead of just modules

Once the poll design changes and host-side adapter are landed, we can start working on transitioning the legacy implementation out of wasmtime:

  1. [ ] Move all of wasmtime-wasi's existing preview 1 support behind pub mod preview1_legacy and the off-by-default preview1_legacy cargo feature. Move the preview2 module out to the root. The new host-side adapter interface to provides preview 1 support to all users, except those who have regressions and need to use the legacy support for a limited time.
  2. [ ] The known regressions for preview 2 support include some trailing slash filesystem behaviors, and the support for preview 1's late-added sock_* functions. Presently, the sock functions have no tests in the tree, or anywhere else I can find, so I don't have a way to port those to preview 2 without taking a wild guess at what semantics they need. The only known user of these functions was Enarx, who are no longer in business. If any other users need these sock functions to keep working in either the host or wasm adapter for preview 1, we'll need them to contribute to that support- all of the Bytecode Alliance contributors I have spoke to so far don't have a business case for supporting preview 1's sockets, and instead are all in on preview 2 sockets.
  3. [ ] After 2 releases in off-by-default legacy mode, we will delete the legacy preview 1 code from the repository.

view this post on Zulip Wasmtime GitHub notifications bot (May 17 2023 at 20:24):

pchickey edited issue #6370:

We have been working on a prototype of what WASI Preview 2 support will look like in Wasmtime for 7 months now! https://github.com/bytecodealliance/preview2-prototyping/

The work is not yet totally complete, but I believe that it is mature enough that we should start landing it into this repository, and keep iterating on it here. It was included in the March 28 tooling compatibility matrix, but it turns out we skipped compatibility with the wasmtime 8.0 release due to some bindgen changes we needed. I'd like to get everything landed and part of the regular Wasmtime release cycle for 10.0, which will cut at the beginning of June.

Here is my rough plan for how we will land changes. These plans are open to changes so please leave your feedback!

  1. [x] Move the adapter (found at the root of the p2p repo) to crates/wasi-preview1-component-adapter. We will bring its child proc-macro crate byte-array, and the test harness validate, in as well underneath that directory. Add building, validating, and publishing the adapter binary (for both commands and reactors) to the wasmtime repository's CI. Runtime (integration) tests will be added in the subsequent steps.
  2. [x] Restructure the test-programs crate to support building both modules and components (using the adapter from step 1). Keep all the existing tests of wasi-common (which covers both the cap-std-sync and tokio backends) intact, but refactor like we did in p2p to make the structure of how tests are run driven by hand-written code, rather than packing a ton of magic into the macros. The components will be built, but not actually executed yet.
  3. [ ] Land the contents of preview2-prototyping's wasi-common crate under wasmtime-wasi in the pub mod preview2 space, and behind the (enabled by default) preview2 cargo feature. Leave this crate's wasi-common, wasi-cap-std-sync, and wasi-tokio crates untouched. They will continue to provide all preview 1 functionality for the moment. The preview 2 functionality will be passing all tests that it does in p2p through the test-programs (at the moment this means poll-oneoff and some trailing slashes behavior is broken), and this PR will port in the other tests from the p2p repo as well.

At this point we can say the wasmtime project has preview 2 support! It is, however, behind a different WasiCtx api (with some changes to the builder, and the way to add it to a linker), and is totally separate from the preview 1 support.

The next steps are:

  1. [ ] Finish the development work I have planned to straighten out the poll-oneoff / streams / async vs sync embedding design in preview 2, which is currently not passing all its tests. At this point, we can just ignore some trailing slash behavior in the test suite, but the stuff everyone really cares about ought to be done.
  2. [ ] Land @rvolosatovs 's work on a host-side preview 2 wit-bindgen to preview 1 wiggle adapter. Hook this up to the test suite, passing all the same tests as the legacy implementation, without using the wasi-preview1-component-adapter or any component-model features in wasmtime.
  3. [ ] Land preview2-prototyping's wasmtime-wasi-sockets crate into this repository. This crate is separate from, but depends upon, what will land in wasmtime-wasi::preview2. It need to be integrated with the poll oneoff design fixes, and it needs a test suite, in order to land.
  4. [ ] Port the wasmtime-wasi-http crate to use the poll-oneoff fixes above, making it compatible with components instead of just modules

Once the poll design changes and host-side adapter are landed, we can start working on transitioning the legacy implementation out of wasmtime:

  1. [ ] Move all of wasmtime-wasi's existing preview 1 support behind pub mod preview1_legacy and the off-by-default preview1_legacy cargo feature. Move the preview2 module out to the root. The new host-side adapter interface to provides preview 1 support to all users, except those who have regressions and need to use the legacy support for a limited time.
  2. [ ] The known regressions for preview 2 support include some trailing slash filesystem behaviors, and the support for preview 1's late-added sock_* functions. Presently, the sock functions have no tests in the tree, or anywhere else I can find, so I don't have a way to port those to preview 2 without taking a wild guess at what semantics they need. The only known user of these functions was Enarx, who are no longer in business. If any other users need these sock functions to keep working in either the host or wasm adapter for preview 1, we'll need them to contribute to that support- all of the Bytecode Alliance contributors I have spoke to so far don't have a business case for supporting preview 1's sockets, and instead are all in on preview 2 sockets.
  3. [ ] After 2 releases in off-by-default legacy mode, we will delete the legacy preview 1 code from the repository.

view this post on Zulip Wasmtime GitHub notifications bot (May 18 2023 at 18:37):

pchickey edited issue #6370:

We have been working on a prototype of what WASI Preview 2 support will look like in Wasmtime for 7 months now! https://github.com/bytecodealliance/preview2-prototyping/

The work is not yet totally complete, but I believe that it is mature enough that we should start landing it into this repository, and keep iterating on it here. It was included in the March 28 tooling compatibility matrix, but it turns out we skipped compatibility with the wasmtime 8.0 release due to some bindgen changes we needed. I'd like to get everything landed and part of the regular Wasmtime release cycle for 10.0, which will cut at the beginning of June.

Here is my rough plan for how we will land changes. These plans are open to changes so please leave your feedback!

  1. [x] Move the adapter (found at the root of the p2p repo) to crates/wasi-preview1-component-adapter. We will bring its child proc-macro crate byte-array, and the test harness validate, in as well underneath that directory. Add building, validating, and publishing the adapter binary (for both commands and reactors) to the wasmtime repository's CI. Runtime (integration) tests will be added in the subsequent steps.
  2. [x] Restructure the test-programs crate to support building both modules and components (using the adapter from step 1). Keep all the existing tests of wasi-common (which covers both the cap-std-sync and tokio backends) intact, but refactor like we did in p2p to make the structure of how tests are run driven by hand-written code, rather than packing a ton of magic into the macros. The components will be built, but not actually executed yet.
  3. [ ] Land the contents of preview2-prototyping's wasi-common crate under wasmtime-wasi in the pub mod preview2 space, and behind the (enabled by default) preview2 cargo feature. Leave this repository's wasi-common, wasi-cap-std-sync, and wasi-tokio crates untouched. They will continue to provide all preview 1 functionality for the moment. The preview 2 functionality will be passing all tests that it does in p2p through the test-programs (at the moment this means poll-oneoff and some trailing slashes behavior is broken), and this PR will port in the other tests from the p2p repo as well.

At this point we can say the wasmtime project has preview 2 support! It is, however, behind a different WasiCtx api (with some changes to the builder, and the way to add it to a linker), and is totally separate from the preview 1 support.

The next steps are:

  1. [ ] Finish the development work I have planned to straighten out the poll-oneoff / streams / async vs sync embedding design in preview 2, which is currently not passing all its tests. At this point, we can just ignore some trailing slash behavior in the test suite, but the stuff everyone really cares about ought to be done.
  2. [ ] Land @rvolosatovs 's work on a host-side preview 2 wit-bindgen to preview 1 wiggle adapter. Hook this up to the test suite, passing all the same tests as the legacy implementation, without using the wasi-preview1-component-adapter or any component-model features in wasmtime.
  3. [ ] Land preview2-prototyping's wasmtime-wasi-sockets crate into this repository. This crate is separate from, but depends upon, what will land in wasmtime-wasi::preview2. It need to be integrated with the poll oneoff design fixes, and it needs a test suite, in order to land.
  4. [ ] Port the wasmtime-wasi-http crate to use the poll-oneoff fixes above, making it compatible with components instead of just modules

Once the poll design changes and host-side adapter are landed, we can start working on transitioning the legacy implementation out of wasmtime:

  1. [ ] Move all of wasmtime-wasi's existing preview 1 support behind pub mod preview1_legacy and the off-by-default preview1_legacy cargo feature. Move the preview2 module out to the root. The new host-side adapter interface to provides preview 1 support to all users, except those who have regressions and need to use the legacy support for a limited time.
  2. [ ] The known regressions for preview 2 support include some trailing slash filesystem behaviors, and the support for preview 1's late-added sock_* functions. Presently, the sock functions have no tests in the tree, or anywhere else I can find, so I don't have a way to port those to preview 2 without taking a wild guess at what semantics they need. The only known user of these functions was Enarx, who are no longer in business. If any other users need these sock functions to keep working in either the host or wasm adapter for preview 1, we'll need them to contribute to that support- all of the Bytecode Alliance contributors I have spoke to so far don't have a business case for supporting preview 1's sockets, and instead are all in on preview 2 sockets.
  3. [ ] After 2 releases in off-by-default legacy mode, we will delete the legacy preview 1 code from the repository. This means retiring the wasi-common, wasi-cap-std-sync, and wasi-tokio crates. We could publish a final empty version indicating they wont get any more updates and have been replaced with wasmtime-wasi.

view this post on Zulip Wasmtime GitHub notifications bot (May 19 2023 at 01:50):

pchickey edited issue #6370:

We have been working on a prototype of what WASI Preview 2 support will look like in Wasmtime for 7 months now! https://github.com/bytecodealliance/preview2-prototyping/

The work is not yet totally complete, but I believe that it is mature enough that we should start landing it into this repository, and keep iterating on it here. It was included in the March 28 tooling compatibility matrix, but it turns out we skipped compatibility with the wasmtime 8.0 release due to some bindgen changes we needed. I'd like to get everything landed and part of the regular Wasmtime release cycle for 10.0, which will cut at the beginning of June.

Here is my rough plan for how we will land changes. These plans are open to changes so please leave your feedback!

  1. [x] Move the adapter (found at the root of the p2p repo) to crates/wasi-preview1-component-adapter. We will bring its child proc-macro crate byte-array, and the test harness validate, in as well underneath that directory. Add building, validating, and publishing the adapter binary (for both commands and reactors) to the wasmtime repository's CI. Runtime (integration) tests will be added in the subsequent steps.
  2. [x] Restructure the test-programs crate to support building both modules and components (using the adapter from step 1). Keep all the existing tests of wasi-common (which covers both the cap-std-sync and tokio backends) intact, but refactor like we did in p2p to make the structure of how tests are run driven by hand-written code, rather than packing a ton of magic into the macros. The components will be built, but not actually executed yet.
  3. [x] Land the contents of preview2-prototyping's wasi-common crate under wasmtime-wasi in the pub mod preview2 space, and behind the (enabled by default) preview2 cargo feature. Leave this repository's wasi-common, wasi-cap-std-sync, and wasi-tokio crates untouched. They will continue to provide all preview 1 functionality for the moment. The preview 2 functionality will be passing all tests that it does in p2p through the test-programs (at the moment this means poll-oneoff and some trailing slashes behavior is broken), and this PR will port in the other tests from the p2p repo as well.

At this point we can say the wasmtime project has preview 2 support! It is, however, behind a different WasiCtx api (with some changes to the builder, and the way to add it to a linker), and is totally separate from the preview 1 support.

The next steps are:

  1. [ ] Finish the development work I have planned to straighten out the poll-oneoff / streams / async vs sync embedding design in preview 2, which is currently not passing all its tests. At this point, we can just ignore some trailing slash behavior in the test suite, but the stuff everyone really cares about ought to be done.
  2. [ ] Land @rvolosatovs 's work on a host-side preview 2 wit-bindgen to preview 1 wiggle adapter. Hook this up to the test suite, passing all the same tests as the legacy implementation, without using the wasi-preview1-component-adapter or any component-model features in wasmtime.
  3. [ ] Land preview2-prototyping's wasmtime-wasi-sockets crate into this repository. This crate is separate from, but depends upon, what will land in wasmtime-wasi::preview2. It need to be integrated with the poll oneoff design fixes, and it needs a test suite, in order to land.
  4. [ ] Port the wasmtime-wasi-http crate to use the poll-oneoff fixes above, making it compatible with components instead of just modules

Once the poll design changes and host-side adapter are landed, we can start working on transitioning the legacy implementation out of wasmtime:

  1. [ ] Move all of wasmtime-wasi's existing preview 1 support behind pub mod preview1_legacy and the off-by-default preview1_legacy cargo feature. Move the preview2 module out to the root. The new host-side adapter interface to provides preview 1 support to all users, except those who have regressions and need to use the legacy support for a limited time.
  2. [ ] The known regressions for preview 2 support include some trailing slash filesystem behaviors, and the support for preview 1's late-added sock_* functions. Presently, the sock functions have no tests in the tree, or anywhere else I can find, so I don't have a way to port those to preview 2 without taking a wild guess at what semantics they need. The only known user of these functions was Enarx, who are no longer in business. If any other users need these sock functions to keep working in either the host or wasm adapter for preview 1, we'll need them to contribute to that support- all of the Bytecode Alliance contributors I have spoke to so far don't have a business case for supporting preview 1's sockets, and instead are all in on preview 2 sockets.
  3. [ ] After 2 releases in off-by-default legacy mode, we will delete the legacy preview 1 code from the repository. This means retiring the wasi-common, wasi-cap-std-sync, and wasi-tokio crates. We could publish a final empty version indicating they wont get any more updates and have been replaced with wasmtime-wasi.

view this post on Zulip Wasmtime GitHub notifications bot (May 19 2023 at 01:54):

pchickey edited issue #6370:

We have been working on a prototype of what WASI Preview 2 support will look like in Wasmtime for 7 months now! https://github.com/bytecodealliance/preview2-prototyping/

The work is not yet totally complete, but I believe that it is mature enough that we should start landing it into this repository, and keep iterating on it here. It was included in the March 28 tooling compatibility matrix, but it turns out we skipped compatibility with the wasmtime 8.0 release due to some bindgen changes we needed. I'd like to get everything landed and part of the regular Wasmtime release cycle for 10.0, which will cut at the beginning of June.

Here is my rough plan for how we will land changes. These plans are open to changes so please leave your feedback!

  1. [x] Move the adapter (found at the root of the p2p repo) to crates/wasi-preview1-component-adapter. We will bring its child proc-macro crate byte-array, and the test harness validate, in as well underneath that directory. Add building, validating, and publishing the adapter binary (for both commands and reactors) to the wasmtime repository's CI. Runtime (integration) tests will be added in the subsequent steps.
  2. [x] Restructure the test-programs crate to support building both modules and components (using the adapter from step 1). Keep all the existing tests of wasi-common (which covers both the cap-std-sync and tokio backends) intact, but refactor like we did in p2p to make the structure of how tests are run driven by hand-written code, rather than packing a ton of magic into the macros. The components will be built, but not actually executed yet.
  3. [x] Land the contents of preview2-prototyping's wasi-common crate under wasmtime-wasi in the pub mod preview2 space, and behind the (enabled by default) preview2 cargo feature. Leave this repository's wasi-common, wasi-cap-std-sync, and wasi-tokio crates untouched. They will continue to provide all preview 1 functionality for the moment. The preview 2 functionality will be passing all tests that it does in p2p through the test-programs (at the moment this means poll-oneoff and some trailing slashes behavior is broken), and this PR will port in the other tests from the p2p repo as well.

At this point we can say the wasmtime project has preview 2 support! It is, however, behind a different WasiCtx api (with some changes to the builder, and the way to add it to a linker), and is totally separate from the preview 1 support.

The next steps are:

  1. [ ] Finish the development work I have planned to straighten out the poll-oneoff / streams / async vs sync embedding design in preview 2, which is currently not passing all its tests. At this point, we can just ignore some trailing slash behavior in the test suite, but the stuff everyone really cares about ought to be done.
  2. [ ] Land @rvolosatovs 's work on a host-side preview 2 wit-bindgen to preview 1 wiggle adapter. Hook this up to the test suite, passing all the same tests as the legacy implementation, without using the wasi-preview1-component-adapter or any component-model features in wasmtime.
  3. [ ] Land preview2-prototyping's wasmtime-wasi-sockets crate into this repository. This crate is separate from, but depends upon, what will land in wasmtime-wasi::preview2. It need to be integrated with the poll oneoff design fixes, and it needs a test suite, in order to land.
  4. [ ] Port the wasmtime-wasi-http crate to use the poll-oneoff fixes above, making it compatible with components instead of just modules]
  5. [ ] Bring wasmtime-wasi::preview2's docs up to Wasmtime project standards.

Once the poll design changes and host-side adapter are landed, we can start working on transitioning the legacy implementation out of wasmtime:

  1. [ ] Move all of wasmtime-wasi's existing preview 1 support behind pub mod preview1_legacy and the off-by-default preview1_legacy cargo feature. Move the preview2 module out to the root. The new host-side adapter interface to provides preview 1 support to all users, except those who have regressions and need to use the legacy support for a limited time.
  2. [ ] The known regressions for preview 2 support include some trailing slash filesystem behaviors, and the support for preview 1's late-added sock_* functions. Presently, the sock functions have no tests in the tree, or anywhere else I can find, so I don't have a way to port those to preview 2 without taking a wild guess at what semantics they need. The only known user of these functions was Enarx, who are no longer in business. If any other users need these sock functions to keep working in either the host or wasm adapter for preview 1, we'll need them to contribute to that support- all of the Bytecode Alliance contributors I have spoke to so far don't have a business case for supporting preview 1's sockets, and instead are all in on preview 2 sockets.
  3. [ ] After 2 releases in off-by-default legacy mode, we will delete the legacy preview 1 code from the repository. This means retiring the wasi-common, wasi-cap-std-sync, and wasi-tokio crates. We could publish a final empty version indicating they wont get any more updates and have been replaced with wasmtime-wasi.

view this post on Zulip Wasmtime GitHub notifications bot (May 30 2023 at 18:17):

pchickey edited issue #6370:

We have been working on a prototype of what WASI Preview 2 support will look like in Wasmtime for 7 months now! https://github.com/bytecodealliance/preview2-prototyping/

The work is not yet totally complete, but I believe that it is mature enough that we should start landing it into this repository, and keep iterating on it here. It was included in the March 28 tooling compatibility matrix, but it turns out we skipped compatibility with the wasmtime 8.0 release due to some bindgen changes we needed. I'd like to get everything landed and part of the regular Wasmtime release cycle for 10.0, which will cut at the beginning of June.

Here is my rough plan for how we will land changes. These plans are open to changes so please leave your feedback!

  1. [x] Move the adapter (found at the root of the p2p repo) to crates/wasi-preview1-component-adapter. We will bring its child proc-macro crate byte-array, and the test harness validate, in as well underneath that directory. Add building, validating, and publishing the adapter binary (for both commands and reactors) to the wasmtime repository's CI. Runtime (integration) tests will be added in the subsequent steps.
  2. [x] Restructure the test-programs crate to support building both modules and components (using the adapter from step 1). Keep all the existing tests of wasi-common (which covers both the cap-std-sync and tokio backends) intact, but refactor like we did in p2p to make the structure of how tests are run driven by hand-written code, rather than packing a ton of magic into the macros. The components will be built, but not actually executed yet.
  3. [x] Land the contents of preview2-prototyping's wasi-common crate under wasmtime-wasi in the pub mod preview2 space, and behind the (enabled by default) preview2 cargo feature. Leave this repository's wasi-common, wasi-cap-std-sync, and wasi-tokio crates untouched. They will continue to provide all preview 1 functionality for the moment. The preview 2 functionality will be passing all tests that it does in p2p through the test-programs (at the moment this means poll-oneoff and some trailing slashes behavior is broken), and this PR will port in the other tests from the p2p repo as well.

At this point we can say the wasmtime project has preview 2 support! It is, however, behind a different WasiCtx api (with some changes to the builder, and the way to add it to a linker), and is totally separate from the preview 1 support.

The next steps are:

  1. [ ] Finish the development work I have planned to straighten out the poll-oneoff / streams / async vs sync embedding design in preview 2, which is currently not passing all its tests. At this point, we can just ignore some trailing slash behavior in the test suite, but the stuff everyone really cares about ought to be done.
  2. [x] Land @rvolosatovs 's work on a host-side preview 2 wit-bindgen to preview 1 wiggle adapter. Hook this up to the test suite, passing all the same tests as the legacy implementation, without using the wasi-preview1-component-adapter or any component-model features in wasmtime.
  3. [ ] Land preview2-prototyping's wasmtime-wasi-sockets crate into this repository. This crate is separate from, but depends upon, what will land in wasmtime-wasi::preview2. It need to be integrated with the poll oneoff design fixes, and it needs a test suite, in order to land.
  4. [ ] Port the wasmtime-wasi-http crate to use the poll-oneoff fixes above, making it compatible with components instead of just modules]
  5. [ ] Bring wasmtime-wasi::preview2's docs up to Wasmtime project standards.

Once the poll design changes and host-side adapter are landed, we can start working on transitioning the legacy implementation out of wasmtime:

  1. [ ] Move all of wasmtime-wasi's existing preview 1 support behind pub mod preview1_legacy and the off-by-default preview1_legacy cargo feature. Move the preview2 module out to the root. The new host-side adapter interface to provides preview 1 support to all users, except those who have regressions and need to use the legacy support for a limited time.
  2. [ ] The known regressions for preview 2 support include some trailing slash filesystem behaviors, and the support for preview 1's late-added sock_* functions. Presently, the sock functions have no tests in the tree, or anywhere else I can find, so I don't have a way to port those to preview 2 without taking a wild guess at what semantics they need. The only known user of these functions was Enarx, who are no longer in business. If any other users need these sock functions to keep working in either the host or wasm adapter for preview 1, we'll need them to contribute to that support- all of the Bytecode Alliance contributors I have spoke to so far don't have a business case for supporting preview 1's sockets, and instead are all in on preview 2 sockets.
  3. [ ] After 2 releases in off-by-default legacy mode, we will delete the legacy preview 1 code from the repository. This means retiring the wasi-common, wasi-cap-std-sync, and wasi-tokio crates. We could publish a final empty version indicating they wont get any more updates and have been replaced with wasmtime-wasi.

view this post on Zulip Wasmtime GitHub notifications bot (Jul 26 2023 at 14:45):

pchickey edited issue #6370:

We have been working on a prototype of what WASI Preview 2 support will look like in Wasmtime for 7 months now! https://github.com/bytecodealliance/preview2-prototyping/

The work is not yet totally complete, but I believe that it is mature enough that we should start landing it into this repository, and keep iterating on it here. It was included in the March 28 tooling compatibility matrix, but it turns out we skipped compatibility with the wasmtime 8.0 release due to some bindgen changes we needed. I'd like to get everything landed and part of the regular Wasmtime release cycle for 10.0, which will cut at the beginning of June.

Here is my rough plan for how we will land changes. These plans are open to changes so please leave your feedback!

  1. [x] Move the adapter (found at the root of the p2p repo) to crates/wasi-preview1-component-adapter. We will bring its child proc-macro crate byte-array, and the test harness validate, in as well underneath that directory. Add building, validating, and publishing the adapter binary (for both commands and reactors) to the wasmtime repository's CI. Runtime (integration) tests will be added in the subsequent steps.
  2. [x] Restructure the test-programs crate to support building both modules and components (using the adapter from step 1). Keep all the existing tests of wasi-common (which covers both the cap-std-sync and tokio backends) intact, but refactor like we did in p2p to make the structure of how tests are run driven by hand-written code, rather than packing a ton of magic into the macros. The components will be built, but not actually executed yet.
  3. [x] Land the contents of preview2-prototyping's wasi-common crate under wasmtime-wasi in the pub mod preview2 space, and behind the (enabled by default) preview2 cargo feature. Leave this repository's wasi-common, wasi-cap-std-sync, and wasi-tokio crates untouched. They will continue to provide all preview 1 functionality for the moment. The preview 2 functionality will be passing all tests that it does in p2p through the test-programs (at the moment this means poll-oneoff and some trailing slashes behavior is broken), and this PR will port in the other tests from the p2p repo as well.

At this point we can say the wasmtime project has preview 2 support! It is, however, behind a different WasiCtx api (with some changes to the builder, and the way to add it to a linker), and is totally separate from the preview 1 support.

The next steps are:

  1. [x] Finish the development work I have planned to straighten out the poll-oneoff / streams / async vs sync embedding design in preview 2, which is currently not passing all its tests. At this point, we can just ignore some trailing slash behavior in the test suite, but the stuff everyone really cares about ought to be done.
  2. [x] Land @rvolosatovs 's work on a host-side preview 2 wit-bindgen to preview 1 wiggle adapter. Hook this up to the test suite, passing all the same tests as the legacy implementation, without using the wasi-preview1-component-adapter or any component-model features in wasmtime.
  3. [ ] Land preview2-prototyping's wasmtime-wasi-sockets crate into this repository. This crate is separate from, but depends upon, what will land in wasmtime-wasi::preview2. It need to be integrated with the poll oneoff design fixes, and it needs a test suite, in order to land.
  4. [ ] Port the wasmtime-wasi-http crate to use the poll-oneoff fixes above, making it compatible with components instead of just modules]
  5. [ ] Bring wasmtime-wasi::preview2's docs up to Wasmtime project standards.

Once the poll design changes and host-side adapter are landed, we can start working on transitioning the legacy implementation out of wasmtime:

  1. [ ] Move all of wasmtime-wasi's existing preview 1 support behind pub mod preview1_legacy and the off-by-default preview1_legacy cargo feature. Move the preview2 module out to the root. The new host-side adapter interface to provides preview 1 support to all users, except those who have regressions and need to use the legacy support for a limited time.
  2. [ ] The known regressions for preview 2 support include some trailing slash filesystem behaviors, and the support for preview 1's late-added sock_* functions. Presently, the sock functions have no tests in the tree, or anywhere else I can find, so I don't have a way to port those to preview 2 without taking a wild guess at what semantics they need. The only known user of these functions was Enarx, who are no longer in business. If any other users need these sock functions to keep working in either the host or wasm adapter for preview 1, we'll need them to contribute to that support- all of the Bytecode Alliance contributors I have spoke to so far don't have a business case for supporting preview 1's sockets, and instead are all in on preview 2 sockets.
  3. [ ] After 2 releases in off-by-default legacy mode, we will delete the legacy preview 1 code from the repository. This means retiring the wasi-common, wasi-cap-std-sync, and wasi-tokio crates. We could publish a final empty version indicating they wont get any more updates and have been replaced with wasmtime-wasi.

view this post on Zulip Wasmtime GitHub notifications bot (Sep 27 2023 at 22:48):

pchickey edited issue #6370:

We have been working on a prototype of what WASI Preview 2 support will look like in Wasmtime for 7 months now! https://github.com/bytecodealliance/preview2-prototyping/

The work is not yet totally complete, but I believe that it is mature enough that we should start landing it into this repository, and keep iterating on it here. It was included in the March 28 tooling compatibility matrix, but it turns out we skipped compatibility with the wasmtime 8.0 release due to some bindgen changes we needed. I'd like to get everything landed and part of the regular Wasmtime release cycle for 10.0, which will cut at the beginning of June.

Here is my rough plan for how we will land changes. These plans are open to changes so please leave your feedback!

  1. [x] Move the adapter (found at the root of the p2p repo) to crates/wasi-preview1-component-adapter. We will bring its child proc-macro crate byte-array, and the test harness validate, in as well underneath that directory. Add building, validating, and publishing the adapter binary (for both commands and reactors) to the wasmtime repository's CI. Runtime (integration) tests will be added in the subsequent steps.
  2. [x] Restructure the test-programs crate to support building both modules and components (using the adapter from step 1). Keep all the existing tests of wasi-common (which covers both the cap-std-sync and tokio backends) intact, but refactor like we did in p2p to make the structure of how tests are run driven by hand-written code, rather than packing a ton of magic into the macros. The components will be built, but not actually executed yet.
  3. [x] Land the contents of preview2-prototyping's wasi-common crate under wasmtime-wasi in the pub mod preview2 space, and behind the (enabled by default) preview2 cargo feature. Leave this repository's wasi-common, wasi-cap-std-sync, and wasi-tokio crates untouched. They will continue to provide all preview 1 functionality for the moment. The preview 2 functionality will be passing all tests that it does in p2p through the test-programs (at the moment this means poll-oneoff and some trailing slashes behavior is broken), and this PR will port in the other tests from the p2p repo as well.

At this point we can say the wasmtime project has preview 2 support! It is, however, behind a different WasiCtx api (with some changes to the builder, and the way to add it to a linker), and is totally separate from the preview 1 support.

The next steps are:

  1. [x] Finish the development work I have planned to straighten out the poll-oneoff / streams / async vs sync embedding design in preview 2, which is currently not passing all its tests. At this point, we can just ignore some trailing slash behavior in the test suite, but the stuff everyone really cares about ought to be done.
  2. [x] Land @rvolosatovs 's work on a host-side preview 2 wit-bindgen to preview 1 wiggle adapter. Hook this up to the test suite, passing all the same tests as the legacy implementation, without using the wasi-preview1-component-adapter or any component-model features in wasmtime.
  3. [x] Land preview2-prototyping's wasmtime-wasi-sockets crate into this repository. This crate is separate from, but depends upon, what will land in wasmtime-wasi::preview2. It need to be integrated with the poll oneoff design fixes, and it needs a test suite, in order to land.
  4. [ ] Port the wasmtime-wasi-http crate to use the poll-oneoff fixes above, making it compatible with components instead of just modules]
  5. [ ] Bring wasmtime-wasi::preview2's docs up to Wasmtime project standards.

Once the poll design changes and host-side adapter are landed, we can start working on transitioning the legacy implementation out of wasmtime:

  1. [ ] Move all of wasmtime-wasi's existing preview 1 support behind pub mod preview1_legacy and the off-by-default preview1_legacy cargo feature. Move the preview2 module out to the root. The new host-side adapter interface to provides preview 1 support to all users, except those who have regressions and need to use the legacy support for a limited time.
  2. [ ] The known regressions for preview 2 support include some trailing slash filesystem behaviors, and the support for preview 1's late-added sock_* functions. Presently, the sock functions have no tests in the tree, or anywhere else I can find, so I don't have a way to port those to preview 2 without taking a wild guess at what semantics they need. The only known user of these functions was Enarx, who are no longer in business. If any other users need these sock functions to keep working in either the host or wasm adapter for preview 1, we'll need them to contribute to that support- all of the Bytecode Alliance contributors I have spoke to so far don't have a business case for supporting preview 1's sockets, and instead are all in on preview 2 sockets.
  3. [ ] After 2 releases in off-by-default legacy mode, we will delete the legacy preview 1 code from the repository. This means retiring the wasi-common, wasi-cap-std-sync, and wasi-tokio crates. We could publish a final empty version indicating they wont get any more updates and have been replaced with wasmtime-wasi.

view this post on Zulip Wasmtime GitHub notifications bot (Sep 27 2023 at 22:48):

pchickey edited issue #6370:

We have been working on a prototype of what WASI Preview 2 support will look like in Wasmtime for 7 months now! https://github.com/bytecodealliance/preview2-prototyping/

The work is not yet totally complete, but I believe that it is mature enough that we should start landing it into this repository, and keep iterating on it here. It was included in the March 28 tooling compatibility matrix, but it turns out we skipped compatibility with the wasmtime 8.0 release due to some bindgen changes we needed. I'd like to get everything landed and part of the regular Wasmtime release cycle for 10.0, which will cut at the beginning of June.

Here is my rough plan for how we will land changes. These plans are open to changes so please leave your feedback!

  1. [x] Move the adapter (found at the root of the p2p repo) to crates/wasi-preview1-component-adapter. We will bring its child proc-macro crate byte-array, and the test harness validate, in as well underneath that directory. Add building, validating, and publishing the adapter binary (for both commands and reactors) to the wasmtime repository's CI. Runtime (integration) tests will be added in the subsequent steps.
  2. [x] Restructure the test-programs crate to support building both modules and components (using the adapter from step 1). Keep all the existing tests of wasi-common (which covers both the cap-std-sync and tokio backends) intact, but refactor like we did in p2p to make the structure of how tests are run driven by hand-written code, rather than packing a ton of magic into the macros. The components will be built, but not actually executed yet.
  3. [x] Land the contents of preview2-prototyping's wasi-common crate under wasmtime-wasi in the pub mod preview2 space, and behind the (enabled by default) preview2 cargo feature. Leave this repository's wasi-common, wasi-cap-std-sync, and wasi-tokio crates untouched. They will continue to provide all preview 1 functionality for the moment. The preview 2 functionality will be passing all tests that it does in p2p through the test-programs (at the moment this means poll-oneoff and some trailing slashes behavior is broken), and this PR will port in the other tests from the p2p repo as well.

At this point we can say the wasmtime project has preview 2 support! It is, however, behind a different WasiCtx api (with some changes to the builder, and the way to add it to a linker), and is totally separate from the preview 1 support.

The next steps are:

  1. [x] Finish the development work I have planned to straighten out the poll-oneoff / streams / async vs sync embedding design in preview 2, which is currently not passing all its tests. At this point, we can just ignore some trailing slash behavior in the test suite, but the stuff everyone really cares about ought to be done.
  2. [x] Land @rvolosatovs 's work on a host-side preview 2 wit-bindgen to preview 1 wiggle adapter. Hook this up to the test suite, passing all the same tests as the legacy implementation, without using the wasi-preview1-component-adapter or any component-model features in wasmtime.
  3. [x] Land preview2-prototyping's wasmtime-wasi-sockets crate into this repository. This crate is separate from, but depends upon, what will land in wasmtime-wasi::preview2. It need to be integrated with the poll oneoff design fixes, and it needs a test suite, in order to land.
  4. [x] Port the wasmtime-wasi-http crate to use the poll-oneoff fixes above, making it compatible with components instead of just modules]
  5. [ ] Bring wasmtime-wasi::preview2's docs up to Wasmtime project standards.

Once the poll design changes and host-side adapter are landed, we can start working on transitioning the legacy implementation out of wasmtime:

  1. [ ] Move all of wasmtime-wasi's existing preview 1 support behind pub mod preview1_legacy and the off-by-default preview1_legacy cargo feature. Move the preview2 module out to the root. The new host-side adapter interface to provides preview 1 support to all users, except those who have regressions and need to use the legacy support for a limited time.
  2. [ ] The known regressions for preview 2 support include some trailing slash filesystem behaviors, and the support for preview 1's late-added sock_* functions. Presently, the sock functions have no tests in the tree, or anywhere else I can find, so I don't have a way to port those to preview 2 without taking a wild guess at what semantics they need. The only known user of these functions was Enarx, who are no longer in business. If any other users need these sock functions to keep working in either the host or wasm adapter for preview 1, we'll need them to contribute to that support- all of the Bytecode Alliance contributors I have spoke to so far don't have a business case for supporting preview 1's sockets, and instead are all in on preview 2 sockets.
  3. [ ] After 2 releases in off-by-default legacy mode, we will delete the legacy preview 1 code from the repository. This means retiring the wasi-common, wasi-cap-std-sync, and wasi-tokio crates. We could publish a final empty version indicating they wont get any more updates and have been replaced with wasmtime-wasi.

view this post on Zulip Wasmtime GitHub notifications bot (Sep 27 2023 at 22:49):

pchickey commented on issue #6370:

I think this issue has lived its useful life and we can now call it closed - the preview 2 support is still not totally complete in wasmtime-wasi but it, and the specs themselves, are maturing rapidly.

view this post on Zulip Wasmtime GitHub notifications bot (Sep 27 2023 at 22:49):

pchickey closed issue #6370:

We have been working on a prototype of what WASI Preview 2 support will look like in Wasmtime for 7 months now! https://github.com/bytecodealliance/preview2-prototyping/

The work is not yet totally complete, but I believe that it is mature enough that we should start landing it into this repository, and keep iterating on it here. It was included in the March 28 tooling compatibility matrix, but it turns out we skipped compatibility with the wasmtime 8.0 release due to some bindgen changes we needed. I'd like to get everything landed and part of the regular Wasmtime release cycle for 10.0, which will cut at the beginning of June.

Here is my rough plan for how we will land changes. These plans are open to changes so please leave your feedback!

  1. [x] Move the adapter (found at the root of the p2p repo) to crates/wasi-preview1-component-adapter. We will bring its child proc-macro crate byte-array, and the test harness validate, in as well underneath that directory. Add building, validating, and publishing the adapter binary (for both commands and reactors) to the wasmtime repository's CI. Runtime (integration) tests will be added in the subsequent steps.
  2. [x] Restructure the test-programs crate to support building both modules and components (using the adapter from step 1). Keep all the existing tests of wasi-common (which covers both the cap-std-sync and tokio backends) intact, but refactor like we did in p2p to make the structure of how tests are run driven by hand-written code, rather than packing a ton of magic into the macros. The components will be built, but not actually executed yet.
  3. [x] Land the contents of preview2-prototyping's wasi-common crate under wasmtime-wasi in the pub mod preview2 space, and behind the (enabled by default) preview2 cargo feature. Leave this repository's wasi-common, wasi-cap-std-sync, and wasi-tokio crates untouched. They will continue to provide all preview 1 functionality for the moment. The preview 2 functionality will be passing all tests that it does in p2p through the test-programs (at the moment this means poll-oneoff and some trailing slashes behavior is broken), and this PR will port in the other tests from the p2p repo as well.

At this point we can say the wasmtime project has preview 2 support! It is, however, behind a different WasiCtx api (with some changes to the builder, and the way to add it to a linker), and is totally separate from the preview 1 support.

The next steps are:

  1. [x] Finish the development work I have planned to straighten out the poll-oneoff / streams / async vs sync embedding design in preview 2, which is currently not passing all its tests. At this point, we can just ignore some trailing slash behavior in the test suite, but the stuff everyone really cares about ought to be done.
  2. [x] Land @rvolosatovs 's work on a host-side preview 2 wit-bindgen to preview 1 wiggle adapter. Hook this up to the test suite, passing all the same tests as the legacy implementation, without using the wasi-preview1-component-adapter or any component-model features in wasmtime.
  3. [x] Land preview2-prototyping's wasmtime-wasi-sockets crate into this repository. This crate is separate from, but depends upon, what will land in wasmtime-wasi::preview2. It need to be integrated with the poll oneoff design fixes, and it needs a test suite, in order to land.
  4. [x] Port the wasmtime-wasi-http crate to use the poll-oneoff fixes above, making it compatible with components instead of just modules]
  5. [ ] Bring wasmtime-wasi::preview2's docs up to Wasmtime project standards.

Once the poll design changes and host-side adapter are landed, we can start working on transitioning the legacy implementation out of wasmtime:

  1. [ ] Move all of wasmtime-wasi's existing preview 1 support behind pub mod preview1_legacy and the off-by-default preview1_legacy cargo feature. Move the preview2 module out to the root. The new host-side adapter interface to provides preview 1 support to all users, except those who have regressions and need to use the legacy support for a limited time.
  2. [ ] The known regressions for preview 2 support include some trailing slash filesystem behaviors, and the support for preview 1's late-added sock_* functions. Presently, the sock functions have no tests in the tree, or anywhere else I can find, so I don't have a way to port those to preview 2 without taking a wild guess at what semantics they need. The only known user of these functions was Enarx, who are no longer in business. If any other users need these sock functions to keep working in either the host or wasm adapter for preview 1, we'll need them to contribute to that support- all of the Bytecode Alliance contributors I have spoke to so far don't have a business case for supporting preview 1's sockets, and instead are all in on preview 2 sockets.
  3. [ ] After 2 releases in off-by-default legacy mode, we will delete the legacy preview 1 code from the repository. This means retiring the wasi-common, wasi-cap-std-sync, and wasi-tokio crates. We could publish a final empty version indicating they wont get any more updates and have been replaced with wasmtime-wasi.

Last updated: Nov 22 2024 at 17:03 UTC