Stream: git-wasmtime

Topic: wasmtime / PR #9582 Add support for async/streams/futures

Wasmtime GitHub notifications bot (Nov 07 2024 at 19:28):

dicej opened PR #9582 from dicej:async to bytecodealliance:main:

This adds support for loading, compiling, linking, and running components which use the Async ABI along with the stream, future, and error-context types. It also adds support for generating host bindings such that multiple host functions can be run concurrently with guest tasks -- without monopolizing the Store.

See the implementation RFC for details, as well as this repo containing end-to-end smoke tests.

This is very much a work-in progress, with a number of tasks remaining:

• [ ] Avoid exposing global task IDs to guests and use per-instance IDs instead
• [ ] Track task.return type during compilation and assert the actual and expected types match at runtime
• [ ] Ensure all guest pointers are bounds-checked when lifting, lowering, or copying values
• [ ] Reduce code duplication in wasmtime_cranelift::compiler::component
• [ ] Reduce code duplication between StoreContextMut::on_fiber and concurrent::on_fiber
• [ ] Minimize and/or document the use of unsafe code
• [ ] Add support for (Typed)Func::call_concurrent per the RFC
• [ ] Add support for multiplexing stream/future reads/writes and concurrent calls to guest exports per the RFC
• [ ] Refactor, clean up, and unify handling of backpressure, yields, and even polling
• [ ] Guard against reentrance where required (e.g. in certain fused adapter calls)
• [ ] Add integration test cases covering new functionality to tests/all/component_model (starting by porting over the tests in https://github.com/dicej/component-async-demo)
• [ ] Add binding generation test cases to crates/component-macro/tests
• [ ] Add WAST tests to tests/misc_testsuite/component-model
• [ ] Add support and test coverage for callback-less async functions (e.g. goroutines)
• [ ] Switch to back to upstream wasm-tools once https://github.com/bytecodealliance/wasm-tools/pull/1895 has been merged and released

Wasmtime GitHub notifications bot (Nov 07 2024 at 19:30):

dicej assigned dicej to PR #9582.

Wasmtime GitHub notifications bot (Nov 07 2024 at 20:49):

dicej updated PR #9582 (assigned to dicej).

Wasmtime GitHub notifications bot (Nov 07 2024 at 21:01):

dicej edited PR #9582 (assigned to dicej):

This adds support for loading, compiling, linking, and running components which use the Async ABI along with the stream, future, and error-context types. It also adds support for generating host bindings such that multiple host functions can be run concurrently with guest tasks -- without monopolizing the Store.

See the implementation RFC for details, as well as this repo containing end-to-end smoke tests.

This is very much a work-in progress, with a number of tasks remaining:

• [ ] Avoid exposing global task IDs to guests and use per-instance IDs instead
• [ ] Track task.return type during compilation and assert the actual and expected types match at runtime
• [ ] Ensure all guest pointers are bounds-checked when lifting, lowering, or copying values
• [ ] Reduce code duplication in wasmtime_cranelift::compiler::component
• [ ] Reduce code duplication between StoreContextMut::on_fiber and concurrent::on_fiber
• [ ] Minimize and/or document the use of unsafe code
• [ ] Implement {stream|future}.cancel-{read|write} and error-context.{new|debug-string}
• [ ] Add support for (Typed)Func::call_concurrent per the RFC
• [ ] Add support for multiplexing stream/future reads/writes and concurrent calls to guest exports per the RFC
• [ ] Refactor, clean up, and unify handling of backpressure, yields, and even polling
• [ ] Guard against reentrance where required (e.g. in certain fused adapter calls)
• [ ] Add integration test cases covering new functionality to tests/all/component_model (starting by porting over the tests in https://github.com/dicej/component-async-demo)
• [ ] Add binding generation test cases to crates/component-macro/tests
• [ ] Add WAST tests to tests/misc_testsuite/component-model
• [ ] Add support and test coverage for callback-less async functions (e.g. goroutines)
• [ ] Switch to back to upstream wasm-tools once https://github.com/bytecodealliance/wasm-tools/pull/1895 has been merged and released

Wasmtime GitHub notifications bot (Nov 07 2024 at 21:19):

dicej updated PR #9582 (assigned to dicej).

Wasmtime GitHub notifications bot (Nov 07 2024 at 21:30):

dicej updated PR #9582 (assigned to dicej).

Wasmtime GitHub notifications bot (Nov 07 2024 at 22:22):

dicej updated PR #9582 (assigned to dicej).

Wasmtime GitHub notifications bot (Nov 07 2024 at 22:24):

dicej edited PR #9582 (assigned to dicej):

This adds support for loading, compiling, linking, and running components which use the Async ABI along with the stream, future, and error-context types. It also adds support for generating host bindings such that multiple host functions can be run concurrently with guest tasks -- without monopolizing the Store.

See the implementation RFC for details, as well as this repo containing end-to-end smoke tests.

This is very much a work-in progress, with a number of tasks remaining:

• [ ] Avoid exposing global task IDs to guests and use per-instance IDs instead (and fix no_std build at the same time)
• [ ] Track task.return type during compilation and assert the actual and expected types match at runtime
• [ ] Ensure all guest pointers are bounds-checked when lifting, lowering, or copying values
• [ ] Reduce code duplication in wasmtime_cranelift::compiler::component
• [ ] Reduce code duplication between StoreContextMut::on_fiber and concurrent::on_fiber
• [ ] Minimize and/or document the use of unsafe code
• [ ] Implement {stream|future}.cancel-{read|write} and error-context.{new|debug-string}
• [ ] Add support for (Typed)Func::call_concurrent per the RFC
• [ ] Add support for multiplexing stream/future reads/writes and concurrent calls to guest exports per the RFC
• [ ] Refactor, clean up, and unify handling of backpressure, yields, and even polling
• [ ] Guard against reentrance where required (e.g. in certain fused adapter calls)
• [ ] Add integration test cases covering new functionality to tests/all/component_model (starting by porting over the tests in https://github.com/dicej/component-async-demo)
• [ ] Add binding generation test cases to crates/component-macro/tests
• [ ] Add WAST tests to tests/misc_testsuite/component-model
• [ ] Add support and test coverage for callback-less async functions (e.g. goroutines)
• [ ] Switch to back to upstream wasm-tools once https://github.com/bytecodealliance/wasm-tools/pull/1895 has been merged and released

Wasmtime GitHub notifications bot (Nov 08 2024 at 00:14):

dicej edited PR #9582 (assigned to dicej):

This adds support for loading, compiling, linking, and running components which use the Async ABI along with the stream, future, and error-context types. It also adds support for generating host bindings such that multiple host functions can be run concurrently with guest tasks -- without monopolizing the Store.

See the implementation RFC for details, as well as this repo containing end-to-end smoke tests.

This is very much a work-in progress, with a number of tasks remaining:

• [ ] Avoid exposing global task IDs to guests and use per-instance IDs instead (and fix no_std build at the same time)
• [ ] Track task.return type during compilation and assert the actual and expected types match at runtime
• [ ] Ensure all guest pointers are bounds-checked when lifting, lowering, or copying values
• [ ] Reduce code duplication in wasmtime_cranelift::compiler::component
• [ ] Reduce code duplication between StoreContextMut::on_fiber and concurrent::on_fiber
• [ ] Minimize and/or document the use of unsafe code
• [ ] Implement {stream|future}.cancel-{read|write} and error-context.{new|debug-string}
• [ ] Add support for (Typed)Func::call_concurrent per the RFC
• [ ] Add support for multiplexing stream/future reads/writes and concurrent calls to guest exports per the RFC
• [ ] Refactor, clean up, and unify handling of backpressure, yields, and even polling
• [ ] Guard against reentrance where required (e.g. in certain fused adapter calls)
• [ ] Add integration test cases covering new functionality to tests/all/component_model (starting by porting over the tests in https://github.com/dicej/component-async-demo)
• [ ] Add binding generation test cases to crates/component-macro/tests
• [ ] Add WAST tests to tests/misc_testsuite/component-model
• [ ] Add support and test coverage for callback-less async functions (e.g. goroutines)
• [ ] Add doc and code comments to new, non-trivial code (especially concurrent.rs and futures_and_streams.rs)
• [ ] Switch to back to upstream wasm-tools once https://github.com/bytecodealliance/wasm-tools/pull/1895 has been merged and released

Wasmtime GitHub notifications bot (Nov 14 2024 at 21:07):

dicej edited PR #9582 (assigned to dicej):

This adds support for loading, compiling, linking, and running components which use the Async ABI along with the stream, future, and error-context types. It also adds support for generating host bindings such that multiple host functions can be run concurrently with guest tasks -- without monopolizing the Store.

See the implementation RFC for details, as well as this repo containing end-to-end smoke tests.

This is very much a work-in progress, with a number of tasks remaining:

• [ ] Avoid exposing global task IDs to guests and use per-instance IDs instead (and fix no_std build at the same time)
• [ ] Track task.return type during compilation and assert the actual and expected types match at runtime
• [ ] Ensure all guest pointers are bounds-checked when lifting, lowering, or copying values
• [ ] Reduce code duplication in wasmtime_cranelift::compiler::component
• [ ] Reduce code duplication between StoreContextMut::on_fiber and concurrent::on_fiber
• [ ] Minimize and/or document the use of unsafe code
• [ ] Implement {stream|future}.cancel-{read|write} and error-context.{new|debug-string}
• [ ] Add support for (Typed)Func::call_concurrent per the RFC
• [ ] Add support for multiplexing stream/future reads/writes and concurrent calls to guest exports per the RFC
• [ ] Refactor, clean up, and unify handling of backpressure, yields, and event polling
• [ ] Guard against reentrance where required (e.g. in certain fused adapter calls)
• [ ] Add integration test cases covering new functionality to tests/all/component_model (starting by porting over the tests in https://github.com/dicej/component-async-demo)
• [ ] Add binding generation test cases to crates/component-macro/tests
• [ ] Add WAST tests to tests/misc_testsuite/component-model
• [ ] Add support and test coverage for callback-less async functions (e.g. goroutines)
• [ ] Add doc and code comments to new, non-trivial code (especially concurrent.rs and futures_and_streams.rs)
• [ ] Switch to back to upstream wasm-tools once https://github.com/bytecodealliance/wasm-tools/pull/1895 has been merged and released

Wasmtime GitHub notifications bot (Nov 21 2024 at 18:55):

dicej updated PR #9582 (assigned to dicej).

Wasmtime GitHub notifications bot (Nov 21 2024 at 19:08):

dicej updated PR #9582 (assigned to dicej).

Wasmtime GitHub notifications bot (Nov 21 2024 at 19:53):

dicej updated PR #9582 (assigned to dicej).

Wasmtime GitHub notifications bot (Nov 21 2024 at 19:55):

dicej edited PR #9582 (assigned to dicej):

This adds support for loading, compiling, linking, and running components which use the Async ABI along with the stream, future, and error-context types. It also adds support for generating host bindings such that multiple host functions can be run concurrently with guest tasks -- without monopolizing the Store.

See the implementation RFC for details, as well as this repo containing end-to-end smoke tests.

This is very much a work-in progress, with a number of tasks remaining:

• [x] Avoid exposing global task IDs to guests and use per-instance IDs instead (and fix no_std build at the same time)
• [ ] Track task.return type during compilation and assert the actual and expected types match at runtime
• [ ] Ensure all guest pointers are bounds-checked when lifting, lowering, or copying values
• [ ] Reduce code duplication in wasmtime_cranelift::compiler::component
• [ ] Reduce code duplication between StoreContextMut::on_fiber and concurrent::on_fiber
• [ ] Minimize and/or document the use of unsafe code
• [ ] Implement {stream|future}.cancel-{read|write} and error-context.{new|debug-string}
• [ ] Add support for (Typed)Func::call_concurrent per the RFC
• [ ] Add support for multiplexing stream/future reads/writes and concurrent calls to guest exports per the RFC
• [ ] Refactor, clean up, and unify handling of backpressure, yields, and event polling
• [ ] Guard against reentrance where required (e.g. in certain fused adapter calls)
• [ ] Add integration test cases covering new functionality to tests/all/component_model (starting by porting over the tests in https://github.com/dicej/component-async-demo)
• [ ] Add binding generation test cases to crates/component-macro/tests
• [ ] Add WAST tests to tests/misc_testsuite/component-model
• [ ] Add support and test coverage for callback-less async functions (e.g. goroutines)
• [ ] Add doc and code comments to new, non-trivial code (especially concurrent.rs and futures_and_streams.rs)
• [ ] Switch to back to upstream wasm-tools once https://github.com/bytecodealliance/wasm-tools/pull/1895 has been merged and released

Wasmtime GitHub notifications bot (Nov 22 2024 at 00:37):

dicej updated PR #9582 (assigned to dicej).

Wasmtime GitHub notifications bot (Nov 22 2024 at 00:50):

dicej edited PR #9582 (assigned to dicej):

This adds support for loading, compiling, linking, and running components which use the Async ABI along with the stream, future, and error-context types. It also adds support for generating host bindings such that multiple host functions can be run concurrently with guest tasks -- without monopolizing the Store.

See the implementation RFC for details, as well as this repo containing end-to-end smoke tests.

This is very much a work-in progress, with a number of tasks remaining:

• [x] Avoid exposing global task IDs to guests and use per-instance IDs instead (and fix no_std build at the same time)
• [x] Track task.return type during compilation and assert the actual and expected types match at runtime
• [ ] Ensure all guest pointers are bounds-checked when lifting, lowering, or copying values
• [ ] Reduce code duplication in wasmtime_cranelift::compiler::component
• [ ] Reduce code duplication between StoreContextMut::on_fiber and concurrent::on_fiber
• [ ] Minimize and/or document the use of unsafe code
• [ ] Implement {stream|future}.cancel-{read|write} and error-context.{new|debug-string}
• [ ] Add support for (Typed)Func::call_concurrent per the RFC
• [ ] Add support for multiplexing stream/future reads/writes and concurrent calls to guest exports per the RFC
• [ ] Refactor, clean up, and unify handling of backpressure, yields, and event polling
• [ ] Guard against reentrance where required (e.g. in certain fused adapter calls)
• [ ] Add integration test cases covering new functionality to tests/all/component_model (starting by porting over the tests in https://github.com/dicej/component-async-demo)
• [ ] Add binding generation test cases to crates/component-macro/tests
• [ ] Add WAST tests to tests/misc_testsuite/component-model
• [ ] Add support and test coverage for callback-less async functions (e.g. goroutines)
• [ ] Add doc and code comments to new, non-trivial code (especially concurrent.rs and futures_and_streams.rs)
• [ ] Switch to back to upstream wasm-tools once https://github.com/bytecodealliance/wasm-tools/pull/1895 has been merged and released

Wasmtime GitHub notifications bot (Nov 22 2024 at 15:49):

dicej edited PR #9582 (assigned to dicej):

This adds support for loading, compiling, linking, and running components which use the Async ABI along with the stream, future, and error-context types. It also adds support for generating host bindings such that multiple host functions can be run concurrently with guest tasks -- without monopolizing the Store.

See the implementation RFC for details, as well as this repo containing end-to-end smoke tests.

This is very much a work-in progress, with a number of tasks remaining:

• [x] Avoid exposing global task IDs to guests and use per-instance IDs instead (and fix no_std build at the same time)
• [x] Track task.return type during compilation and assert the actual and expected types match at runtime
• [ ] Implement {stream|future}.cancel-{read|write} and error-context.{new|debug-string}
• [ ] Add support for (Typed)Func::call_concurrent per the RFC
• [ ] Add support for multiplexing stream/future reads/writes and concurrent calls to guest exports per the RFC
• [ ] Add integration test cases covering new functionality to tests/all/component_model (starting by porting over the tests in https://github.com/dicej/component-async-demo)
• [ ] Add binding generation test cases to crates/component-macro/tests
• [ ] Add WAST tests to tests/misc_testsuite/component-model
• [ ] Add support and test coverage for callback-less async functions (e.g. goroutines)
• [ ] Add support and test coverage for dynamic APIs for async/streams/futures/error-contexts
• [ ] Add doc and code comments to new, non-trivial code (especially concurrent.rs and futures_and_streams.rs)
• [ ] Ensure all guest pointers are bounds-checked when lifting, lowering, or copying values
• [ ] Guard against reentrance where required (e.g. in certain fused adapter calls)
• [ ] Minimize and/or document the use of unsafe code
• [ ] Refactor, clean up, and unify handling of backpressure, yields, and event polling
• [ ] Reduce code duplication in wasmtime_cranelift::compiler::component
• [ ] Reduce code duplication between StoreContextMut::on_fiber and concurrent::on_fiber
• [ ] Switch to back to upstream wasm-tools once https://github.com/bytecodealliance/wasm-tools/pull/1895 has been merged and released

Last updated: Nov 22 2024 at 17:03 UTC