Stream: git-wasmtime

Topic: wasmtime / PR #2487 rewrite wasi-common in terms of cap-std

Wasmtime GitHub notifications bot (Dec 08 2020 at 01:12):

pchickey opened PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 08 2020 at 02:21):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 08 2020 at 05:43):

sunfishcode submitted PR Review.

Wasmtime GitHub notifications bot (Dec 08 2020 at 05:43):

sunfishcode submitted PR Review.

Wasmtime GitHub notifications bot (Dec 08 2020 at 05:43):

sunfishcode created PR Review Comment:

system-interface 0.1.0 now has FileIoExt::allocate which does this.

Wasmtime GitHub notifications bot (Dec 08 2020 at 05:43):

sunfishcode created PR Review Comment:

Ah, sorry I neglected to mention it, that's in a separate crate -- fs-set-times. If you use that crate's SystemTimeSpec in place the FilestatSetTime definition above, then usage is basically

use fs_set_times::SetTimes;
file.set_times(atime, mtime)

Or something like pub trait WasiFile: FileIoExt + SetTimes might work too.

Wasmtime GitHub notifications bot (Dec 08 2020 at 05:43):

sunfishcode created PR Review Comment:

cap_std::fs::Metadata implements Unix's MetadataExt on Unix, and Windows' MetadataExt on Windows, with functions that provide all the info here, though I can look into something more portable too.

Wasmtime GitHub notifications bot (Dec 08 2020 at 05:43):

sunfishcode created PR Review Comment:

Yeah, cap-std is following std in this respect. On Unix, PermissionsExt does have a from_mode, but I'll think about adding something portable too.

Wasmtime GitHub notifications bot (Dec 10 2020 at 20:06):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 10 2020 at 21:57):

pchickey submitted PR Review.

Wasmtime GitHub notifications bot (Dec 10 2020 at 21:57):

pchickey created PR Review Comment:

I'll prototype something portable that lets us implement this in the wasi-c2 crate, and we can figure out where to upstream it after.

Wasmtime GitHub notifications bot (Dec 10 2020 at 22:00):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 10 2020 at 22:51):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 10 2020 at 23:07):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 10 2020 at 23:37):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 10 2020 at 23:42):

pchickey submitted PR Review.

Wasmtime GitHub notifications bot (Dec 10 2020 at 23:42):

pchickey created PR Review Comment:

I came up with something janky here which depends on nightly to work on windows...

Wasmtime GitHub notifications bot (Dec 10 2020 at 23:43):

pchickey submitted PR Review.

Wasmtime GitHub notifications bot (Dec 10 2020 at 23:43):

pchickey created PR Review Comment:

Implemented this except for making the flags canonical, which is the sort of thing we can sort out later.

On windows, should we reject this with an error or should we silently do nothing?

Wasmtime GitHub notifications bot (Dec 11 2020 at 01:31):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 11 2020 at 20:14):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 11 2020 at 23:01):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 11 2020 at 23:34):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 12 2020 at 00:43):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 12 2020 at 01:54):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 12 2020 at 02:22):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 15 2020 at 03:48):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 16 2020 at 02:09):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 16 2020 at 23:17):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 17 2020 at 00:07):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 17 2020 at 01:19):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 17 2020 at 01:54):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 18 2020 at 01:30):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 18 2020 at 01:48):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 18 2020 at 02:12):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 18 2020 at 20:31):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 19 2020 at 00:13):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Dec 19 2020 at 01:04):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 04 2021 at 20:21):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 04 2021 at 20:33):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 04 2021 at 20:37):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 05 2021 at 00:04):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 05 2021 at 00:22):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 05 2021 at 00:28):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 05 2021 at 00:42):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 05 2021 at 01:23):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 05 2021 at 01:28):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 05 2021 at 20:20):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 05 2021 at 21:57):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 05 2021 at 22:28):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 05 2021 at 22:36):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 07 2021 at 22:44):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 07 2021 at 22:56):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 08 2021 at 00:39):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 08 2021 at 00:47):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 08 2021 at 14:36):

ueno submitted PR Review.

Wasmtime GitHub notifications bot (Jan 08 2021 at 14:36):

ueno created PR Review Comment:

Probably a dumb question, but I wonder if there is a reason to have separate structs for read and write (ReadPipe and WritePipe) rather than a single bi-directional pipe, something like below?
https://github.com/ueno/wasmtime/blob/wip/dueno/wasi-c2-virt/crates/wasi-c2/src/virt/pipe.rs#L375

Wasmtime GitHub notifications bot (Jan 08 2021 at 16:52):

sunfishcode submitted PR Review.

Wasmtime GitHub notifications bot (Jan 08 2021 at 16:52):

sunfishcode created PR Review Comment:

From my perspective, the granularity of various traits in this area is something we're still figuring out. Could you say more about why Pipe in your link here uses Seek? Elsewhere you mentioned using TCP, QUIC, etc. sockets, however they don't support Seek. Things that do implement Seek such as files and memory buffers aren't interactive and don't make sense for interactive protocols, which is what one typically does on top of TCP, QUIC, etc. sockets.

Wasmtime GitHub notifications bot (Jan 08 2021 at 18:30):

pchickey submitted PR Review.

Wasmtime GitHub notifications bot (Jan 08 2021 at 18:30):

pchickey created PR Review Comment:

The use case we wrote ReadPipe and WritePipe for is just to buffering stdin and stdout/err for an instance, rather than communicate between instances. Bi-directional communication is more complex and something I wasn't ready to bite off yet.

Wasmtime GitHub notifications bot (Jan 11 2021 at 14:47):

ueno submitted PR Review.

Wasmtime GitHub notifications bot (Jan 11 2021 at 14:47):

ueno created PR Review Comment:

Could you say more about why Pipe in your link here uses Seek? Elsewhere you mentioned using TCP, QUIC, etc. sockets, however they don't support Seek.

Yes, that's true; I added it just for my demo of the other branch. On the other hand, I suspect Seek will be desirable in the case with running Wasm programs in TEE, to support transparent encryption of files.

Wasmtime GitHub notifications bot (Jan 11 2021 at 23:48):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 11 2021 at 23:57):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 12 2021 at 01:12):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 12 2021 at 01:48):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 12 2021 at 01:59):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 12 2021 at 02:34):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 12 2021 at 20:07):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 13 2021 at 01:43):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 13 2021 at 19:03):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 13 2021 at 19:07):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 13 2021 at 23:11):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 15 2021 at 01:44):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 19 2021 at 17:57):

sunfishcode submitted PR Review.

Wasmtime GitHub notifications bot (Jan 19 2021 at 17:57):

sunfishcode created PR Review Comment:

system-interface has this in num_ready_bytes. One difference from how wasi-common does it is that system-interface will sometimes return 0 if the underlying device can't report how many bytes it can read. Here, since we just called poll and it told us that something was ready, we should do let n = sub.num_ready_bytes() and then max(n, 1).

Wasmtime GitHub notifications bot (Jan 19 2021 at 22:39):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 19 2021 at 22:43):

pchickey submitted PR Review.

Wasmtime GitHub notifications bot (Jan 19 2021 at 22:43):

pchickey created PR Review Comment:

Fixed now- thanks!

Wasmtime GitHub notifications bot (Jan 19 2021 at 23:14):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 21 2021 at 03:09):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 21 2021 at 04:54):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 21 2021 at 05:03):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 21 2021 at 21:51):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 21 2021 at 22:14):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 22 2021 at 00:55):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 22 2021 at 01:57):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 22 2021 at 02:08):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 22 2021 at 02:18):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 22 2021 at 21:44):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 22 2021 at 23:33):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 22 2021 at 23:44):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 23 2021 at 02:27):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 25 2021 at 20:20):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 25 2021 at 20:54):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 25 2021 at 23:20):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 25 2021 at 23:28):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 26 2021 at 00:59):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 26 2021 at 01:02):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 26 2021 at 01:08):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 26 2021 at 01:15):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 26 2021 at 20:25):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 26 2021 at 20:27):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 27 2021 at 20:18):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 27 2021 at 20:27):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 27 2021 at 20:43):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 28 2021 at 00:16):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 28 2021 at 02:10):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 28 2021 at 02:18):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 28 2021 at 03:06):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 28 2021 at 17:44):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 28 2021 at 19:44):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 28 2021 at 20:24):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 28 2021 at 21:03):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 28 2021 at 21:23):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 28 2021 at 23:17):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 29 2021 at 01:00):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 29 2021 at 20:07):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 29 2021 at 20:29):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 29 2021 at 20:38):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 29 2021 at 21:25):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 29 2021 at 22:10):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 29 2021 at 22:27):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 29 2021 at 23:40):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 29 2021 at 23:43):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 29 2021 at 23:49):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 30 2021 at 01:24):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 30 2021 at 01:27):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 30 2021 at 02:02):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 30 2021 at 03:52):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 30 2021 at 03:53):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 30 2021 at 03:54):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 30 2021 at 04:58):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 30 2021 at 05:09):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 30 2021 at 05:25):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Jan 30 2021 at 21:58):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 01 2021 at 19:01):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 01 2021 at 19:10):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 01 2021 at 20:43):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 01 2021 at 21:16):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 01 2021 at 21:21):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 01 2021 at 21:21):

pchickey has marked PR #2487 as ready for review.

Wasmtime GitHub notifications bot (Feb 01 2021 at 22:14):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 01 2021 at 22:26):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 01 2021 at 22:33):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 01 2021 at 22:44):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 01 2021 at 23:26):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

WIP

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 01 2021 at 23:47):

pchickey edited PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

The WasiFile and WasiDir traits

Traits for the rest of WASI's features

The cap-std-sync implementation of the WASI traits

Improving the WASI test suite

Wasi-common's test suite lives at crates/test-programs.

Changes to wiggle-wasmtime and dependents

Changes to wasmtime-wasi

Regressions

Some behavior around trailing slashes in paths has changed from the test suite. We expect there are some improvements we'll make to these corner cases after this PR lands.

For now, the path_rename_file_trailing_slashes and remove_directory_trailing_slashes tests are #[ignore]'d on all platforms, and additionally the interesting_paths test is ignored on Windows.

Additionally, some behavior around the FdFlags::{SYNC, DSYNC, RSYNC} flags have changed. For now, cap-std does not support opening a file with these flags. Using these flags will result in an Errno::NOTSUP (Not supported).

Outstanding issues required to merge this PR:

• [ ] Metadata not associated with open file panics on windows: https://github.com/bytecodealliance/cap-std/issues/142
• [ ] DirExt::delete_file_or_symlink for symlink_create test to pass on Windows

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 01 2021 at 23:50):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

The WasiFile and WasiDir traits

Traits for the rest of WASI's features

The cap-std-sync implementation of the WASI traits

Improving the WASI test suite

Wasi-common's test suite lives at crates/test-programs.

Changes to wiggle-wasmtime and dependents

Changes to wasmtime-wasi

Regressions

Some behavior around trailing slashes in paths has changed from the test suite. We expect there are some improvements we'll make to these corner cases after this PR lands.

For now, the path_rename_file_trailing_slashes and remove_directory_trailing_slashes tests are #[ignore]'d on all platforms, and additionally the interesting_paths test is ignored on Windows.

Additionally, some behavior around the FdFlags::{SYNC, DSYNC, RSYNC} flags have changed. For now, cap-std does not support opening a file with these flags. Using these flags will result in an Errno::NOTSUP (Not supported).

Outstanding issues required to merge this PR:

• [ ] Metadata not associated with open file panics on windows: https://github.com/bytecodealliance/cap-std/issues/142
• [ ] DirExt::delete_file_or_symlink for symlink_create test to pass on Windows

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 02 2021 at 00:03):

pchickey edited PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types.

Traits for the rest of WASI's features

Snapshot architecture

The cap-std-sync implementation of the WASI traits

Improving the WASI test suite

Wasi-common's test suite lives at crates/test-programs.

Changes to wiggle-wasmtime and dependents

Changes to wasmtime-wasi

Regressions

Some behavior around trailing slashes in paths has changed from the test suite. We expect there are some improvements we'll make to these corner cases after this PR lands.

For now, the path_rename_file_trailing_slashes and remove_directory_trailing_slashes tests are #[ignore]'d on all platforms, and additionally the interesting_paths test is ignored on Windows.

Additionally, some behavior around the FdFlags::{SYNC, DSYNC, RSYNC} flags have changed. For now, cap-std does not support opening a file with these flags. Using these flags will result in an Errno::NOTSUP (Not supported).

Outstanding issues required to merge this PR:

• [ ] Metadata not associated with open file panics on windows: https://github.com/bytecodealliance/cap-std/issues/142
• [ ] DirExt::delete_file_or_symlink for symlink_create test to pass on Windows

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 02 2021 at 00:11):

pchickey edited PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR)

Traits for the rest of WASI's features

Snapshot architecture

The cap-std-sync implementation of the WASI traits

Improving the WASI test suite

Wasi-common's test suite lives at crates/test-programs.

Changes to wiggle-wasmtime and dependents

Changes to wasmtime-wasi

Regressions

Some behavior around trailing slashes in paths has changed from the test suite. We expect there are some improvements we'll make to these corner cases after this PR lands.

For now, the path_rename_file_trailing_slashes and remove_directory_trailing_slashes tests are #[ignore]'d on all platforms, and additionally the interesting_paths test is ignored on Windows.

Additionally, some behavior around the FdFlags::{SYNC, DSYNC, RSYNC} flags have changed. For now, cap-std does not support opening a file with these flags. Using these flags will result in an Errno::NOTSUP (Not supported).

Outstanding issues required to merge this PR:

• [ ] Metadata not associated with open file panics on windows: https://github.com/bytecodealliance/cap-std/issues/142
• [ ] DirExt::delete_file_or_symlink for symlink_create test to pass on Windows
• [ ] Yeet the virtfs out to a separate PR

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 02 2021 at 00:25):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR)

Traits for the rest of WASI's features

Snapshot architecture

The cap-std-sync implementation of the WASI traits

Improving the WASI test suite

Wasi-common's test suite lives at crates/test-programs.

Changes to wiggle-wasmtime and dependents

Changes to wasmtime-wasi

Regressions

Some behavior around trailing slashes in paths has changed from the test suite. We expect there are some improvements we'll make to these corner cases after this PR lands.

For now, the path_rename_file_trailing_slashes and remove_directory_trailing_slashes tests are #[ignore]'d on all platforms, and additionally the interesting_paths test is ignored on Windows.

Additionally, some behavior around the FdFlags::{SYNC, DSYNC, RSYNC} flags have changed. For now, cap-std does not support opening a file with these flags. Using these flags will result in an Errno::NOTSUP (Not supported).

Outstanding issues required to merge this PR:

• [ ] Metadata not associated with open file panics on windows: https://github.com/bytecodealliance/cap-std/issues/142
• [ ] DirExt::delete_file_or_symlink for symlink_create test to pass on Windows
• [ ] Yeet the virtfs out to a separate PR

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 02 2021 at 00:45):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR)

Traits for the rest of WASI's features

Snapshot architecture

The cap-std-sync implementation of the WASI traits

Improving the WASI test suite

Wasi-common's test suite lives at crates/test-programs.

Changes to wiggle-wasmtime and dependents

Changes to wasmtime-wasi

Regressions

Some behavior around trailing slashes in paths has changed from the test suite. We expect there are some improvements we'll make to these corner cases after this PR lands.

For now, the path_rename_file_trailing_slashes and remove_directory_trailing_slashes tests are #[ignore]'d on all platforms, and additionally the interesting_paths test is ignored on Windows.

Additionally, some behavior around the FdFlags::{SYNC, DSYNC, RSYNC} flags have changed. For now, cap-std does not support opening a file with these flags. Using these flags will result in an Errno::NOTSUP (Not supported).

Outstanding issues required to merge this PR:

• [ ] Metadata not associated with open file panics on windows: https://github.com/bytecodealliance/cap-std/issues/142
• [ ] DirExt::delete_file_or_symlink for symlink_create test to pass on Windows
• [ ] Yeet the virtfs out to a separate PR

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 02 2021 at 00:57):

pchickey edited PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Note that this is a breaking change for all users of WASI with Wasmtime, except for the C API.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR).

Traits for the rest of WASI's features

Other aspects of a WASI implementation are not yet considered resources and accessed by handle. We plan to correct this design deficiency in WASI in the future, but for now we have designed the following traits to provide embedders with the same sort of implementation flexibility they get with WasiFile/WasiDir:

• Timekeeping: clocks::WasiSystemClock and clock::WasiMonotonicClock provide the two interfaces for a clock. WasiSystemClock represents time as a cap_std::time::SystemTime, and WasiMonotonicClock represents time as cap_std::time::Instant.
• Randomness: we re-use the cap_rand::RngCore trait to represent a randomness source. A trivial Deterministic impl is provided.
• Scheduling: The WasiSched trait abstracts over the sched_yield and poll_oneoff functions.

Users can provide implementations of each of these interfaces to the WasiCtx::builder(...) function. The wasi_cap_std_sync::WasiCtxBuilder::new() function uses this public interface to plug in its own implementations of each of these resources.

Snapshot architecture

One goal we've had for a while, but not quite met, is for multiple WASI snapshots to provide an interface to the same underlying WasiCtx. This provides us a path to evolve WASI by allowing the same WASI Command to import functions from different snapshots - e.g. the user could use Rust's std which imports snapshot 1, but also depend directly on the wasi crate which imports some future snapshot 2. Right now, this amounts to supporting snapshot 1 and "snapshot 0" aka wasi_unstable at once.

The architectural rules for snapshots are:

• Snapshots are arranged into modules under crate::snapshots::.
• Each snapshot should invoke wiggle::from_witx! with ctx: crate::WasiCtx in its module, and impl all of the required traits.
• Snapshots can be implemented in terms of other snapshots. For example, snapshot 0 is mostly implemented by calling the snapshot 1 implementation, and converting its own types back and forth with the snapshot 1 types. In a few cases, that is not feasible, so snapshot 0 carries its own implementations in terms of the WasiFile and WasiSched traits.
• Snapshots can be implemented in terms of the Wasi* traits given by WasiCtx. No further downcasting via the as_any escape hatch is permitted.

The cap-std-sync implementation of the WASI traits

Improving the WASI test suite

Wasi-common's test suite lives at crates/test-programs.

Changes to wiggle-wasmtime and dependents

Changes to wasmtime-wasi

Regressions

Some behavior around trailing slashes in paths has changed from the test suite. We expect there are some improvements we'll make to these corner cases after this PR lands.

For now, the path_rename_file_trailing_slashes and remove_directory_trailing_slashes tests are #[ignore]'d on all platforms, and additionally the interesting_paths test is ignored on Windows.

Additionally, some behavior around the FdFlags::{SYNC, DSYNC, RSYNC} flags have changed. For now, cap-std does not support opening a file with these flags. Using these flags will result in an Errno::NOTSUP (Not supported).

Outstanding issues required to merge this PR:

• [ ] Metadata not associated with open file panics on windows: https://github.com/bytecodealliance/cap-std/issues/142
• [ ] DirExt::delete_file_or_symlink for symlink_create test to pass on Windows
• [ ] Yeet the virtfs out to a separate PR

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 02 2021 at 01:06):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Note that this is a breaking change for all users of WASI with Wasmtime, except for the C API.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR).

Traits for the rest of WASI's features

Other aspects of a WASI implementation are not yet considered resources and accessed by handle. We plan to correct this design deficiency in WASI in the future, but for now we have designed the following traits to provide embedders with the same sort of implementation flexibility they get with WasiFile/WasiDir:

• Timekeeping: clocks::WasiSystemClock and clock::WasiMonotonicClock provide the two interfaces for a clock. WasiSystemClock represents time as a cap_std::time::SystemTime, and WasiMonotonicClock represents time as cap_std::time::Instant.
• Randomness: we re-use the cap_rand::RngCore trait to represent a randomness source. A trivial Deterministic impl is provided.
• Scheduling: The WasiSched trait abstracts over the sched_yield and poll_oneoff functions.

Users can provide implementations of each of these interfaces to the WasiCtx::builder(...) function. The wasi_cap_std_sync::WasiCtxBuilder::new() function uses this public interface to plug in its own implementations of each of these resources.

Snapshot architecture

One goal we've had for a while, but not quite met, is for multiple WASI snapshots to provide an interface to the same underlying WasiCtx. This provides us a path to evolve WASI by allowing the same WASI Command to import functions from different snapshots - e.g. the user could use Rust's std which imports snapshot 1, but also depend directly on the wasi crate which imports some future snapshot 2. Right now, this amounts to supporting snapshot 1 and "snapshot 0" aka wasi_unstable at once.

The architectural rules for snapshots are:

• Snapshots are arranged into modules under crate::snapshots::.
• Each snapshot should invoke wiggle::from_witx! with ctx: crate::WasiCtx in its module, and impl all of the required traits.
• Snapshots can be implemented in terms of other snapshots. For example, snapshot 0 is mostly implemented by calling the snapshot 1 implementation, and converting its own types back and forth with the snapshot 1 types. In a few cases, that is not feasible, so snapshot 0 carries its own implementations in terms of the WasiFile and WasiSched traits.
• Snapshots can be implemented in terms of the Wasi* traits given by WasiCtx. No further downcasting via the as_any escape hatch is permitted.

The cap-std-sync implementation of the WASI traits

Improving the WASI test suite

Wasi-common's test suite lives at crates/test-programs.

Changes to wiggle-wasmtime and dependents

Changes to wasmtime-wasi

Regressions

Some behavior around trailing slashes in paths has changed from the test suite. We expect there are some improvements we'll make to these corner cases after this PR lands.

For now, the path_rename_file_trailing_slashes and remove_directory_trailing_slashes tests are #[ignore]'d on all platforms, and additionally the interesting_paths test is ignored on Windows.

Additionally, some behavior around the FdFlags::{SYNC, DSYNC, RSYNC} flags have changed. For now, cap-std does not support opening a file with these flags. Using these flags will result in an Errno::NOTSUP (Not supported).

Outstanding issues required to merge this PR:

• [ ] Metadata not associated with open file panics on windows: https://github.com/bytecodealliance/cap-std/issues/142
• [ ] DirExt::delete_file_or_symlink for symlink_create test to pass on Windows
• [ ] Yeet the virtfs out to a separate PR

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 02 2021 at 01:07):

pchickey edited PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Note that this is a breaking change for all users of WASI with Wasmtime, except for the C API.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR).

Traits for the rest of WASI's features

Other aspects of a WASI implementation are not yet considered resources and accessed by handle. We plan to correct this design deficiency in WASI in the future, but for now we have designed the following traits to provide embedders with the same sort of implementation flexibility they get with WasiFile/WasiDir:

• Timekeeping: clocks::WasiSystemClock and clock::WasiMonotonicClock provide the two interfaces for a clock. WasiSystemClock represents time as a cap_std::time::SystemTime, and WasiMonotonicClock represents time as cap_std::time::Instant.
• Randomness: we re-use the cap_rand::RngCore trait to represent a randomness source. A trivial Deterministic impl is provided.
• Scheduling: The WasiSched trait abstracts over the sched_yield and poll_oneoff functions.

Users can provide implementations of each of these interfaces to the WasiCtx::builder(...) function. The wasi_cap_std_sync::WasiCtxBuilder::new() function uses this public interface to plug in its own implementations of each of these resources.

Snapshot architecture

One goal we've had for a while, but not quite met, is for multiple WASI snapshots to provide an interface to the same underlying WasiCtx. This provides us a path to evolve WASI by allowing the same WASI Command to import functions from different snapshots - e.g. the user could use Rust's std which imports snapshot 1, but also depend directly on the wasi crate which imports some future snapshot 2. Right now, this amounts to supporting snapshot 1 and "snapshot 0" aka wasi_unstable at once.

The architectural rules for snapshots are:

• Snapshots are arranged into modules under crate::snapshots::.
• Each snapshot should invoke wiggle::from_witx! with ctx: crate::WasiCtx in its module, and impl all of the required traits.
• Snapshots can be implemented in terms of other snapshots. For example, snapshot 0 is mostly implemented by calling the snapshot 1 implementation, and converting its own types back and forth with the snapshot 1 types. In a few cases, that is not feasible, so snapshot 0 carries its own implementations in terms of the WasiFile and WasiSched traits.
• Snapshots can be implemented in terms of the Wasi* traits given by WasiCtx. No further downcasting via the as_any escape hatch is permitted.

The cap-std-sync implementation of the WASI traits

Why is this impl suffixed with -sync? https://github.com/bytecodealliance/wasmtime/pull/2434 async is coming soon!

Improving the WASI test suite

Wasi-common's test suite lives at crates/test-programs.

Changes to wiggle-wasmtime and dependents

Changes to wasmtime-wasi

Regressions

Some behavior around trailing slashes in paths has changed from the test suite. We expect there are some improvements we'll make to these corner cases after this PR lands.

For now, the path_rename_file_trailing_slashes and remove_directory_trailing_slashes tests are #[ignore]'d on all platforms, and additionally the interesting_paths test is ignored on Windows.

Additionally, some behavior around the FdFlags::{SYNC, DSYNC, RSYNC} flags have changed. For now, cap-std does not support opening a file with these flags. Using these flags will result in an Errno::NOTSUP (Not supported).

Outstanding issues required to merge this PR:

• [ ] Metadata not associated with open file panics on windows: https://github.com/bytecodealliance/cap-std/issues/142
• [ ] DirExt::delete_file_or_symlink for symlink_create test to pass on Windows
• [ ] Yeet the virtfs out to a separate PR

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 02 2021 at 01:08):

pchickey edited PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Note that this is a breaking change for all users of WASI with Wasmtime, except for the C API.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR).

Traits for the rest of WASI's features

Other aspects of a WASI implementation are not yet considered resources and accessed by handle. We plan to correct this design deficiency in WASI in the future, but for now we have designed the following traits to provide embedders with the same sort of implementation flexibility they get with WasiFile/WasiDir:

• Timekeeping: clocks::WasiSystemClock and clock::WasiMonotonicClock provide the two interfaces for a clock. WasiSystemClock represents time as a cap_std::time::SystemTime, and WasiMonotonicClock represents time as cap_std::time::Instant.
• Randomness: we re-use the cap_rand::RngCore trait to represent a randomness source. A trivial Deterministic impl is provided.
• Scheduling: The WasiSched trait abstracts over the sched_yield and poll_oneoff functions.

Users can provide implementations of each of these interfaces to the WasiCtx::builder(...) function. The wasi_cap_std_sync::WasiCtxBuilder::new() function uses this public interface to plug in its own implementations of each of these resources.

Snapshot architecture

One goal we've had for a while, but not quite met, is for multiple WASI snapshots to provide an interface to the same underlying WasiCtx. This provides us a path to evolve WASI by allowing the same WASI Command to import functions from different snapshots - e.g. the user could use Rust's std which imports snapshot 1, but also depend directly on the wasi crate which imports some future snapshot 2. Right now, this amounts to supporting snapshot 1 and "snapshot 0" aka wasi_unstable at once.

The architectural rules for snapshots are:

• Snapshots are arranged into modules under crate::snapshots::.
• Each snapshot should invoke wiggle::from_witx! with ctx: crate::WasiCtx in its module, and impl all of the required traits.
• Snapshots can be implemented in terms of other snapshots. For example, snapshot 0 is mostly implemented by calling the snapshot 1 implementation, and converting its own types back and forth with the snapshot 1 types. In a few cases, that is not feasible, so snapshot 0 carries its own implementations in terms of the WasiFile and WasiSched traits.
• Snapshots can be implemented in terms of the Wasi* traits given by WasiCtx. No further downcasting via the as_any escape hatch is permitted.

The cap-std-sync implementation of the WASI traits

Why is this impl suffixed with -sync? https://github.com/bytecodealliance/wasmtime/pull/2434 async is coming soon! The async impl may end up depending on tokio or other relatively heavy deps, so we will retain a sync implementation so that wasi-common users have an option of not pulling in an async runtime.

Improving the WASI test suite

Wasi-common's test suite lives at crates/test-programs.

Changes to wiggle-wasmtime and dependents

Changes to wasmtime-wasi

Regressions

Some behavior around trailing slashes in paths has changed from the test suite. We expect there are some improvements we'll make to these corner cases after this PR lands.

For now, the path_rename_file_trailing_slashes and remove_directory_trailing_slashes tests are #[ignore]'d on all platforms, and additionally the interesting_paths test is ignored on Windows.

Additionally, some behavior around the FdFlags::{SYNC, DSYNC, RSYNC} flags have changed. For now, cap-std does not support opening a file with these flags. Using these flags will result in an Errno::NOTSUP (Not supported).

Outstanding issues required to merge this PR:

• [ ] Metadata not associated with open file panics on windows: https://github.com/bytecodealliance/cap-std/issues/142
• [ ] DirExt::delete_file_or_symlink for symlink_create test to pass on Windows
• [ ] Yeet the virtfs out to a separate PR

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 02 2021 at 01:18):

pchickey edited PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Note that this is a breaking change for all users of WASI with Wasmtime, except for the C API.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR).

Traits for the rest of WASI's features

Other aspects of a WASI implementation are not yet considered resources and accessed by handle. We plan to correct this design deficiency in WASI in the future, but for now we have designed the following traits to provide embedders with the same sort of implementation flexibility they get with WasiFile/WasiDir:

• Timekeeping: clocks::WasiSystemClock and clock::WasiMonotonicClock provide the two interfaces for a clock. WasiSystemClock represents time as a cap_std::time::SystemTime, and WasiMonotonicClock represents time as cap_std::time::Instant.
• Randomness: we re-use the cap_rand::RngCore trait to represent a randomness source. A trivial Deterministic impl is provided.
• Scheduling: The WasiSched trait abstracts over the sched_yield and poll_oneoff functions.

Users can provide implementations of each of these interfaces to the WasiCtx::builder(...) function. The wasi_cap_std_sync::WasiCtxBuilder::new() function uses this public interface to plug in its own implementations of each of these resources.

Snapshot architecture

One goal we've had for a while, but not quite met, is for multiple WASI snapshots to provide an interface to the same underlying WasiCtx. This provides us a path to evolve WASI by allowing the same WASI Command to import functions from different snapshots - e.g. the user could use Rust's std which imports snapshot 1, but also depend directly on the wasi crate which imports some future snapshot 2. Right now, this amounts to supporting snapshot 1 and "snapshot 0" aka wasi_unstable at once.

The architectural rules for snapshots are:

• Snapshots are arranged into modules under crate::snapshots::.
• Each snapshot should invoke wiggle::from_witx! with ctx: crate::WasiCtx in its module, and impl all of the required traits.
• Snapshots can be implemented in terms of other snapshots. For example, snapshot 0 is mostly implemented by calling the snapshot 1 implementation, and converting its own types back and forth with the snapshot 1 types. In a few cases, that is not feasible, so snapshot 0 carries its own implementations in terms of the WasiFile and WasiSched traits.
• Snapshots can be implemented in terms of the Wasi* traits given by WasiCtx. No further downcasting via the as_any escape hatch is permitted.

The wasi-cap-std-sync implementation of the WASI traits

The wasi-cap-std-sync crate provides impl of WasiFile and WasiDir in terms of cap_std::fs::{File, Dir}. These types provide sandboxed access to the local filesystem on both Unix and Windows.

The entire design story of cap-std is much bigger than we can address here, but in short, its functionality replaces all of the wasi_common::sys hierarchy, as well as the yanix / winx crates. All syscalls are hidden behind the cap-std hierarchy, with the lone exception of the sched implementation, which is provided for both unix and windows in separate modules.

Any wasi_common::{WasiCtx, WasiCtxBuilder} is interoperable with the wasi-cap-std-sync crate. However, for convenience, wasi-cap-std-sync provides its own WasiCtxBuilder that hooks up to all of the crate's components, i.e. it fills in all of the arguments to WasiCtx::builder(...), presents preopen_dir in terms of cap_std::fs::Dir, and provides convenience methods for inheriting the parent process's stdio, args, and env.

The only place we expect to run into long-term compatibility issues between wasi-cap-std-sync and the other impl crates that will come later is in the Sched abstraction. Once we can build an async scheduler based on Rust Futures, async impls will be able to interoperate, but the synchronous scheduler depends on downcasting the WasiFile type down to concrete types it knows about (which in turn impl AsRawFd for passing to unix poll, or the analogous traits on windows).

Why is this impl suffixed with -sync? https://github.com/bytecodealliance/wasmtime/pull/2434 async is coming soon! The async impl may end up depending on tokio or other relatively heavy deps, so we will retain a sync implementation so that wasi-common users have an option of not pulling in an async runtime.

Improving the WASI test suite

Wasi-common's test suite lives at crates/test-programs.

Changes to wiggle-wasmtime and dependents

Changes to wasmtime-wasi

Regressions

Some behavior around trailing slashes in paths has changed from the test suite. We expect there are some improvements we'll make to these corner cases after this PR lands.

For now, the path_rename_file_trailing_slashes and remove_directory_trailing_slashes tests are #[ignore]'d on all platforms, and additionally the interesting_paths test is ignored on Windows.

Additionally, some behavior around the FdFlags::{SYNC, DSYNC, RSYNC} flags have changed. For now, cap-std does not support opening a file with these flags. Using these flags will result in an Errno::NOTSUP (Not supported).

Outstanding issues required to merge this PR:

• [ ] Metadata not associated with open file panics on windows: https://github.com/bytecodealliance/cap-std/issues/142
• [ ] DirExt::delete_file_or_symlink for symlink_create test to pass on Windows
• [ ] Yeet the virtfs out to a separate PR

<!--

Please ensure that the following steps are all taken care of before submitting
the PR.

• [ ] This has been discussed in issue #..., or if not, please tell us why
  here.

• [ ] A short description of what this does, why it is needed; if the
  description becomes long, the matter should probably be discussed in an issue
  first.

• [ ] This PR contains test cases, if meaningful.

• [ ] A reviewer from the core maintainer team has been assigned for this PR.
  If you don't know who could review this, please indicate so. The list of
  suggested reviewers on the right can help you.

Please ensure all communication adheres to the code of conduct.
-->

Wasmtime GitHub notifications bot (Feb 02 2021 at 01:29):

pchickey edited PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Note that this is a breaking change for all users of WASI with Wasmtime, except for the C API.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR).

Traits for the rest of WASI's features

Other aspects of a WASI implementation are not yet considered resources and accessed by handle. We plan to correct this design deficiency in WASI in the future, but for now we have designed the following traits to provide embedders with the same sort of implementation flexibility they get with WasiFile/WasiDir:

• Timekeeping: clocks::WasiSystemClock and clock::WasiMonotonicClock provide the two interfaces for a clock. WasiSystemClock represents time as a cap_std::time::SystemTime, and WasiMonotonicClock represents time as cap_std::time::Instant.
• Randomness: we re-use the cap_rand::RngCore trait to represent a randomness source. A trivial Deterministic impl is provided.
• Scheduling: The WasiSched trait abstracts over the sched_yield and poll_oneoff functions.

Users can provide implementations of each of these interfaces to the WasiCtx::builder(...) function. The wasi_cap_std_sync::WasiCtxBuilder::new() function uses this public interface to plug in its own implementations of each of these resources.

Snapshot architecture

One goal we've had for a while, but not quite met, is for multiple WASI snapshots to provide an interface to the same underlying WasiCtx. This provides us a path to evolve WASI by allowing the same WASI Command to import functions from different snapshots - e.g. the user could use Rust's std which imports snapshot 1, but also depend directly on the wasi crate which imports some future snapshot 2. Right now, this amounts to supporting snapshot 1 and "snapshot 0" aka wasi_unstable at once.

The architectural rules for snapshots are:

• Snapshots are arranged into modules under crate::snapshots::.
• Each snapshot should invoke wiggle::from_witx! with ctx: crate::WasiCtx in its module, and impl all of the required traits.
• Snapshots can be implemented in terms of other snapshots. For example, snapshot 0 is mostly implemented by calling the snapshot 1 implementation, and converting its own types back and forth with the snapshot 1 types. In a few cases, that is not feasible, so snapshot 0 carries its own implementations in terms of the WasiFile and WasiSched traits.
• Snapshots can be implemented in terms of the Wasi* traits given by WasiCtx. No further downcasting via the as_any escape hatch is permitted.

The wasi-cap-std-sync implementation of the WASI traits

The wasi-cap-std-sync crate provides impl of WasiFile and WasiDir in terms of cap_std::fs::{File, Dir}. These types provide sandboxed access to the local filesystem on both Unix and Windows.

The entire design story of cap-std is much bigger than we can address here, but in short, its functionality replaces all of the wasi_common::sys hierarchy, as well as the yanix / winx crates. All syscalls are hidden behind the cap-std hierarchy, with the lone exception of the sched implementation, which is provided for both unix and windows in separate modules.

Any wasi_common::{WasiCtx, WasiCtxBuilder} is interoperable with the wasi-cap-std-sync crate. However, for convenience, wasi-cap-std-sync provides its own WasiCtxBuilder that hooks up to all of the crate's components, i.e. it fills in all of the arguments to WasiCtx::builder(...), presents preopen_dir in terms of cap_std::fs::Dir, and provides convenience methods for inheriting the parent process's stdio, args, and env.

The only place we expect to run into long-term compatibility issues between wasi-cap-std-sync and the other impl crates that will come later is in the Sched abstraction. Once we can build an async scheduler based on Rust Futures, async impls will be able to interoperate, but the synchronous scheduler depends on downcasting the WasiFile type down to concrete types it knows about (which in turn impl AsRawFd for passing to unix poll, or the analogous traits on windows).

Why is this impl suffixed with -sync? https://github.com/bytecodealliance/wasmtime/pull/2434 async is coming soon! The async impl may end up depending on tokio or other relatively heavy deps, so we will retain a sync implementation so that wasi-common users have an option of not pulling in an async runtime.

Improving the WASI test suite

The bulk of wasi-common's test suite lives at crates/test-programs. This test suite was extremely useful for guiding this rewrite. We improved the test suite in numerous ways, including

• Breaking tests out into smaller units, so that failing behavior can be isolated more easily
• Introducing a scheme which allows the test runner to describe to the wasm code what behaviors to expect of the embedding. Behaviors which vary between platforms (e.g. fd_allcoate is impossible to faithfully implement on Windows and is not provided by MacOS) are communicated to the guest by environment variables, which in turn is available in the guest behind the global TESTCONFIG struct.
• Introducing the assert_errno! macro, which pretty-prints errnos by name rather than number in the error message. assert_errno! integrates with TESTCONFIG to specify which errno is expected on which platform. This allows the test source to describe the full set of acceptable errnos a call may return (all possible errnos will be acceptable if no ERRNO_MODE_* env var is set), and also detect regressions on any given platform.

Changes to wiggle-wasmtime and dependents

Changes to wasmtime-wasi

Regressions

Some behavior around trailing slashes in paths has changed from the test suite. We expect there are some improvements we'll make to these corner cases after this PR lands.

For now, the path_rename_file_trailing_slashes and remove_directory_trailing_slashes tests are #[ignore]'d on all platforms, and additionally the interesting_paths test is ignored on Windows.

Additionally, some behavior around the FdFlags::{SYNC, DSYNC, RSYNC} flags have changed. For now, cap-std does not support opening a file with these flags. Using these flags will result in an Errno::NOTSUP (Not supported).

Outstanding issues required to merge this PR:

• [ ] Metadata not associated with open file panics on windows: https://github.com/bytecodealliance/cap-std/issues/142
• [ ] DirExt::delete_file_or_symlink for symlink_create test to pass on Windows
• [ ] Yeet the virtfs into a separate PR
  

Wasmtime GitHub notifications bot (Feb 02 2021 at 01:39):

pchickey edited PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Note that this is a breaking change for all users of WASI with Wasmtime, except for the C API.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR).

Traits for the rest of WASI's features

Other aspects of a WASI implementation are not yet considered resources and accessed by handle. We plan to correct this design deficiency in WASI in the future, but for now we have designed the following traits to provide embedders with the same sort of implementation flexibility they get with WasiFile/WasiDir:

• Timekeeping: clocks::WasiSystemClock and clock::WasiMonotonicClock provide the two interfaces for a clock. WasiSystemClock represents time as a cap_std::time::SystemTime, and WasiMonotonicClock represents time as cap_std::time::Instant.
• Randomness: we re-use the cap_rand::RngCore trait to represent a randomness source. A trivial Deterministic impl is provided.
• Scheduling: The WasiSched trait abstracts over the sched_yield and poll_oneoff functions.

Users can provide implementations of each of these interfaces to the WasiCtx::builder(...) function. The wasi_cap_std_sync::WasiCtxBuilder::new() function uses this public interface to plug in its own implementations of each of these resources.

Snapshot architecture

One goal we've had for a while, but not quite met, is for multiple WASI snapshots to provide an interface to the same underlying WasiCtx. This provides us a path to evolve WASI by allowing the same WASI Command to import functions from different snapshots - e.g. the user could use Rust's std which imports snapshot 1, but also depend directly on the wasi crate which imports some future snapshot 2. Right now, this amounts to supporting snapshot 1 and "snapshot 0" aka wasi_unstable at once.

The architectural rules for snapshots are:

• Snapshots are arranged into modules under crate::snapshots::.
• Each snapshot should invoke wiggle::from_witx! with ctx: crate::WasiCtx in its module, and impl all of the required traits.
• Snapshots can be implemented in terms of other snapshots. For example, snapshot 0 is mostly implemented by calling the snapshot 1 implementation, and converting its own types back and forth with the snapshot 1 types. In a few cases, that is not feasible, so snapshot 0 carries its own implementations in terms of the WasiFile and WasiSched traits.
• Snapshots can be implemented in terms of the Wasi* traits given by WasiCtx. No further downcasting via the as_any escape hatch is permitted.

The wasi-cap-std-sync implementation of the WASI traits

The wasi-cap-std-sync crate provides impl of WasiFile and WasiDir in terms of cap_std::fs::{File, Dir}. These types provide sandboxed access to the local filesystem on both Unix and Windows.

The entire design story of cap-std is much bigger than we can address here, but in short, its functionality replaces all of the wasi_common::sys hierarchy, as well as the yanix / winx crates. All syscalls are hidden behind the cap-std hierarchy, with the lone exception of the sched implementation, which is provided for both unix and windows in separate modules.

Any wasi_common::{WasiCtx, WasiCtxBuilder} is interoperable with the wasi-cap-std-sync crate. However, for convenience, wasi-cap-std-sync provides its own WasiCtxBuilder that hooks up to all of the crate's components, i.e. it fills in all of the arguments to WasiCtx::builder(...), presents preopen_dir in terms of cap_std::fs::Dir, and provides convenience methods for inheriting the parent process's stdio, args, and env.

The only place we expect to run into long-term compatibility issues between wasi-cap-std-sync and the other impl crates that will come later is in the Sched abstraction. Once we can build an async scheduler based on Rust Futures, async impls will be able to interoperate, but the synchronous scheduler depends on downcasting the WasiFile type down to concrete types it knows about (which in turn impl AsRawFd for passing to unix poll, or the analogous traits on windows).

Why is this impl suffixed with -sync? https://github.com/bytecodealliance/wasmtime/pull/2434 async is coming soon! The async impl may end up depending on tokio or other relatively heavy deps, so we will retain a sync implementation so that wasi-common users have an option of not pulling in an async runtime.

Improving the WASI test suite

The bulk of wasi-common's test suite lives at crates/test-programs. This test suite was extremely useful for guiding this rewrite. We improved the test suite in numerous ways, including

• Breaking tests out into smaller units, so that failing behavior can be isolated more easily
• Introducing a scheme which allows the test runner to describe to the wasm code what behaviors to expect of the embedding. Behaviors which vary between platforms (e.g. fd_allcoate is impossible to faithfully implement on Windows and is not provided by MacOS) are communicated to the guest by environment variables, which in turn is available in the guest behind the global TESTCONFIG struct.
• Introducing the assert_errno! macro, which pretty-prints errnos by name rather than number in the error message. assert_errno! integrates with TESTCONFIG to specify which errno is expected on which platform. This allows the test source to describe the full set of acceptable errnos a call may return (all possible errnos will be acceptable if no ERRNO_MODE_* env var is set), and also detect regressions on any given platform.
• We still use crates/test-platforms/build.rs to generate the test suite, and use functions directly in build.rs to describe the TESTCONFIG expectations, as well as which tests are to be ignored (expected to fail due to regressions) on what platforms.
• The test suite has been generalized to support multiple backends to the wasi-common crate. This PR will only land with support for testing wasi-cap-std-sync, but it will be easy to add more runtimes under tests/wasm_tests/runtime/ as more impls are created, with a minimum of boilerplate in build.rs.

Changes to wiggle-wasmtime and dependents

You used to pass a WasiCtx (or whatever ctx type) directly to the Wasi struct generated by wiggle-wasmtime, and the generated code would wrap the ctx into a Rc<RefCell<ctx>>. This got in the way of supporting multiple snapshots simultaneously, so now callers of the generated code have to pass in an Rc<RefCell<ctx>>.

I made mechanical changes to wasi-nn (cc @abrown) and wasi-crypto (cc @jedisct1) to accommodate these changes. This was of particular help to the wasi-crypto code - I was able to erase an entire Rc<> indirection by getting wiggle out of our way. Sorry that I did not take the time to break this out into a standalone PR.

Changes to wasmtime-wasi

wasmtime-wasi now supports using multiple snapshots to interface to the same WasiCtx!

• `wasmtime_wa
  [message truncated]

Wasmtime GitHub notifications bot (Feb 02 2021 at 01:44):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Note that this is a breaking change for all users of WASI with Wasmtime, except for the C API.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR).

Traits for the rest of WASI's features

Other aspects of a WASI implementation are not yet considered resources and accessed by handle. We plan to correct this design deficiency in WASI in the future, but for now we have designed the following traits to provide embedders with the same sort of implementation flexibility they get with WasiFile/WasiDir:

• Timekeeping: clocks::WasiSystemClock and clock::WasiMonotonicClock provide the two interfaces for a clock. WasiSystemClock represents time as a cap_std::time::SystemTime, and WasiMonotonicClock represents time as cap_std::time::Instant.
• Randomness: we re-use the cap_rand::RngCore trait to represent a randomness source. A trivial Deterministic impl is provided.
• Scheduling: The WasiSched trait abstracts over the sched_yield and poll_oneoff functions.

Users can provide implementations of each of these interfaces to the WasiCtx::builder(...) function. The wasi_cap_std_sync::WasiCtxBuilder::new() function uses this public interface to plug in its own implementations of each of these resources.

Snapshot architecture

One goal we've had for a while, but not quite met, is for multiple WASI snapshots to provide an interface to the same underlying WasiCtx. This provides us a path to evolve WASI by allowing the same WASI Command to import functions from different snapshots - e.g. the user could use Rust's std which imports snapshot 1, but also depend directly on the wasi crate which imports some future snapshot 2. Right now, this amounts to supporting snapshot 1 and "snapshot 0" aka wasi_unstable at once.

The architectural rules for snapshots are:

• Snapshots are arranged into modules under crate::snapshots::.
• Each snapshot should invoke wiggle::from_witx! with ctx: crate::WasiCtx in its module, and impl all of the required traits.
• Snapshots can be implemented in terms of other snapshots. For example, snapshot 0 is mostly implemented by calling the snapshot 1 implementation, and converting its own types back and forth with the snapshot 1 types. In a few cases, that is not feasible, so snapshot 0 carries its own implementations in terms of the WasiFile and WasiSched traits.
• Snapshots can be implemented in terms of the Wasi* traits given by WasiCtx. No further downcasting via the as_any escape hatch is permitted.

The wasi-cap-std-sync implementation of the WASI traits

The wasi-cap-std-sync crate provides impl of WasiFile and WasiDir in terms of cap_std::fs::{File, Dir}. These types provide sandboxed access to the local filesystem on both Unix and Windows.

The entire design story of cap-std is much bigger than we can address here, but in short, its functionality replaces all of the wasi_common::sys hierarchy, as well as the yanix / winx crates. All syscalls are hidden behind the cap-std hierarchy, with the lone exception of the sched implementation, which is provided for both unix and windows in separate modules.

Any wasi_common::{WasiCtx, WasiCtxBuilder} is interoperable with the wasi-cap-std-sync crate. However, for convenience, wasi-cap-std-sync provides its own WasiCtxBuilder that hooks up to all of the crate's components, i.e. it fills in all of the arguments to WasiCtx::builder(...), presents preopen_dir in terms of cap_std::fs::Dir, and provides convenience methods for inheriting the parent process's stdio, args, and env.

The only place we expect to run into long-term compatibility issues between wasi-cap-std-sync and the other impl crates that will come later is in the Sched abstraction. Once we can build an async scheduler based on Rust Futures, async impls will be able to interoperate, but the synchronous scheduler depends on downcasting the WasiFile type down to concrete types it knows about (which in turn impl AsRawFd for passing to unix poll, or the analogous traits on windows).

Why is this impl suffixed with -sync? https://github.com/bytecodealliance/wasmtime/pull/2434 async is coming soon! The async impl may end up depending on tokio or other relatively heavy deps, so we will retain a sync implementation so that wasi-common users have an option of not pulling in an async runtime.

Improving the WASI test suite

The bulk of wasi-common's test suite lives at crates/test-programs. This test suite was extremely useful for guiding this rewrite. We improved the test suite in numerous ways, including

• Breaking tests out into smaller units, so that failing behavior can be isolated more easily
• Introducing a scheme which allows the test runner to describe to the wasm code what behaviors to expect of the embedding. Behaviors which vary between platforms (e.g. fd_allcoate is impossible to faithfully implement on Windows and is not provided by MacOS) are communicated to the guest by environment variables, which in turn is available in the guest behind the global TESTCONFIG struct.
• Introducing the assert_errno! macro, which pretty-prints errnos by name rather than number in the error message. assert_errno! integrates with TESTCONFIG to specify which errno is expected on which platform. This allows the test source to describe the full set of acceptable errnos a call may return (all possible errnos will be acceptable if no ERRNO_MODE_* env var is set), and also detect regressions on any given platform.
• We still use crates/test-platforms/build.rs to generate the test suite, and use functions directly in build.rs to describe the TESTCONFIG expectations, as well as which tests are to be ignored (expected to fail due to regressions) on what platforms.
• The test suite has been generalized to support multiple backends to the wasi-common crate. This PR will only land with support for testing wasi-cap-std-sync, but it will be easy to add more runtimes under tests/wasm_tests/runtime/ as more impls are created, with a minimum of boilerplate in build.rs.

Changes to wiggle-wasmtime and dependents

You used to pass a WasiCtx (or whatever ctx type) directly to the Wasi struct generated by wiggle-wasmtime, and the generated code would wrap the ctx into a Rc<RefCell<ctx>>. This got in the way of supporting multiple snapshots simultaneously, so now callers of the generated code have to pass in an Rc<RefCell<ctx>>.

I made mechanical changes to wasi-nn (cc @abrown) and wasi-crypto (cc @jedisct1) to accommodate these changes. This was of particular help to the wasi-crypto code - I was able to erase an entire Rc<> indirection by getting wiggle out of our way. Sorry that I did not take the time to break this out into a standalone PR.

Changes to wasmtime-wasi

wasmtime-wasi now supports using multiple snapshots to interface to the same WasiCtx!

• `wasmtime_w
  [message truncated]

Wasmtime GitHub notifications bot (Feb 02 2021 at 01:46):

pchickey edited PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Note that this is a breaking change for all users of WASI with Wasmtime, except for the C API.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR).

Traits for the rest of WASI's features

Other aspects of a WASI implementation are not yet considered resources and accessed by handle. We plan to correct this design deficiency in WASI in the future, but for now we have designed the following traits to provide embedders with the same sort of implementation flexibility they get with WasiFile/WasiDir:

• Timekeeping: clocks::WasiSystemClock and clock::WasiMonotonicClock provide the two interfaces for a clock. WasiSystemClock represents time as a cap_std::time::SystemTime, and WasiMonotonicClock represents time as cap_std::time::Instant.
• Randomness: we re-use the cap_rand::RngCore trait to represent a randomness source. A trivial Deterministic impl is provided.
• Scheduling: The WasiSched trait abstracts over the sched_yield and poll_oneoff functions.

Users can provide implementations of each of these interfaces to the WasiCtx::builder(...) function. The wasi_cap_std_sync::WasiCtxBuilder::new() function uses this public interface to plug in its own implementations of each of these resources.

Snapshot architecture

One goal we've had for a while, but not quite met, is for multiple WASI snapshots to provide an interface to the same underlying WasiCtx. This provides us a path to evolve WASI by allowing the same WASI Command to import functions from different snapshots - e.g. the user could use Rust's std which imports snapshot 1, but also depend directly on the wasi crate which imports some future snapshot 2. Right now, this amounts to supporting snapshot 1 and "snapshot 0" aka wasi_unstable at once.

The architectural rules for snapshots are:

• Snapshots are arranged into modules under crate::snapshots::.
• Each snapshot should invoke wiggle::from_witx! with ctx: crate::WasiCtx in its module, and impl all of the required traits.
• Snapshots can be implemented in terms of other snapshots. For example, snapshot 0 is mostly implemented by calling the snapshot 1 implementation, and converting its own types back and forth with the snapshot 1 types. In a few cases, that is not feasible, so snapshot 0 carries its own implementations in terms of the WasiFile and WasiSched traits.
• Snapshots can be implemented in terms of the Wasi* traits given by WasiCtx. No further downcasting via the as_any escape hatch is permitted.

The wasi_common::Error type

wasi_common::Error is now anyhow::Error. wasi_common::snapshots::preview_1 contains all of the logic for transforming an Error into an Errno, by downcasting the error into any of

• std::io::Error
• wasi_common::ErrorKind
• wiggle::GuestError
• std::num::TryFromIntError
• std::str::Utf8Error
  and then applying specialized logic to translate each of those into Errnos.

If you throw an error that does not downcast to one of those, it will turn into a wiggle::Trap and terminate execution.

The wasi-cap-std-sync implementation of the WASI traits

The wasi-cap-std-sync crate provides impl of WasiFile and WasiDir in terms of cap_std::fs::{File, Dir}. These types provide sandboxed access to the local filesystem on both Unix and Windows.

The entire design story of cap-std is much bigger than we can address here, but in short, its functionality replaces all of the wasi_common::sys hierarchy, as well as the yanix / winx crates. All syscalls are hidden behind the cap-std hierarchy, with the lone exception of the sched implementation, which is provided for both unix and windows in separate modules.

Any wasi_common::{WasiCtx, WasiCtxBuilder} is interoperable with the wasi-cap-std-sync crate. However, for convenience, wasi-cap-std-sync provides its own WasiCtxBuilder that hooks up to all of the crate's components, i.e. it fills in all of the arguments to WasiCtx::builder(...), presents preopen_dir in terms of cap_std::fs::Dir, and provides convenience methods for inheriting the parent process's stdio, args, and env.

The only place we expect to run into long-term compatibility issues between wasi-cap-std-sync and the other impl crates that will come later is in the Sched abstraction. Once we can build an async scheduler based on Rust Futures, async impls will be able to interoperate, but the synchronous scheduler depends on downcasting the WasiFile type down to concrete types it knows about (which in turn impl AsRawFd for passing to unix poll, or the analogous traits on windows).

Why is this impl suffixed with -sync? https://github.com/bytecodealliance/wasmtime/pull/2434 async is coming soon! The async impl may end up depending on tokio or other relatively heavy deps, so we will retain a sync implementation so that wasi-common users have an option of not pulling in an async runtime.

Improving the WASI test suite

The bulk of wasi-common's test suite lives at crates/test-programs. This test suite was extremely useful for guiding this rewrite. We improved the test suite in numerous ways, including

• Breaking tests out into smaller units, so that failing behavior can be isolated more easily
• Introducing a scheme which allows the test runner to describe to the wasm code what behaviors to expect of the embedding. Behaviors which vary between platforms (e.g. fd_allcoate is impossible to faithfully implement on Windows and is not provided by MacOS) are communicated to the guest by environment variables, which in turn is available in the guest behind the global TESTCONFIG struct.
• Introducing the assert_errno! macro, which pretty-prints errnos by name rather than number in the error message. assert_errno! integrates with TESTCONFIG to specify which errno is expected on which platform. This allows the test source to describe the full set of acceptable errnos a call may return (all possible errnos will be acceptable if no ERRNO_MODE_* env var is set), and also detect regressions on any given platform.
• We still use crates/test-platforms/build.rs to generate the test suite, and use functions directly in build.rs to describe the TESTCONFIG expectations, as well as which tests are to be ignored (expected to fail due to regressions) on what platforms.
• The test suite has been generalized to support multiple backends to the wasi-common crate. This PR will only land with support for testing wasi-cap-std-sync, but it will be easy to add more runtimes under tests/wasm_tests/runtime/ as more impls are created, with a minimum of boilerplate in build.rs.

Changes to wiggle-wasmtime and dependents

You used to pass a WasiCtx (or whatever ctx type) directly to the Wasi struct generated by wiggle-wasmtime, and the generated code would wrap the ctx into a Rc<RefCell<ctx>>. This got in the way of supporting multiple snapshots simul
[message truncated]

Wasmtime GitHub notifications bot (Feb 02 2021 at 01:48):

pchickey edited PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Note that this is a breaking change for all users of WASI with Wasmtime, except for the C API.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR).

Traits for the rest of WASI's features

Other aspects of a WASI implementation are not yet considered resources and accessed by handle. We plan to correct this design deficiency in WASI in the future, but for now we have designed the following traits to provide embedders with the same sort of implementation flexibility they get with WasiFile/WasiDir:

• Timekeeping: clocks::WasiSystemClock and clock::WasiMonotonicClock provide the two interfaces for a clock. WasiSystemClock represents time as a cap_std::time::SystemTime, and WasiMonotonicClock represents time as cap_std::time::Instant.
• Randomness: we re-use the cap_rand::RngCore trait to represent a randomness source. A trivial Deterministic impl is provided.
• Scheduling: The WasiSched trait abstracts over the sched_yield and poll_oneoff functions.

Users can provide implementations of each of these interfaces to the WasiCtx::builder(...) function. The wasi_cap_std_sync::WasiCtxBuilder::new() function uses this public interface to plug in its own implementations of each of these resources.

Snapshot architecture

One goal we've had for a while, but not quite met, is for multiple WASI snapshots to provide an interface to the same underlying WasiCtx. This provides us a path to evolve WASI by allowing the same WASI Command to import functions from different snapshots - e.g. the user could use Rust's std which imports snapshot 1, but also depend directly on the wasi crate which imports some future snapshot 2. Right now, this amounts to supporting snapshot 1 and "snapshot 0" aka wasi_unstable at once.

The architectural rules for snapshots are:

• Snapshots are arranged into modules under crate::snapshots::.
• Each snapshot should invoke wiggle::from_witx! with ctx: crate::WasiCtx in its module, and impl all of the required traits.
• Snapshots can be implemented in terms of other snapshots. For example, snapshot 0 is mostly implemented by calling the snapshot 1 implementation, and converting its own types back and forth with the snapshot 1 types. In a few cases, that is not feasible, so snapshot 0 carries its own implementations in terms of the WasiFile and WasiSched traits.
• Snapshots can be implemented in terms of the Wasi* traits given by WasiCtx. No further downcasting via the as_any escape hatch is permitted.

The wasi_common::Error type

wasi_common::Error is now anyhow::Error. wasi_common::snapshots::preview_1 contains all of the logic for transforming an Error into an Errno, by downcasting the error into any of

• std::io::Error - these are thrown by std, cap_std, etc for most of the operations WASI is concerned with.
• wasi_common::ErrorKind - these are a subset of the Errnos, and are constructed directly by wasi-common or an impl rather than coming from the OS or some library which doesn't know about WASI.
• wiggle::GuestError
• std::num::TryFromIntError
• std::str::Utf8Error
  and then applying specialized logic to translate each of those into Errnos.

The wasi_common::ErrorExt trait provides human-friendly constructors for the wasi_common::ErrorKind variants

If you throw an error that does not downcast to one of those, it will turn into a wiggle::Trap and terminate execution.

The wasi-cap-std-sync implementation of the WASI traits

The wasi-cap-std-sync crate provides impl of WasiFile and WasiDir in terms of cap_std::fs::{File, Dir}. These types provide sandboxed access to the local filesystem on both Unix and Windows.

The entire design story of cap-std is much bigger than we can address here, but in short, its functionality replaces all of the wasi_common::sys hierarchy, as well as the yanix / winx crates. All syscalls are hidden behind the cap-std hierarchy, with the lone exception of the sched implementation, which is provided for both unix and windows in separate modules.

Any wasi_common::{WasiCtx, WasiCtxBuilder} is interoperable with the wasi-cap-std-sync crate. However, for convenience, wasi-cap-std-sync provides its own WasiCtxBuilder that hooks up to all of the crate's components, i.e. it fills in all of the arguments to WasiCtx::builder(...), presents preopen_dir in terms of cap_std::fs::Dir, and provides convenience methods for inheriting the parent process's stdio, args, and env.

The only place we expect to run into long-term compatibility issues between wasi-cap-std-sync and the other impl crates that will come later is in the Sched abstraction. Once we can build an async scheduler based on Rust Futures, async impls will be able to interoperate, but the synchronous scheduler depends on downcasting the WasiFile type down to concrete types it knows about (which in turn impl AsRawFd for passing to unix poll, or the analogous traits on windows).

Why is this impl suffixed with -sync? https://github.com/bytecodealliance/wasmtime/pull/2434 async is coming soon! The async impl may end up depending on tokio or other relatively heavy deps, so we will retain a sync implementation so that wasi-common users have an option of not pulling in an async runtime.

Improving the WASI test suite

The bulk of wasi-common's test suite lives at crates/test-programs. This test suite was extremely useful for guiding this rewrite. We improved the test suite in numerous ways, including

• Breaking tests out into smaller units, so that failing behavior can be isolated more easily
• Introducing a scheme which allows the test runner to describe to the wasm code what behaviors to expect of the embedding. Behaviors which vary between platforms (e.g. fd_allcoate is impossible to faithfully implement on Windows and is not provided by MacOS) are communicated to the guest by environment variables, which in turn is available in the guest behind the global TESTCONFIG struct.
• Introducing the assert_errno! macro, which pretty-prints errnos by name rather than number in the error message. assert_errno! integrates with TESTCONFIG to specify which errno is expected on which platform. This allows the test source to describe the full set of acceptable errnos a call may return (all possible errnos will be acceptable if no ERRNO_MODE_* env var is set), and also detect regressions on any given platform.
• We still use crates/test-platforms/build.rs to generate the test suite, and use functions directly in build.rs to describe the TESTCONFIG expectations, as well as which tests are to be ignored (expected to fail due to regressions) on what platforms.
• The test suite has been generalized to support multiple backends to the wasi-common crate. This PR will only land with support for testing wasi-cap-std-sync, but it will be easy to add more runtimes under `tests/wasm_
  [message truncated]

Wasmtime GitHub notifications bot (Feb 02 2021 at 01:49):

pchickey edited PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Note that this is a breaking change for all users of WASI with Wasmtime, except for the C API.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR).

Traits for the rest of WASI's features

Other aspects of a WASI implementation are not yet considered resources and accessed by handle. We plan to correct this design deficiency in WASI in the future, but for now we have designed the following traits to provide embedders with the same sort of implementation flexibility they get with WasiFile/WasiDir:

• Timekeeping: clocks::WasiSystemClock and clock::WasiMonotonicClock provide the two interfaces for a clock. WasiSystemClock represents time as a cap_std::time::SystemTime, and WasiMonotonicClock represents time as cap_std::time::Instant.
• Randomness: we re-use the cap_rand::RngCore trait to represent a randomness source. A trivial Deterministic impl is provided.
• Scheduling: The WasiSched trait abstracts over the sched_yield and poll_oneoff functions.

Users can provide implementations of each of these interfaces to the WasiCtx::builder(...) function. The wasi_cap_std_sync::WasiCtxBuilder::new() function uses this public interface to plug in its own implementations of each of these resources.

Snapshot architecture

One goal we've had for a while, but not quite met, is for multiple WASI snapshots to provide an interface to the same underlying WasiCtx. This provides us a path to evolve WASI by allowing the same WASI Command to import functions from different snapshots - e.g. the user could use Rust's std which imports snapshot 1, but also depend directly on the wasi crate which imports some future snapshot 2. Right now, this amounts to supporting snapshot 1 and "snapshot 0" aka wasi_unstable at once.

The architectural rules for snapshots are:

• Snapshots are arranged into modules under crate::snapshots::.
• Each snapshot should invoke wiggle::from_witx! with ctx: crate::WasiCtx in its module, and impl all of the required traits.
• Snapshots can be implemented in terms of other snapshots. For example, snapshot 0 is mostly implemented by calling the snapshot 1 implementation, and converting its own types back and forth with the snapshot 1 types. In a few cases, that is not feasible, so snapshot 0 carries its own implementations in terms of the WasiFile and WasiSched traits.
• Snapshots can be implemented in terms of the Wasi* traits given by WasiCtx. No further downcasting via the as_any escape hatch is permitted.

The wasi_common::Error type

wasi_common::Error is now anyhow::Error. wasi_common::snapshots::preview_1 contains all of the logic for transforming an Error into an Errno, by downcasting the error into any of

• std::io::Error - these are thrown by std, cap_std, etc for most of the operations WASI is concerned with.
• wasi_common::ErrorKind - these are a subset of the Errnos, and are constructed directly by wasi-common or an impl rather than coming from the OS or some library which doesn't know about WASI.
• wiggle::GuestError
• std::num::TryFromIntError
• std::str::Utf8Error
  and then applying specialized logic to translate each of those into Errnos.

The wasi_common::ErrorExt trait provides human-friendly constructors for the wasi_common::ErrorKind variants .

If you throw an error that does not downcast to one of those, it will turn into a wiggle::Trap and terminate execution.

The real value of using anyhow::Error here is being able to use anyhow::Result::context to aid in debugging of errors.

The wasi-cap-std-sync implementation of the WASI traits

The wasi-cap-std-sync crate provides impl of WasiFile and WasiDir in terms of cap_std::fs::{File, Dir}. These types provide sandboxed access to the local filesystem on both Unix and Windows.

The entire design story of cap-std is much bigger than we can address here, but in short, its functionality replaces all of the wasi_common::sys hierarchy, as well as the yanix / winx crates. All syscalls are hidden behind the cap-std hierarchy, with the lone exception of the sched implementation, which is provided for both unix and windows in separate modules.

Any wasi_common::{WasiCtx, WasiCtxBuilder} is interoperable with the wasi-cap-std-sync crate. However, for convenience, wasi-cap-std-sync provides its own WasiCtxBuilder that hooks up to all of the crate's components, i.e. it fills in all of the arguments to WasiCtx::builder(...), presents preopen_dir in terms of cap_std::fs::Dir, and provides convenience methods for inheriting the parent process's stdio, args, and env.

The only place we expect to run into long-term compatibility issues between wasi-cap-std-sync and the other impl crates that will come later is in the Sched abstraction. Once we can build an async scheduler based on Rust Futures, async impls will be able to interoperate, but the synchronous scheduler depends on downcasting the WasiFile type down to concrete types it knows about (which in turn impl AsRawFd for passing to unix poll, or the analogous traits on windows).

Why is this impl suffixed with -sync? https://github.com/bytecodealliance/wasmtime/pull/2434 async is coming soon! The async impl may end up depending on tokio or other relatively heavy deps, so we will retain a sync implementation so that wasi-common users have an option of not pulling in an async runtime.

Improving the WASI test suite

The bulk of wasi-common's test suite lives at crates/test-programs. This test suite was extremely useful for guiding this rewrite. We improved the test suite in numerous ways, including

• Breaking tests out into smaller units, so that failing behavior can be isolated more easily
• Introducing a scheme which allows the test runner to describe to the wasm code what behaviors to expect of the embedding. Behaviors which vary between platforms (e.g. fd_allcoate is impossible to faithfully implement on Windows and is not provided by MacOS) are communicated to the guest by environment variables, which in turn is available in the guest behind the global TESTCONFIG struct.
• Introducing the assert_errno! macro, which pretty-prints errnos by name rather than number in the error message. assert_errno! integrates with TESTCONFIG to specify which errno is expected on which platform. This allows the test source to describe the full set of acceptable errnos a call may return (all possible errnos will be acceptable if no ERRNO_MODE_* env var is set), and also detect regressions on any given platform.
• We still use crates/test-platforms/build.rs to generate the test suite, and use functions directly in build.rs to describe the TESTCONFIG expectations, as well as which tests are to be ignored (expected to fail due to regressions) on what platforms.
• The test suite has been generalized to support multiple backends to the wasi-common crate. T
  [message truncated]

Wasmtime GitHub notifications bot (Feb 02 2021 at 02:22):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Note that this is a breaking change for all users of WASI with Wasmtime, except for the C API.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR).

Traits for the rest of WASI's features

Other aspects of a WASI implementation are not yet considered resources and accessed by handle. We plan to correct this design deficiency in WASI in the future, but for now we have designed the following traits to provide embedders with the same sort of implementation flexibility they get with WasiFile/WasiDir:

• Timekeeping: clocks::WasiSystemClock and clock::WasiMonotonicClock provide the two interfaces for a clock. WasiSystemClock represents time as a cap_std::time::SystemTime, and WasiMonotonicClock represents time as cap_std::time::Instant.
• Randomness: we re-use the cap_rand::RngCore trait to represent a randomness source. A trivial Deterministic impl is provided.
• Scheduling: The WasiSched trait abstracts over the sched_yield and poll_oneoff functions.

Users can provide implementations of each of these interfaces to the WasiCtx::builder(...) function. The wasi_cap_std_sync::WasiCtxBuilder::new() function uses this public interface to plug in its own implementations of each of these resources.

Snapshot architecture

One goal we've had for a while, but not quite met, is for multiple WASI snapshots to provide an interface to the same underlying WasiCtx. This provides us a path to evolve WASI by allowing the same WASI Command to import functions from different snapshots - e.g. the user could use Rust's std which imports snapshot 1, but also depend directly on the wasi crate which imports some future snapshot 2. Right now, this amounts to supporting snapshot 1 and "snapshot 0" aka wasi_unstable at once.

The architectural rules for snapshots are:

• Snapshots are arranged into modules under crate::snapshots::.
• Each snapshot should invoke wiggle::from_witx! with ctx: crate::WasiCtx in its module, and impl all of the required traits.
• Snapshots can be implemented in terms of other snapshots. For example, snapshot 0 is mostly implemented by calling the snapshot 1 implementation, and converting its own types back and forth with the snapshot 1 types. In a few cases, that is not feasible, so snapshot 0 carries its own implementations in terms of the WasiFile and WasiSched traits.
• Snapshots can be implemented in terms of the Wasi* traits given by WasiCtx. No further downcasting via the as_any escape hatch is permitted.

The wasi_common::Error type

wasi_common::Error is now anyhow::Error. wasi_common::snapshots::preview_1 contains all of the logic for transforming an Error into an Errno, by downcasting the error into any of

• std::io::Error - these are thrown by std, cap_std, etc for most of the operations WASI is concerned with.
• wasi_common::ErrorKind - these are a subset of the Errnos, and are constructed directly by wasi-common or an impl rather than coming from the OS or some library which doesn't know about WASI.
• wiggle::GuestError
• std::num::TryFromIntError
• std::str::Utf8Error
  and then applying specialized logic to translate each of those into Errnos.

The wasi_common::ErrorExt trait provides human-friendly constructors for the wasi_common::ErrorKind variants .

If you throw an error that does not downcast to one of those, it will turn into a wiggle::Trap and terminate execution.

The real value of using anyhow::Error here is being able to use anyhow::Result::context to aid in debugging of errors.

The wasi-cap-std-sync implementation of the WASI traits

The wasi-cap-std-sync crate provides impl of WasiFile and WasiDir in terms of cap_std::fs::{File, Dir}. These types provide sandboxed access to the local filesystem on both Unix and Windows.

The entire design story of cap-std is much bigger than we can address here, but in short, its functionality replaces all of the wasi_common::sys hierarchy, as well as the yanix / winx crates. All syscalls are hidden behind the cap-std hierarchy, with the lone exception of the sched implementation, which is provided for both unix and windows in separate modules.

Any wasi_common::{WasiCtx, WasiCtxBuilder} is interoperable with the wasi-cap-std-sync crate. However, for convenience, wasi-cap-std-sync provides its own WasiCtxBuilder that hooks up to all of the crate's components, i.e. it fills in all of the arguments to WasiCtx::builder(...), presents preopen_dir in terms of cap_std::fs::Dir, and provides convenience methods for inheriting the parent process's stdio, args, and env.

The only place we expect to run into long-term compatibility issues between wasi-cap-std-sync and the other impl crates that will come later is in the Sched abstraction. Once we can build an async scheduler based on Rust Futures, async impls will be able to interoperate, but the synchronous scheduler depends on downcasting the WasiFile type down to concrete types it knows about (which in turn impl AsRawFd for passing to unix poll, or the analogous traits on windows).

Why is this impl suffixed with -sync? https://github.com/bytecodealliance/wasmtime/pull/2434 async is coming soon! The async impl may end up depending on tokio or other relatively heavy deps, so we will retain a sync implementation so that wasi-common users have an option of not pulling in an async runtime.

Improving the WASI test suite

The bulk of wasi-common's test suite lives at crates/test-programs. This test suite was extremely useful for guiding this rewrite. We improved the test suite in numerous ways, including

• Breaking tests out into smaller units, so that failing behavior can be isolated more easily
• Introducing a scheme which allows the test runner to describe to the wasm code what behaviors to expect of the embedding. Behaviors which vary between platforms (e.g. fd_allcoate is impossible to faithfully implement on Windows and is not provided by MacOS) are communicated to the guest by environment variables, which in turn is available in the guest behind the global TESTCONFIG struct.
• Introducing the assert_errno! macro, which pretty-prints errnos by name rather than number in the error message. assert_errno! integrates with TESTCONFIG to specify which errno is expected on which platform. This allows the test source to describe the full set of acceptable errnos a call may return (all possible errnos will be acceptable if no ERRNO_MODE_* env var is set), and also detect regressions on any given platform.
• We still use crates/test-platforms/build.rs to generate the test suite, and use functions directly in build.rs to describe the TESTCONFIG expectations, as well as which tests are to be ignored (expected to fail due to regressions) on what platforms.
• The test suite has been generalized to support multiple backends to the wasi-common crate.
  [message truncated]

Wasmtime GitHub notifications bot (Feb 02 2021 at 02:31):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Note that this is a breaking change for all users of WASI with Wasmtime, except for the C API.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR).

Traits for the rest of WASI's features

Other aspects of a WASI implementation are not yet considered resources and accessed by handle. We plan to correct this design deficiency in WASI in the future, but for now we have designed the following traits to provide embedders with the same sort of implementation flexibility they get with WasiFile/WasiDir:

• Timekeeping: clocks::WasiSystemClock and clock::WasiMonotonicClock provide the two interfaces for a clock. WasiSystemClock represents time as a cap_std::time::SystemTime, and WasiMonotonicClock represents time as cap_std::time::Instant.
• Randomness: we re-use the cap_rand::RngCore trait to represent a randomness source. A trivial Deterministic impl is provided.
• Scheduling: The WasiSched trait abstracts over the sched_yield and poll_oneoff functions.

Users can provide implementations of each of these interfaces to the WasiCtx::builder(...) function. The wasi_cap_std_sync::WasiCtxBuilder::new() function uses this public interface to plug in its own implementations of each of these resources.

Snapshot architecture

One goal we've had for a while, but not quite met, is for multiple WASI snapshots to provide an interface to the same underlying WasiCtx. This provides us a path to evolve WASI by allowing the same WASI Command to import functions from different snapshots - e.g. the user could use Rust's std which imports snapshot 1, but also depend directly on the wasi crate which imports some future snapshot 2. Right now, this amounts to supporting snapshot 1 and "snapshot 0" aka wasi_unstable at once.

The architectural rules for snapshots are:

• Snapshots are arranged into modules under crate::snapshots::.
• Each snapshot should invoke wiggle::from_witx! with ctx: crate::WasiCtx in its module, and impl all of the required traits.
• Snapshots can be implemented in terms of other snapshots. For example, snapshot 0 is mostly implemented by calling the snapshot 1 implementation, and converting its own types back and forth with the snapshot 1 types. In a few cases, that is not feasible, so snapshot 0 carries its own implementations in terms of the WasiFile and WasiSched traits.
• Snapshots can be implemented in terms of the Wasi* traits given by WasiCtx. No further downcasting via the as_any escape hatch is permitted.

The wasi_common::Error type

wasi_common::Error is now anyhow::Error. wasi_common::snapshots::preview_1 contains all of the logic for transforming an Error into an Errno, by downcasting the error into any of

• std::io::Error - these are thrown by std, cap_std, etc for most of the operations WASI is concerned with.
• wasi_common::ErrorKind - these are a subset of the Errnos, and are constructed directly by wasi-common or an impl rather than coming from the OS or some library which doesn't know about WASI.
• wiggle::GuestError
• std::num::TryFromIntError
• std::str::Utf8Error
  and then applying specialized logic to translate each of those into Errnos.

The wasi_common::ErrorExt trait provides human-friendly constructors for the wasi_common::ErrorKind variants .

If you throw an error that does not downcast to one of those, it will turn into a wiggle::Trap and terminate execution.

The real value of using anyhow::Error here is being able to use anyhow::Result::context to aid in debugging of errors.

The wasi-cap-std-sync implementation of the WASI traits

The wasi-cap-std-sync crate provides impl of WasiFile and WasiDir in terms of cap_std::fs::{File, Dir}. These types provide sandboxed access to the local filesystem on both Unix and Windows.

The entire design story of cap-std is much bigger than we can address here, but in short, its functionality replaces all of the wasi_common::sys hierarchy, as well as the yanix / winx crates. All syscalls are hidden behind the cap-std hierarchy, with the lone exception of the sched implementation, which is provided for both unix and windows in separate modules.

Any wasi_common::{WasiCtx, WasiCtxBuilder} is interoperable with the wasi-cap-std-sync crate. However, for convenience, wasi-cap-std-sync provides its own WasiCtxBuilder that hooks up to all of the crate's components, i.e. it fills in all of the arguments to WasiCtx::builder(...), presents preopen_dir in terms of cap_std::fs::Dir, and provides convenience methods for inheriting the parent process's stdio, args, and env.

The only place we expect to run into long-term compatibility issues between wasi-cap-std-sync and the other impl crates that will come later is in the Sched abstraction. Once we can build an async scheduler based on Rust Futures, async impls will be able to interoperate, but the synchronous scheduler depends on downcasting the WasiFile type down to concrete types it knows about (which in turn impl AsRawFd for passing to unix poll, or the analogous traits on windows).

Why is this impl suffixed with -sync? https://github.com/bytecodealliance/wasmtime/pull/2434 async is coming soon! The async impl may end up depending on tokio or other relatively heavy deps, so we will retain a sync implementation so that wasi-common users have an option of not pulling in an async runtime.

Improving the WASI test suite

The bulk of wasi-common's test suite lives at crates/test-programs. This test suite was extremely useful for guiding this rewrite. We improved the test suite in numerous ways, including

• Breaking tests out into smaller units, so that failing behavior can be isolated more easily
• Introducing a scheme which allows the test runner to describe to the wasm code what behaviors to expect of the embedding. Behaviors which vary between platforms (e.g. fd_allcoate is impossible to faithfully implement on Windows and is not provided by MacOS) are communicated to the guest by environment variables, which in turn is available in the guest behind the global TESTCONFIG struct.
• Introducing the assert_errno! macro, which pretty-prints errnos by name rather than number in the error message. assert_errno! integrates with TESTCONFIG to specify which errno is expected on which platform. This allows the test source to describe the full set of acceptable errnos a call may return (all possible errnos will be acceptable if no ERRNO_MODE_* env var is set), and also detect regressions on any given platform.
• We still use crates/test-platforms/build.rs to generate the test suite, and use functions directly in build.rs to describe the TESTCONFIG expectations, as well as which tests are to be ignored (expected to fail due to regressions) on what platforms.
• The test suite has been generalized to support multiple backends to the wasi-common crate.
  [message truncated]

Wasmtime GitHub notifications bot (Feb 02 2021 at 02:38):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Note that this is a breaking change for all users of WASI with Wasmtime, except for the C API.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR).

Traits for the rest of WASI's features

Other aspects of a WASI implementation are not yet considered resources and accessed by handle. We plan to correct this design deficiency in WASI in the future, but for now we have designed the following traits to provide embedders with the same sort of implementation flexibility they get with WasiFile/WasiDir:

• Timekeeping: clocks::WasiSystemClock and clock::WasiMonotonicClock provide the two interfaces for a clock. WasiSystemClock represents time as a cap_std::time::SystemTime, and WasiMonotonicClock represents time as cap_std::time::Instant.
• Randomness: we re-use the cap_rand::RngCore trait to represent a randomness source. A trivial Deterministic impl is provided.
• Scheduling: The WasiSched trait abstracts over the sched_yield and poll_oneoff functions.

Users can provide implementations of each of these interfaces to the WasiCtx::builder(...) function. The wasi_cap_std_sync::WasiCtxBuilder::new() function uses this public interface to plug in its own implementations of each of these resources.

Snapshot architecture

One goal we've had for a while, but not quite met, is for multiple WASI snapshots to provide an interface to the same underlying WasiCtx. This provides us a path to evolve WASI by allowing the same WASI Command to import functions from different snapshots - e.g. the user could use Rust's std which imports snapshot 1, but also depend directly on the wasi crate which imports some future snapshot 2. Right now, this amounts to supporting snapshot 1 and "snapshot 0" aka wasi_unstable at once.

The architectural rules for snapshots are:

• Snapshots are arranged into modules under crate::snapshots::.
• Each snapshot should invoke wiggle::from_witx! with ctx: crate::WasiCtx in its module, and impl all of the required traits.
• Snapshots can be implemented in terms of other snapshots. For example, snapshot 0 is mostly implemented by calling the snapshot 1 implementation, and converting its own types back and forth with the snapshot 1 types. In a few cases, that is not feasible, so snapshot 0 carries its own implementations in terms of the WasiFile and WasiSched traits.
• Snapshots can be implemented in terms of the Wasi* traits given by WasiCtx. No further downcasting via the as_any escape hatch is permitted.

The wasi_common::Error type

wasi_common::Error is now anyhow::Error. wasi_common::snapshots::preview_1 contains all of the logic for transforming an Error into an Errno, by downcasting the error into any of

• std::io::Error - these are thrown by std, cap_std, etc for most of the operations WASI is concerned with.
• wasi_common::ErrorKind - these are a subset of the Errnos, and are constructed directly by wasi-common or an impl rather than coming from the OS or some library which doesn't know about WASI.
• wiggle::GuestError
• std::num::TryFromIntError
• std::str::Utf8Error
  and then applying specialized logic to translate each of those into Errnos.

The wasi_common::ErrorExt trait provides human-friendly constructors for the wasi_common::ErrorKind variants .

If you throw an error that does not downcast to one of those, it will turn into a wiggle::Trap and terminate execution.

The real value of using anyhow::Error here is being able to use anyhow::Result::context to aid in debugging of errors.

The wasi-cap-std-sync implementation of the WASI traits

The wasi-cap-std-sync crate provides impl of WasiFile and WasiDir in terms of cap_std::fs::{File, Dir}. These types provide sandboxed access to the local filesystem on both Unix and Windows.

The entire design story of cap-std is much bigger than we can address here, but in short, its functionality replaces all of the wasi_common::sys hierarchy, as well as the yanix / winx crates. All syscalls are hidden behind the cap-std hierarchy, with the lone exception of the sched implementation, which is provided for both unix and windows in separate modules.

Any wasi_common::{WasiCtx, WasiCtxBuilder} is interoperable with the wasi-cap-std-sync crate. However, for convenience, wasi-cap-std-sync provides its own WasiCtxBuilder that hooks up to all of the crate's components, i.e. it fills in all of the arguments to WasiCtx::builder(...), presents preopen_dir in terms of cap_std::fs::Dir, and provides convenience methods for inheriting the parent process's stdio, args, and env.

The only place we expect to run into long-term compatibility issues between wasi-cap-std-sync and the other impl crates that will come later is in the Sched abstraction. Once we can build an async scheduler based on Rust Futures, async impls will be able to interoperate, but the synchronous scheduler depends on downcasting the WasiFile type down to concrete types it knows about (which in turn impl AsRawFd for passing to unix poll, or the analogous traits on windows).

Why is this impl suffixed with -sync? https://github.com/bytecodealliance/wasmtime/pull/2434 async is coming soon! The async impl may end up depending on tokio or other relatively heavy deps, so we will retain a sync implementation so that wasi-common users have an option of not pulling in an async runtime.

Improving the WASI test suite

The bulk of wasi-common's test suite lives at crates/test-programs. This test suite was extremely useful for guiding this rewrite. We improved the test suite in numerous ways, including

• Breaking tests out into smaller units, so that failing behavior can be isolated more easily
• Introducing a scheme which allows the test runner to describe to the wasm code what behaviors to expect of the embedding. Behaviors which vary between platforms (e.g. fd_allcoate is impossible to faithfully implement on Windows and is not provided by MacOS) are communicated to the guest by environment variables, which in turn is available in the guest behind the global TESTCONFIG struct.
• Introducing the assert_errno! macro, which pretty-prints errnos by name rather than number in the error message. assert_errno! integrates with TESTCONFIG to specify which errno is expected on which platform. This allows the test source to describe the full set of acceptable errnos a call may return (all possible errnos will be acceptable if no ERRNO_MODE_* env var is set), and also detect regressions on any given platform.
• We still use crates/test-platforms/build.rs to generate the test suite, and use functions directly in build.rs to describe the TESTCONFIG expectations, as well as which tests are to be ignored (expected to fail due to regressions) on what platforms.
• The test suite has been generalized to support multiple backends to the wasi-common crate.
  [message truncated]

Wasmtime GitHub notifications bot (Feb 02 2021 at 03:12):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Note that this is a breaking change for all users of WASI with Wasmtime, except for the C API.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR).

Traits for the rest of WASI's features

Other aspects of a WASI implementation are not yet considered resources and accessed by handle. We plan to correct this design deficiency in WASI in the future, but for now we have designed the following traits to provide embedders with the same sort of implementation flexibility they get with WasiFile/WasiDir:

• Timekeeping: clocks::WasiSystemClock and clock::WasiMonotonicClock provide the two interfaces for a clock. WasiSystemClock represents time as a cap_std::time::SystemTime, and WasiMonotonicClock represents time as cap_std::time::Instant.
• Randomness: we re-use the cap_rand::RngCore trait to represent a randomness source. A trivial Deterministic impl is provided.
• Scheduling: The WasiSched trait abstracts over the sched_yield and poll_oneoff functions.

Users can provide implementations of each of these interfaces to the WasiCtx::builder(...) function. The wasi_cap_std_sync::WasiCtxBuilder::new() function uses this public interface to plug in its own implementations of each of these resources.

Snapshot architecture

One goal we've had for a while, but not quite met, is for multiple WASI snapshots to provide an interface to the same underlying WasiCtx. This provides us a path to evolve WASI by allowing the same WASI Command to import functions from different snapshots - e.g. the user could use Rust's std which imports snapshot 1, but also depend directly on the wasi crate which imports some future snapshot 2. Right now, this amounts to supporting snapshot 1 and "snapshot 0" aka wasi_unstable at once.

The architectural rules for snapshots are:

• Snapshots are arranged into modules under crate::snapshots::.
• Each snapshot should invoke wiggle::from_witx! with ctx: crate::WasiCtx in its module, and impl all of the required traits.
• Snapshots can be implemented in terms of other snapshots. For example, snapshot 0 is mostly implemented by calling the snapshot 1 implementation, and converting its own types back and forth with the snapshot 1 types. In a few cases, that is not feasible, so snapshot 0 carries its own implementations in terms of the WasiFile and WasiSched traits.
• Snapshots can be implemented in terms of the Wasi* traits given by WasiCtx. No further downcasting via the as_any escape hatch is permitted.

The wasi_common::Error type

wasi_common::Error is now anyhow::Error. wasi_common::snapshots::preview_1 contains all of the logic for transforming an Error into an Errno, by downcasting the error into any of

• std::io::Error - these are thrown by std, cap_std, etc for most of the operations WASI is concerned with.
• wasi_common::ErrorKind - these are a subset of the Errnos, and are constructed directly by wasi-common or an impl rather than coming from the OS or some library which doesn't know about WASI.
• wiggle::GuestError
• std::num::TryFromIntError
• std::str::Utf8Error
  and then applying specialized logic to translate each of those into Errnos.

The wasi_common::ErrorExt trait provides human-friendly constructors for the wasi_common::ErrorKind variants .

If you throw an error that does not downcast to one of those, it will turn into a wiggle::Trap and terminate execution.

The real value of using anyhow::Error here is being able to use anyhow::Result::context to aid in debugging of errors.

The wasi-cap-std-sync implementation of the WASI traits

The wasi-cap-std-sync crate provides impl of WasiFile and WasiDir in terms of cap_std::fs::{File, Dir}. These types provide sandboxed access to the local filesystem on both Unix and Windows.

The entire design story of cap-std is much bigger than we can address here, but in short, its functionality replaces all of the wasi_common::sys hierarchy, as well as the yanix / winx crates. All syscalls are hidden behind the cap-std hierarchy, with the lone exception of the sched implementation, which is provided for both unix and windows in separate modules.

Any wasi_common::{WasiCtx, WasiCtxBuilder} is interoperable with the wasi-cap-std-sync crate. However, for convenience, wasi-cap-std-sync provides its own WasiCtxBuilder that hooks up to all of the crate's components, i.e. it fills in all of the arguments to WasiCtx::builder(...), presents preopen_dir in terms of cap_std::fs::Dir, and provides convenience methods for inheriting the parent process's stdio, args, and env.

The only place we expect to run into long-term compatibility issues between wasi-cap-std-sync and the other impl crates that will come later is in the Sched abstraction. Once we can build an async scheduler based on Rust Futures, async impls will be able to interoperate, but the synchronous scheduler depends on downcasting the WasiFile type down to concrete types it knows about (which in turn impl AsRawFd for passing to unix poll, or the analogous traits on windows).

Why is this impl suffixed with -sync? https://github.com/bytecodealliance/wasmtime/pull/2434 async is coming soon! The async impl may end up depending on tokio or other relatively heavy deps, so we will retain a sync implementation so that wasi-common users have an option of not pulling in an async runtime.

Improving the WASI test suite

The bulk of wasi-common's test suite lives at crates/test-programs. This test suite was extremely useful for guiding this rewrite. We improved the test suite in numerous ways, including

• Breaking tests out into smaller units, so that failing behavior can be isolated more easily
• Introducing a scheme which allows the test runner to describe to the wasm code what behaviors to expect of the embedding. Behaviors which vary between platforms (e.g. fd_allcoate is impossible to faithfully implement on Windows and is not provided by MacOS) are communicated to the guest by environment variables, which in turn is available in the guest behind the global TESTCONFIG struct.
• Introducing the assert_errno! macro, which pretty-prints errnos by name rather than number in the error message. assert_errno! integrates with TESTCONFIG to specify which errno is expected on which platform. This allows the test source to describe the full set of acceptable errnos a call may return (all possible errnos will be acceptable if no ERRNO_MODE_* env var is set), and also detect regressions on any given platform.
• We still use crates/test-platforms/build.rs to generate the test suite, and use functions directly in build.rs to describe the TESTCONFIG expectations, as well as which tests are to be ignored (expected to fail due to regressions) on what platforms.
• The test suite has been generalized to support multiple backends to the wasi-common crate.
  [message truncated]

Wasmtime GitHub notifications bot (Feb 02 2021 at 03:22):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Note that this is a breaking change for all users of WASI with Wasmtime, except for the C API.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR).

Traits for the rest of WASI's features

Other aspects of a WASI implementation are not yet considered resources and accessed by handle. We plan to correct this design deficiency in WASI in the future, but for now we have designed the following traits to provide embedders with the same sort of implementation flexibility they get with WasiFile/WasiDir:

• Timekeeping: clocks::WasiSystemClock and clock::WasiMonotonicClock provide the two interfaces for a clock. WasiSystemClock represents time as a cap_std::time::SystemTime, and WasiMonotonicClock represents time as cap_std::time::Instant.
• Randomness: we re-use the cap_rand::RngCore trait to represent a randomness source. A trivial Deterministic impl is provided.
• Scheduling: The WasiSched trait abstracts over the sched_yield and poll_oneoff functions.

Users can provide implementations of each of these interfaces to the WasiCtx::builder(...) function. The wasi_cap_std_sync::WasiCtxBuilder::new() function uses this public interface to plug in its own implementations of each of these resources.

Snapshot architecture

One goal we've had for a while, but not quite met, is for multiple WASI snapshots to provide an interface to the same underlying WasiCtx. This provides us a path to evolve WASI by allowing the same WASI Command to import functions from different snapshots - e.g. the user could use Rust's std which imports snapshot 1, but also depend directly on the wasi crate which imports some future snapshot 2. Right now, this amounts to supporting snapshot 1 and "snapshot 0" aka wasi_unstable at once.

The architectural rules for snapshots are:

• Snapshots are arranged into modules under crate::snapshots::.
• Each snapshot should invoke wiggle::from_witx! with ctx: crate::WasiCtx in its module, and impl all of the required traits.
• Snapshots can be implemented in terms of other snapshots. For example, snapshot 0 is mostly implemented by calling the snapshot 1 implementation, and converting its own types back and forth with the snapshot 1 types. In a few cases, that is not feasible, so snapshot 0 carries its own implementations in terms of the WasiFile and WasiSched traits.
• Snapshots can be implemented in terms of the Wasi* traits given by WasiCtx. No further downcasting via the as_any escape hatch is permitted.

The wasi_common::Error type

wasi_common::Error is now anyhow::Error. wasi_common::snapshots::preview_1 contains all of the logic for transforming an Error into an Errno, by downcasting the error into any of

• std::io::Error - these are thrown by std, cap_std, etc for most of the operations WASI is concerned with.
• wasi_common::ErrorKind - these are a subset of the Errnos, and are constructed directly by wasi-common or an impl rather than coming from the OS or some library which doesn't know about WASI.
• wiggle::GuestError
• std::num::TryFromIntError
• std::str::Utf8Error
  and then applying specialized logic to translate each of those into Errnos.

The wasi_common::ErrorExt trait provides human-friendly constructors for the wasi_common::ErrorKind variants .

If you throw an error that does not downcast to one of those, it will turn into a wiggle::Trap and terminate execution.

The real value of using anyhow::Error here is being able to use anyhow::Result::context to aid in debugging of errors.

The wasi-cap-std-sync implementation of the WASI traits

The wasi-cap-std-sync crate provides impl of WasiFile and WasiDir in terms of cap_std::fs::{File, Dir}. These types provide sandboxed access to the local filesystem on both Unix and Windows.

The entire design story of cap-std is much bigger than we can address here, but in short, its functionality replaces all of the wasi_common::sys hierarchy, as well as the yanix / winx crates. All syscalls are hidden behind the cap-std hierarchy, with the lone exception of the sched implementation, which is provided for both unix and windows in separate modules.

Any wasi_common::{WasiCtx, WasiCtxBuilder} is interoperable with the wasi-cap-std-sync crate. However, for convenience, wasi-cap-std-sync provides its own WasiCtxBuilder that hooks up to all of the crate's components, i.e. it fills in all of the arguments to WasiCtx::builder(...), presents preopen_dir in terms of cap_std::fs::Dir, and provides convenience methods for inheriting the parent process's stdio, args, and env.

The only place we expect to run into long-term compatibility issues between wasi-cap-std-sync and the other impl crates that will come later is in the Sched abstraction. Once we can build an async scheduler based on Rust Futures, async impls will be able to interoperate, but the synchronous scheduler depends on downcasting the WasiFile type down to concrete types it knows about (which in turn impl AsRawFd for passing to unix poll, or the analogous traits on windows).

Why is this impl suffixed with -sync? https://github.com/bytecodealliance/wasmtime/pull/2434 async is coming soon! The async impl may end up depending on tokio or other relatively heavy deps, so we will retain a sync implementation so that wasi-common users have an option of not pulling in an async runtime.

Improving the WASI test suite

The bulk of wasi-common's test suite lives at crates/test-programs. This test suite was extremely useful for guiding this rewrite. We improved the test suite in numerous ways, including

• Breaking tests out into smaller units, so that failing behavior can be isolated more easily
• Introducing a scheme which allows the test runner to describe to the wasm code what behaviors to expect of the embedding. Behaviors which vary between platforms (e.g. fd_allcoate is impossible to faithfully implement on Windows and is not provided by MacOS) are communicated to the guest by environment variables, which in turn is available in the guest behind the global TESTCONFIG struct.
• Introducing the assert_errno! macro, which pretty-prints errnos by name rather than number in the error message. assert_errno! integrates with TESTCONFIG to specify which errno is expected on which platform. This allows the test source to describe the full set of acceptable errnos a call may return (all possible errnos will be acceptable if no ERRNO_MODE_* env var is set), and also detect regressions on any given platform.
• We still use crates/test-platforms/build.rs to generate the test suite, and use functions directly in build.rs to describe the TESTCONFIG expectations, as well as which tests are to be ignored (expected to fail due to regressions) on what platforms.
• The test suite has been generalized to support multiple backends to the wasi-common crate.
  [message truncated]

Wasmtime GitHub notifications bot (Feb 02 2021 at 03:40):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Note that this is a breaking change for all users of WASI with Wasmtime, except for the C API.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR).

Traits for the rest of WASI's features

Other aspects of a WASI implementation are not yet considered resources and accessed by handle. We plan to correct this design deficiency in WASI in the future, but for now we have designed the following traits to provide embedders with the same sort of implementation flexibility they get with WasiFile/WasiDir:

• Timekeeping: clocks::WasiSystemClock and clock::WasiMonotonicClock provide the two interfaces for a clock. WasiSystemClock represents time as a cap_std::time::SystemTime, and WasiMonotonicClock represents time as cap_std::time::Instant.
• Randomness: we re-use the cap_rand::RngCore trait to represent a randomness source. A trivial Deterministic impl is provided.
• Scheduling: The WasiSched trait abstracts over the sched_yield and poll_oneoff functions.

Users can provide implementations of each of these interfaces to the WasiCtx::builder(...) function. The wasi_cap_std_sync::WasiCtxBuilder::new() function uses this public interface to plug in its own implementations of each of these resources.

Snapshot architecture

One goal we've had for a while, but not quite met, is for multiple WASI snapshots to provide an interface to the same underlying WasiCtx. This provides us a path to evolve WASI by allowing the same WASI Command to import functions from different snapshots - e.g. the user could use Rust's std which imports snapshot 1, but also depend directly on the wasi crate which imports some future snapshot 2. Right now, this amounts to supporting snapshot 1 and "snapshot 0" aka wasi_unstable at once.

The architectural rules for snapshots are:

• Snapshots are arranged into modules under crate::snapshots::.
• Each snapshot should invoke wiggle::from_witx! with ctx: crate::WasiCtx in its module, and impl all of the required traits.
• Snapshots can be implemented in terms of other snapshots. For example, snapshot 0 is mostly implemented by calling the snapshot 1 implementation, and converting its own types back and forth with the snapshot 1 types. In a few cases, that is not feasible, so snapshot 0 carries its own implementations in terms of the WasiFile and WasiSched traits.
• Snapshots can be implemented in terms of the Wasi* traits given by WasiCtx. No further downcasting via the as_any escape hatch is permitted.

The wasi_common::Error type

wasi_common::Error is now anyhow::Error. wasi_common::snapshots::preview_1 contains all of the logic for transforming an Error into an Errno, by downcasting the error into any of

• std::io::Error - these are thrown by std, cap_std, etc for most of the operations WASI is concerned with.
• wasi_common::ErrorKind - these are a subset of the Errnos, and are constructed directly by wasi-common or an impl rather than coming from the OS or some library which doesn't know about WASI.
• wiggle::GuestError
• std::num::TryFromIntError
• std::str::Utf8Error
  and then applying specialized logic to translate each of those into Errnos.

The wasi_common::ErrorExt trait provides human-friendly constructors for the wasi_common::ErrorKind variants .

If you throw an error that does not downcast to one of those, it will turn into a wiggle::Trap and terminate execution.

The real value of using anyhow::Error here is being able to use anyhow::Result::context to aid in debugging of errors.

The wasi-cap-std-sync implementation of the WASI traits

The wasi-cap-std-sync crate provides impl of WasiFile and WasiDir in terms of cap_std::fs::{File, Dir}. These types provide sandboxed access to the local filesystem on both Unix and Windows.

The entire design story of cap-std is much bigger than we can address here, but in short, its functionality replaces all of the wasi_common::sys hierarchy, as well as the yanix / winx crates. All syscalls are hidden behind the cap-std hierarchy, with the lone exception of the sched implementation, which is provided for both unix and windows in separate modules.

Any wasi_common::{WasiCtx, WasiCtxBuilder} is interoperable with the wasi-cap-std-sync crate. However, for convenience, wasi-cap-std-sync provides its own WasiCtxBuilder that hooks up to all of the crate's components, i.e. it fills in all of the arguments to WasiCtx::builder(...), presents preopen_dir in terms of cap_std::fs::Dir, and provides convenience methods for inheriting the parent process's stdio, args, and env.

The only place we expect to run into long-term compatibility issues between wasi-cap-std-sync and the other impl crates that will come later is in the Sched abstraction. Once we can build an async scheduler based on Rust Futures, async impls will be able to interoperate, but the synchronous scheduler depends on downcasting the WasiFile type down to concrete types it knows about (which in turn impl AsRawFd for passing to unix poll, or the analogous traits on windows).

Why is this impl suffixed with -sync? https://github.com/bytecodealliance/wasmtime/pull/2434 async is coming soon! The async impl may end up depending on tokio or other relatively heavy deps, so we will retain a sync implementation so that wasi-common users have an option of not pulling in an async runtime.

Improving the WASI test suite

The bulk of wasi-common's test suite lives at crates/test-programs. This test suite was extremely useful for guiding this rewrite. We improved the test suite in numerous ways, including

• Breaking tests out into smaller units, so that failing behavior can be isolated more easily
• Introducing a scheme which allows the test runner to describe to the wasm code what behaviors to expect of the embedding. Behaviors which vary between platforms (e.g. fd_allcoate is impossible to faithfully implement on Windows and is not provided by MacOS) are communicated to the guest by environment variables, which in turn is available in the guest behind the global TESTCONFIG struct.
• Introducing the assert_errno! macro, which pretty-prints errnos by name rather than number in the error message. assert_errno! integrates with TESTCONFIG to specify which errno is expected on which platform. This allows the test source to describe the full set of acceptable errnos a call may return (all possible errnos will be acceptable if no ERRNO_MODE_* env var is set), and also detect regressions on any given platform.
• We still use crates/test-platforms/build.rs to generate the test suite, and use functions directly in build.rs to describe the TESTCONFIG expectations, as well as which tests are to be ignored (expected to fail due to regressions) on what platforms.
• The test suite has been generalized to support multiple backends to the wasi-common crate.
  [message truncated]

Wasmtime GitHub notifications bot (Feb 02 2021 at 18:44):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Note that this is a breaking change for all users of WASI with Wasmtime, except for the C API.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR).

Traits for the rest of WASI's features

Other aspects of a WASI implementation are not yet considered resources and accessed by handle. We plan to correct this design deficiency in WASI in the future, but for now we have designed the following traits to provide embedders with the same sort of implementation flexibility they get with WasiFile/WasiDir:

• Timekeeping: clocks::WasiSystemClock and clock::WasiMonotonicClock provide the two interfaces for a clock. WasiSystemClock represents time as a cap_std::time::SystemTime, and WasiMonotonicClock represents time as cap_std::time::Instant.
• Randomness: we re-use the cap_rand::RngCore trait to represent a randomness source. A trivial Deterministic impl is provided.
• Scheduling: The WasiSched trait abstracts over the sched_yield and poll_oneoff functions.

Users can provide implementations of each of these interfaces to the WasiCtx::builder(...) function. The wasi_cap_std_sync::WasiCtxBuilder::new() function uses this public interface to plug in its own implementations of each of these resources.

Snapshot architecture

One goal we've had for a while, but not quite met, is for multiple WASI snapshots to provide an interface to the same underlying WasiCtx. This provides us a path to evolve WASI by allowing the same WASI Command to import functions from different snapshots - e.g. the user could use Rust's std which imports snapshot 1, but also depend directly on the wasi crate which imports some future snapshot 2. Right now, this amounts to supporting snapshot 1 and "snapshot 0" aka wasi_unstable at once.

The architectural rules for snapshots are:

• Snapshots are arranged into modules under crate::snapshots::.
• Each snapshot should invoke wiggle::from_witx! with ctx: crate::WasiCtx in its module, and impl all of the required traits.
• Snapshots can be implemented in terms of other snapshots. For example, snapshot 0 is mostly implemented by calling the snapshot 1 implementation, and converting its own types back and forth with the snapshot 1 types. In a few cases, that is not feasible, so snapshot 0 carries its own implementations in terms of the WasiFile and WasiSched traits.
• Snapshots can be implemented in terms of the Wasi* traits given by WasiCtx. No further downcasting via the as_any escape hatch is permitted.

The wasi_common::Error type

wasi_common::Error is now anyhow::Error. wasi_common::snapshots::preview_1 contains all of the logic for transforming an Error into an Errno, by downcasting the error into any of

• std::io::Error - these are thrown by std, cap_std, etc for most of the operations WASI is concerned with.
• wasi_common::ErrorKind - these are a subset of the Errnos, and are constructed directly by wasi-common or an impl rather than coming from the OS or some library which doesn't know about WASI.
• wiggle::GuestError
• std::num::TryFromIntError
• std::str::Utf8Error
  and then applying specialized logic to translate each of those into Errnos.

The wasi_common::ErrorExt trait provides human-friendly constructors for the wasi_common::ErrorKind variants .

If you throw an error that does not downcast to one of those, it will turn into a wiggle::Trap and terminate execution.

The real value of using anyhow::Error here is being able to use anyhow::Result::context to aid in debugging of errors.

The wasi-cap-std-sync implementation of the WASI traits

The wasi-cap-std-sync crate provides impl of WasiFile and WasiDir in terms of cap_std::fs::{File, Dir}. These types provide sandboxed access to the local filesystem on both Unix and Windows.

The entire design story of cap-std is much bigger than we can address here, but in short, its functionality replaces all of the wasi_common::sys hierarchy, as well as the yanix / winx crates. All syscalls are hidden behind the cap-std hierarchy, with the lone exception of the sched implementation, which is provided for both unix and windows in separate modules.

Any wasi_common::{WasiCtx, WasiCtxBuilder} is interoperable with the wasi-cap-std-sync crate. However, for convenience, wasi-cap-std-sync provides its own WasiCtxBuilder that hooks up to all of the crate's components, i.e. it fills in all of the arguments to WasiCtx::builder(...), presents preopen_dir in terms of cap_std::fs::Dir, and provides convenience methods for inheriting the parent process's stdio, args, and env.

The only place we expect to run into long-term compatibility issues between wasi-cap-std-sync and the other impl crates that will come later is in the Sched abstraction. Once we can build an async scheduler based on Rust Futures, async impls will be able to interoperate, but the synchronous scheduler depends on downcasting the WasiFile type down to concrete types it knows about (which in turn impl AsRawFd for passing to unix poll, or the analogous traits on windows).

Why is this impl suffixed with -sync? https://github.com/bytecodealliance/wasmtime/pull/2434 async is coming soon! The async impl may end up depending on tokio or other relatively heavy deps, so we will retain a sync implementation so that wasi-common users have an option of not pulling in an async runtime.

Improving the WASI test suite

The bulk of wasi-common's test suite lives at crates/test-programs. This test suite was extremely useful for guiding this rewrite. We improved the test suite in numerous ways, including

• Breaking tests out into smaller units, so that failing behavior can be isolated more easily
• Introducing a scheme which allows the test runner to describe to the wasm code what behaviors to expect of the embedding. Behaviors which vary between platforms (e.g. fd_allcoate is impossible to faithfully implement on Windows and is not provided by MacOS) are communicated to the guest by environment variables, which in turn is available in the guest behind the global TESTCONFIG struct.
• Introducing the assert_errno! macro, which pretty-prints errnos by name rather than number in the error message. assert_errno! integrates with TESTCONFIG to specify which errno is expected on which platform. This allows the test source to describe the full set of acceptable errnos a call may return (all possible errnos will be acceptable if no ERRNO_MODE_* env var is set), and also detect regressions on any given platform.
• We still use crates/test-platforms/build.rs to generate the test suite, and use functions directly in build.rs to describe the TESTCONFIG expectations, as well as which tests are to be ignored (expected to fail due to regressions) on what platforms.
• The test suite has been generalized to support multiple backends to the wasi-common crate.
  [message truncated]

Wasmtime GitHub notifications bot (Feb 02 2021 at 21:30):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Note that this is a breaking change for all users of WASI with Wasmtime, except for the C API.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR).

Traits for the rest of WASI's features

Other aspects of a WASI implementation are not yet considered resources and accessed by handle. We plan to correct this design deficiency in WASI in the future, but for now we have designed the following traits to provide embedders with the same sort of implementation flexibility they get with WasiFile/WasiDir:

• Timekeeping: clocks::WasiSystemClock and clock::WasiMonotonicClock provide the two interfaces for a clock. WasiSystemClock represents time as a cap_std::time::SystemTime, and WasiMonotonicClock represents time as cap_std::time::Instant.
• Randomness: we re-use the cap_rand::RngCore trait to represent a randomness source. A trivial Deterministic impl is provided.
• Scheduling: The WasiSched trait abstracts over the sched_yield and poll_oneoff functions.

Users can provide implementations of each of these interfaces to the WasiCtx::builder(...) function. The wasi_cap_std_sync::WasiCtxBuilder::new() function uses this public interface to plug in its own implementations of each of these resources.

Snapshot architecture

One goal we've had for a while, but not quite met, is for multiple WASI snapshots to provide an interface to the same underlying WasiCtx. This provides us a path to evolve WASI by allowing the same WASI Command to import functions from different snapshots - e.g. the user could use Rust's std which imports snapshot 1, but also depend directly on the wasi crate which imports some future snapshot 2. Right now, this amounts to supporting snapshot 1 and "snapshot 0" aka wasi_unstable at once.

The architectural rules for snapshots are:

• Snapshots are arranged into modules under crate::snapshots::.
• Each snapshot should invoke wiggle::from_witx! with ctx: crate::WasiCtx in its module, and impl all of the required traits.
• Snapshots can be implemented in terms of other snapshots. For example, snapshot 0 is mostly implemented by calling the snapshot 1 implementation, and converting its own types back and forth with the snapshot 1 types. In a few cases, that is not feasible, so snapshot 0 carries its own implementations in terms of the WasiFile and WasiSched traits.
• Snapshots can be implemented in terms of the Wasi* traits given by WasiCtx. No further downcasting via the as_any escape hatch is permitted.

The wasi_common::Error type

wasi_common::Error is now anyhow::Error. wasi_common::snapshots::preview_1 contains all of the logic for transforming an Error into an Errno, by downcasting the error into any of

• std::io::Error - these are thrown by std, cap_std, etc for most of the operations WASI is concerned with.
• wasi_common::ErrorKind - these are a subset of the Errnos, and are constructed directly by wasi-common or an impl rather than coming from the OS or some library which doesn't know about WASI.
• wiggle::GuestError
• std::num::TryFromIntError
• std::str::Utf8Error
  and then applying specialized logic to translate each of those into Errnos.

The wasi_common::ErrorExt trait provides human-friendly constructors for the wasi_common::ErrorKind variants .

If you throw an error that does not downcast to one of those, it will turn into a wiggle::Trap and terminate execution.

The real value of using anyhow::Error here is being able to use anyhow::Result::context to aid in debugging of errors.

The wasi-cap-std-sync implementation of the WASI traits

The wasi-cap-std-sync crate provides impl of WasiFile and WasiDir in terms of cap_std::fs::{File, Dir}. These types provide sandboxed access to the local filesystem on both Unix and Windows.

The entire design story of cap-std is much bigger than we can address here, but in short, its functionality replaces all of the wasi_common::sys hierarchy, as well as the yanix / winx crates. All syscalls are hidden behind the cap-std hierarchy, with the lone exception of the sched implementation, which is provided for both unix and windows in separate modules.

Any wasi_common::{WasiCtx, WasiCtxBuilder} is interoperable with the wasi-cap-std-sync crate. However, for convenience, wasi-cap-std-sync provides its own WasiCtxBuilder that hooks up to all of the crate's components, i.e. it fills in all of the arguments to WasiCtx::builder(...), presents preopen_dir in terms of cap_std::fs::Dir, and provides convenience methods for inheriting the parent process's stdio, args, and env.

The only place we expect to run into long-term compatibility issues between wasi-cap-std-sync and the other impl crates that will come later is in the Sched abstraction. Once we can build an async scheduler based on Rust Futures, async impls will be able to interoperate, but the synchronous scheduler depends on downcasting the WasiFile type down to concrete types it knows about (which in turn impl AsRawFd for passing to unix poll, or the analogous traits on windows).

Why is this impl suffixed with -sync? https://github.com/bytecodealliance/wasmtime/pull/2434 async is coming soon! The async impl may end up depending on tokio or other relatively heavy deps, so we will retain a sync implementation so that wasi-common users have an option of not pulling in an async runtime.

Improving the WASI test suite

The bulk of wasi-common's test suite lives at crates/test-programs. This test suite was extremely useful for guiding this rewrite. We improved the test suite in numerous ways, including

• Breaking tests out into smaller units, so that failing behavior can be isolated more easily
• Introducing a scheme which allows the test runner to describe to the wasm code what behaviors to expect of the embedding. Behaviors which vary between platforms (e.g. fd_allcoate is impossible to faithfully implement on Windows and is not provided by MacOS) are communicated to the guest by environment variables, which in turn is available in the guest behind the global TESTCONFIG struct.
• Introducing the assert_errno! macro, which pretty-prints errnos by name rather than number in the error message. assert_errno! integrates with TESTCONFIG to specify which errno is expected on which platform. This allows the test source to describe the full set of acceptable errnos a call may return (all possible errnos will be acceptable if no ERRNO_MODE_* env var is set), and also detect regressions on any given platform.
• We still use crates/test-platforms/build.rs to generate the test suite, and use functions directly in build.rs to describe the TESTCONFIG expectations, as well as which tests are to be ignored (expected to fail due to regressions) on what platforms.
• The test suite has been generalized to support multiple backends to the wasi-common crate.
  [message truncated]

Wasmtime GitHub notifications bot (Feb 02 2021 at 21:41):

pchickey updated PR #2487 from pch/wasi_common_cap_std to main:

This PR is a complete rewrite of the wasi-common crate. It uses @sunfishcode 's new cap-std family of crates to provide a sandboxed implementation of WASI on the local filesystem.

Note that this is a breaking change for all users of WASI with Wasmtime, except for the C API.

Re-architecting wasi-common

Over the past year or so, we've run up against many design problems in wasi-common. Many of these problems ended up being so fundamental that a rewrite, as prolonged as it was, ended up being more tractable than incremental changes.

The Table

wasi-common now has a Table type that is designed to map u32 handles to resources. The table is now part of the public interface to a WasiCtx - it is reference counted so that it can be shared beyond a WasiCtx with other WASI proposals (e.g. wasi-crypto and wasi-nn) to manage their resources. Elements in the Table are Any typed.

The Table type is intended to model how the Interface Types concept of Resources is shaping up. Right now it is just an approximation.

The WasiFile and WasiDir traits

The WASI specification only defines one handle type, fd, on which all operations on both files and directories (aka dirfds) are defined. We believe this is a design mistake, and are architecting wasi-common to make this straightforward to correct in future snapshots of WASI. Wasi-common internally treats files and directories as two distinct resource types in the table - Box<dyn WasiFile> and Box<dyn WasiDir>. The snapshot 0 and 1 interfaces via fd will attempt to downcast a table element to one or both of these interfaces depending on what is appropriate - e.g. fd_close operates on both files and directories, fd_read only operates on files, and fd_readdir only operates on directories.

The WasiFile and WasiDir traits are defined by wasi-common in terms of types defined directly in the crate's source code (I decided it should NOT those generated by the wiggle proc macros, see snapshot architecture below), as well as the cap_std::time family of types. And, importantly, wasi-common itself provides no implementation of WasiDir, and only two trivial implementations of WasiFile on the crate::pipe::{ReadPipe, WritePipe} types, which in turn just delegate to std::io::{Read, Write}. In order for wasi-common to access the local filesystem at all, you need to provide WasiFile and WasiDir impls through either the new wasi-cap-std-sync crate found at crates/wasi-common/cap-std-sync - see the section on that crate below - or by providing your own implementation from elsewhere.

This design makes it possible for wasi-common embedders to statically reason about access to the local filesystem by examining what impls are linked into an application. We found that this separation of concerns also makes it pretty enjoyable to write alternative implementations, e.g. a virtual filesystem (which will land in a future PR).

Traits for the rest of WASI's features

Other aspects of a WASI implementation are not yet considered resources and accessed by handle. We plan to correct this design deficiency in WASI in the future, but for now we have designed the following traits to provide embedders with the same sort of implementation flexibility they get with WasiFile/WasiDir:

• Timekeeping: clocks::WasiSystemClock and clock::WasiMonotonicClock provide the two interfaces for a clock. WasiSystemClock represents time as a cap_std::time::SystemTime, and WasiMonotonicClock represents time as cap_std::time::Instant.
• Randomness: we re-use the cap_rand::RngCore trait to represent a randomness source. A trivial Deterministic impl is provided.
• Scheduling: The WasiSched trait abstracts over the sched_yield and poll_oneoff functions.

Users can provide implementations of each of these interfaces to the WasiCtx::builder(...) function. The wasi_cap_std_sync::WasiCtxBuilder::new() function uses this public interface to plug in its own implementations of each of these resources.

Snapshot architecture

One goal we've had for a while, but not quite met, is for multiple WASI snapshots to provide an interface to the same underlying WasiCtx. This provides us a path to evolve WASI by allowing the same WASI Command to import functions from different snapshots - e.g. the user could use Rust's std which imports snapshot 1, but also depend directly on the wasi crate which imports some future snapshot 2. Right now, this amounts to supporting snapshot 1 and "snapshot 0" aka wasi_unstable at once.

The architectural rules for snapshots are:

• Snapshots are arranged into modules under crate::snapshots::.
• Each snapshot should invoke wiggle::from_witx! with ctx: crate::WasiCtx in its module, and impl all of the required traits.
• Snapshots can be implemented in terms of other snapshots. For example, snapshot 0 is mostly implemented by calling the snapshot 1 implementation, and converting its own types back and forth with the snapshot 1 types. In a few cases, that is not feasible, so snapshot 0 carries its own implementations in terms of the WasiFile and WasiSched traits.
• Snapshots can be implemented in terms of the Wasi* traits given by WasiCtx. No further downcasting via the as_any escape hatch is permitted.

The wasi_common::Error type

wasi_common::Error is now anyhow::Error. wasi_common::snapshots::preview_1 contains all of the logic for transforming an Error into an Errno, by downcasting the error into any of

• std::io::Error - these are thrown by std, cap_std, etc for most of the operations WASI is concerned with.
• wasi_common::ErrorKind - these are a subset of the Errnos, and are constructed directly by wasi-common or an impl rather than coming from the OS or some library which doesn't know about WASI.
• wiggle::GuestError
• std::num::TryFromIntError
• std::str::Utf8Error
  and then applying specialized logic to translate each of those into Errnos.

The wasi_common::ErrorExt trait provides human-friendly constructors for the wasi_common::ErrorKind variants .

If you throw an error that does not downcast to one of those, it will turn into a wiggle::Trap and terminate execution.

The real value of using anyhow::Error here is being able to use anyhow::Result::context to aid in debugging of errors.

The wasi-cap-std-sync implementation of the WASI traits

The wasi-cap-std-sync crate provides impl of WasiFile and WasiDir in terms of cap_std::fs::{File, Dir}. These types provide sandboxed access to the local filesystem on both Unix and Windows.

The entire design story of cap-std is much bigger than we can address here, but in short, its functionality replaces all of the wasi_common::sys hierarchy, as well as the yanix / winx crates. All syscalls are hidden behind the cap-std hierarchy, with the lone exception of the sched implementation, which is provided for both unix and windows in separate modules.

Any wasi_common::{WasiCtx, WasiCtxBuilder} is interoperable with the wasi-cap-std-sync crate. However, for convenience, wasi-cap-std-sync provides its own WasiCtxBuilder that hooks up to all of the crate's components, i.e. it fills in all of the arguments to WasiCtx::builder(...), presents preopen_dir in terms of cap_std::fs::Dir, and provides convenience methods for inheriting the parent process's stdio, args, and env.

The only place we expect to run into long-term compatibility issues between wasi-cap-std-sync and the other impl crates that will come later is in the Sched abstraction. Once we can build an async scheduler based on Rust Futures, async impls will be able to interoperate, but the synchronous scheduler depends on downcasting the WasiFile type down to concrete types it knows about (which in turn impl AsRawFd for passing to unix poll, or the analogous traits on windows).

Why is this impl suffixed with -sync? https://github.com/bytecodealliance/wasmtime/pull/2434 async is coming soon! The async impl may end up depending on tokio or other relatively heavy deps, so we will retain a sync implementation so that wasi-common users have an option of not pulling in an async runtime.

Improving the WASI test suite

The bulk of wasi-common's test suite lives at crates/test-programs. This test suite was extremely useful for guiding this rewrite. We improved the test suite in numerous ways, including

• Breaking tests out into smaller units, so that failing behavior can be isolated more easily
• Introducing a scheme which allows the test runner to describe to the wasm code what behaviors to expect of the embedding. Behaviors which vary between platforms (e.g. fd_allcoate is impossible to faithfully implement on Windows and is not provided by MacOS) are communicated to the guest by environment variables, which in turn is available in the guest behind the global TESTCONFIG struct.
• Introducing the assert_errno! macro, which pretty-prints errnos by name rather than number in the error message. assert_errno! integrates with TESTCONFIG to specify which errno is expected on which platform. This allows the test source to describe the full set of acceptable errnos a call may return (all possible errnos will be acceptable if no ERRNO_MODE_* env var is set), and also detect regressions on any given platform.
• We still use crates/test-platforms/build.rs to generate the test suite, and use functions directly in build.rs to describe the TESTCONFIG expectations, as well as which tests are to be ignored (expected to fail due to regressions) on what platforms.
• The test suite has been generalized to support multiple backends to the wasi-common crate.
  [message truncated]