Stream: git-wasmtime

Topic: wasmtime / issue #8029 Properly supporting semver, plus `...

view this post on Zulip Wasmtime GitHub notifications bot (Feb 29 2024 at 18:16):

tschneidereit added the wasmtime label to Issue #8029.

view this post on Zulip Wasmtime GitHub notifications bot (Feb 29 2024 at 18:16):

tschneidereit added the wasm-proposal:component-model label to Issue #8029.

view this post on Zulip Wasmtime GitHub notifications bot (Feb 29 2024 at 18:16):

tschneidereit opened issue #8029:

In the Wasmtime meeting today, we discussed some challenges around semver support for component model packages.

Specifically, we identified two issues:

  1. Right now Wasmtime's semver support is overly permissive, in that it makes symbols available in all semver-compatible versions, not just starting with the version they were introduced in.
  2. How do we support preview/draft releases of upcoming semver-compatible versions, including treating them as semver-compatible with the last stable version? (Assuming the draft version isn't semver breaking.)

During the meeting, I proposed an approach that'd involve maintaining a side-table of symbols to add to the linker under aliases, where for each minor version we'd add all current contents of the side table to the linker aliased for that version, and additionally add all symbols introduced by that version to the table (and the linker.)

I think we could abstract that away by introducing methods on the linker along the lines of

As a concrete example, say we want to support versions 0.2.0, 0.2.1-DRAFT, 0.2.1, and 0.2.2 of a package. (Setting aside the question of whether we'd ever want to support a -DRAFT version in parallel to the respective stable version or any higher minor versions.) This approach would have us follow these steps:

  1. Call start_semver_range
  2. Call add_to_semver_range for each symbol in 0.2.0
  3. Call add_to_linker (:exclamation:) for symbol in 0.2.1-DRAFT
  4. Call add_to_semver_range for each symbol in 0.2.1
  5. Call add_to_semver_range for each symbol in 0.2.2

I'm not particularly familiar with the workings of Wasmtime's linker, so this might not work all that well in practice—but maybe it does?

view this post on Zulip Wasmtime GitHub notifications bot (Feb 29 2024 at 18:56):

alexcrichton commented on issue #8029:

At a high level one of the goals I'd like to have with our WASI support in Wasmtime is that we're not required to keep an ever-expanding list of copies of WASI APIs, e.g. 0.2.0, 0.2.1, 0.2.2, .... If we take this as a goal, however, I don't think that we can solve problem (1) listed above because WIT itself doesn't retain a version number of when a symbol was first defined. Personally I don't think this matters too much in the sense that you would have to really go out of your way to create a component that imports 0.2.1 APIs at an 0.2.0 version and there's not necessarily a big consequence.

Additionally with adding a symbol to each semver range, if we take the above goal as one to pursue, I'm not sure how we would do that. I'm not sure how we could enumerate all the semver versions that a symbol was known at.

Otherwise though in terms of how the Linker type works you can think of it as a glorified component instance. You get to define exports and each export can itself be an instance with its own exports, so you end up with a tree-like structure where you defined all the names in the tree and the values at the leaves. So today add_to_linker is basically building the part of the tree with WASI names, for example.

The semver compat stuff at this point is currently all built where for each component instance in the tree there's a table in that instance of "if you're looking for something semver compatible with 0.2.X then go load import 0.2.3" (or whatever the max registered version is). That's what gives rise to the behavior of if you import timezones at 0.2.0 then you'll get a "hit" on timezones at 0.2.1.

I explain the Linker bits because one issue with the per-symbol versioning is that it's not quite the layer where versioning happens. At the component layer what's happening is that a component is importing a versioned instance from the host, but then in the instance there are no versions. For example wasi:cli/environment@0.2.0 has a version but get-environment does not have a version. This means that there's really a concept of per-symbol versioning but instead you can think of a version as applying to an entire subtree (e.g. an instance). With that if a function were added to an interface then given the above scheme I think it gets a bit tricky.

Speaking again at a high level in terms of requirements, I think another requirement we'd like to have is to avoid the need to have lots of "implement this one WASI proposal in terms of this other one". For example in Wasmtime we don't want to manually implement all of 0.2.0 in terms of 0.2.1.

Now in terms of trying to propose a solution instead of just critiquing them: the best I can think of is to maintain the last stable WASI release and the next draft we're supporting. This would then look like:

Then as part of the ecosystem we'd have an optional feature perhaps on the wasi crate for enabling 0.2.1-draft (or maybe a prerelease of the crate itself? unsure.). That would generate bindings alongside the existing 0.2.0 ones perhaps in a submodule or similar (depends on the publication process of the crate itself I think).

view this post on Zulip Wasmtime GitHub notifications bot (Feb 29 2024 at 18:57):

alexcrichton commented on issue #8029:

I'll note though that my proposed idea does not solve (1), it leaves it as an open problem to be solved in the future, if at all

view this post on Zulip Wasmtime GitHub notifications bot (Apr 17 2024 at 19:07):

alexcrichton closed issue #8029:

In the Wasmtime meeting today, we discussed some challenges around semver support for component model packages.

Specifically, we identified two issues:

  1. Right now Wasmtime's semver support is overly permissive, in that it makes symbols available in all semver-compatible versions, not just starting with the version they were introduced in.
  2. How do we support preview/draft releases of upcoming semver-compatible versions, including treating them as semver-compatible with the last stable version? (Assuming the draft version isn't semver breaking.)

During the meeting, I proposed an approach that'd involve maintaining a side-table of symbols to add to the linker under aliases, where for each minor version we'd add all current contents of the side table to the linker aliased for that version, and additionally add all symbols introduced by that version to the table (and the linker.)

I think we could abstract that away by introducing methods on the linker along the lines of

As a concrete example, say we want to support versions 0.2.0, 0.2.1-DRAFT, 0.2.1, and 0.2.2 of a package. (Setting aside the question of whether we'd ever want to support a -DRAFT version in parallel to the respective stable version or any higher minor versions.) This approach would have us follow these steps:

  1. Call start_semver_range
  2. Call add_to_semver_range for each symbol in 0.2.0
  3. Call add_to_linker (:exclamation:) for symbol in 0.2.1-DRAFT
  4. Call add_to_semver_range for each symbol in 0.2.1
  5. Call add_to_semver_range for each symbol in 0.2.2

I'm not particularly familiar with the workings of Wasmtime's linker, so this might not work all that well in practice—but maybe it does?

view this post on Zulip Wasmtime GitHub notifications bot (Apr 17 2024 at 19:07):

alexcrichton commented on issue #8029:

I'm going to close this as I think it was addressed by https://github.com/bytecodealliance/wasmtime/pull/7994 and the remaining bits are covered by https://github.com/bytecodealliance/wasmtime/issues/8395

Last updated: Oct 23 2024 at 20:03 UTC