RFC notifications bot (Aug 18 2021 at 19:47): RFC notifications bot (Aug 18 2021 at 19:47):alexcrichton opened PR #14 from wasmtime-1.0 to main:
This RFC is a proposal to release Wasmtime with a 1.0 version. In doing so this
proposal also defines what it means to be 1.0 and details areas such as:
- What is Wasmtime's stability story.
- What will Wasmtime's release process be.
- What users can expect from Wasmtime going forward.
RFC notifications bot (Aug 18 2021 at 19:48):alexcrichton edited PR #14 from wasmtime-1.0 to main:
This RFC is a proposal to release Wasmtime with a 1.0 version. In doing so this
proposal also defines what it means to be 1.0 and details areas such as:
- What is Wasmtime's stability story.
- What will Wasmtime's release process be.
- What users can expect from Wasmtime going forward.
RFC notifications bot (Aug 18 2021 at 22:21):abrown submitted PR review.
RFC notifications bot (Aug 18 2021 at 22:21):abrown created PR review comment:
I would not expect this as a user: I would expect the project to follow semantic versioning and decide every four weeks if the release should be major or minor. Users then get some type of clue about the changes coming to them ("oh, this is a major change, some API must have changed--let me go look at the release notes!"). It seems that the "match the Wasmtime version with the bindings version" can be solved some other way (e.g.,
wasmtime-py [Wasmtime version]-[binding version])
RFC notifications bot (Aug 18 2021 at 22:26):abrown submitted PR review.
RFC notifications bot (Aug 18 2021 at 22:26):abrown created PR review comment:
Yes! I feel the project _should_ keep track of breaking changes; the user will have a lot harder time figuring that out (see my semver comment above).
RFC notifications bot (Aug 18 2021 at 22:36):pchickey submitted PR review.
RFC notifications bot (Aug 18 2021 at 22:36):pchickey created PR review comment:
That may have been a usual expectation at some point in the past, but many projects have changed their versioning policies in recent years to this sort of release train, for example the LLVM project switched to this in 2016 http://blog.llvm.org/2016/12/llvms-new-versioning-scheme.html
RFC notifications bot (Aug 18 2021 at 22:36):pchickey edited PR review comment.
RFC notifications bot (Aug 19 2021 at 09:17):bjorn3 submitted PR review.
RFC notifications bot (Aug 19 2021 at 09:17):bjorn3 created PR review comment:
Most of these projects do have breaking changes every release now. Wasmtime probably won't have breaking changws every time when releasing at the suggested interval.
RFC notifications bot (Aug 19 2021 at 09:46):tschneidereit submitted PR review.
RFC notifications bot (Aug 19 2021 at 09:46):tschneidereit created PR review comment:
I started out with this view, too, but talking through it with Alex and others made me pretty convinced that proper fine-grained semver handling would be very, perhaps prohibitively, hard.
As described in the explainer, the language embeddings are a major consideration here: I think it's very clearly desirable to keep their version numbers aligned with Wasmtime's (because you shouldn't have to maintain a mapping table from language embedding version to Wasmtime version in your head). I really don't see a way to achieve this without regularly scheduled version bumps, in particular if we want to enable third-party embeddings to stay version aligned, too.
RFC notifications bot (Aug 19 2021 at 09:48):tschneidereit submitted PR review.
RFC notifications bot (Aug 19 2021 at 09:48):tschneidereit created PR review comment:
This might be too strong a statement as written: as discussed elsewhere in the explainer, the idea is very much to be mindful of API stability and not make updating harder than it needs to be. So actually breaking changes should be documented as such, which means that this sentence should still be true as per this plan.
RFC notifications bot (Aug 19 2021 at 13:21):bjorn3 created PR review comment:
If we bump the major version of all crates part of wasmtime to the same version even if only some crates have breaking changes, I guess it wouldn't be much of a stretch to bump the major version every time any language embedding has a breaking change.
RFC notifications bot (Aug 19 2021 at 13:21):bjorn3 submitted PR review.
RFC notifications bot (Aug 19 2021 at 13:26):tschneidereit submitted PR review.
RFC notifications bot (Aug 19 2021 at 13:26):tschneidereit created PR review comment:
That's true, yes. But I think it'd in reality have the effect of making it harder for embeddings to improve in ways that require these kinds of changes. In particular for lower-tier ones.
As Pat mentioned above, we're not proposing to tread particularly new ground here: LLVM does pretty much the same, and I'd assume for much the same reasons.
RFC notifications bot (Aug 19 2021 at 14:29):alexcrichton submitted PR review.
RFC notifications bot (Aug 19 2021 at 14:29):alexcrichton created PR review comment:
Personally my experience is that maintaining a strategy where you figure out whether breaking changes or not have happened is onerous and difficult to maintain. From the perspective of authoring Wasmtime it means that we'll quickly fall into a trap of "oh when's the right time to have a breaking change?" and try to batch things up, in my opinion creating unnecessary tension for landing improvements and updates to the engine.
From a user's perspective I would consider Wasmtime as a "weighty" enough dependency that even if you want to update patch versions there's likely to be some sort of gotcha depending on how deeply integrated it is into your system. This means that we, as maintainers, will likely spend a lot of time trying to chase down unintended consequences of changes in embeddings we have limited access to.
I feel that the best compromise between these worlds is regularly expected breaking changes (mild breaking) and users having the expectation that they'll have to bump the version. This creates a number of motivations towards what I feel is "the right thing to do":
- As authors we don't feel hindered landing breaking changes (minor ones)
- As authors we are vigilant to clearly document any possible breakage in release notes
- Users should expect that upgrades are hassle-free, not literally free
- Users should expect to read the release notes and determine risks to their application, if any
I don't think it's viable to have a situation where we, as the Wasmtime project, take on 100% of the work of keeping everyone else's applications running. Our job, in my opinion, is providing a solid WebAssembly runtime that users can rely on. Requiring a small amount of end-user assistance, though, greatly helps actually releasing new features and not feeling hindered in development.
RFC notifications bot (Aug 19 2021 at 14:31):alexcrichton submitted PR review.
RFC notifications bot (Aug 19 2021 at 14:31):alexcrichton created PR review comment:
Yeah I'm not getting across what I was thinking with this bullet. It's moreso that having an unpredictable versioning scheme makes it harder for releases, automation, and such. Having to programmatically detect a breaking change somehow does not seem super viable and could complicate what's already bound to be a pretty complicated set of automation for these releases.
I consider it a bare minimum that at all times we're keeping track of breaking changes and logging them in a changelog of some sort.
RFC notifications bot (Aug 19 2021 at 16:56):bnjbvr submitted PR review.
RFC notifications bot (Aug 19 2021 at 16:56):bnjbvr submitted PR review.
RFC notifications bot (Aug 19 2021 at 16:56):bnjbvr created PR review comment:
It might be an open question at the moment, but still curious: until there is such a separate Cranelift release process, does it mean that the Cranelift shipped with Wasmtime LTS is actually in LTS mode itself? It might have different requirements in terms of backports for bugfixes and sec fixes (e.g. some of these might involve only Cranelift and nothing in Wasmtime, and conversely), so this really calls for a different release process as you suggest. Still, it would be interesting to get a clear stand on where Cranelift is at the moment in terms of support and will be once this RFC gets merged.
RFC notifications bot (Aug 19 2021 at 16:56):bnjbvr created PR review comment:
(a bit nit-picky, sorry) Instead of
lts-latest/lts-oldest, how aboutlts/lts-previous?lts-oldestnot technically the _oldest_ one, and previous doesn't leave room to any possible doubt imo.
RFC notifications bot (Aug 19 2021 at 17:08):cfallin submitted PR review.
RFC notifications bot (Aug 19 2021 at 17:08):cfallin created PR review comment:
In my view at least, it makes sense for us to uphold the same LTS promises with Cranelift as with Wasmtime with regard to bug-fixes, because a security bug in Cranelift is a security bug in Wasmtime from the point of view of the consumer. So, yes, I would expect that we backport any correctness or security fix to codegen.
(Also FWIW, it's hard for me to imagine at the moment what would cause Cranelift to diverge to another release process, as riding along with Wasmtime gives all of the needed framework for managing stable branches, security patches, etc. -- at most we might introduce a stronger API-stability guarantee on
cranelift-codegenand friends -- but for completeness, if Cranelift does ever diverge, I would expect it to keep a very similar LTS and backports policy.)
RFC notifications bot (Aug 19 2021 at 18:05):fitzgen submitted PR review.
RFC notifications bot (Aug 19 2021 at 18:05):fitzgen created PR review comment:
How about just many
lts-$VERSIONbranches, the two most recent being the supported ones? No jumpinglts-latestfrom 10.0 to 15.0 etc.
RFC notifications bot (Aug 19 2021 at 18:21):alexcrichton submitted PR review.
RFC notifications bot (Aug 19 2021 at 18:21):alexcrichton created PR review comment:
Yeah in retrospect I don't think
latestandoldestare good names, but I like @fitzgen's idea of just having a branch for all of them. It means they become irrelevant and ossified 10+ months after their creation but that's not really an issue with git
RFC notifications bot (Aug 19 2021 at 18:28):alexcrichton submitted PR review.
RFC notifications bot (Aug 19 2021 at 18:28):alexcrichton created PR review comment:
My view on this is that as part of Wasmtime's 1.0 stability promise we're promising quality in our transitive dependencies, including cranelift, so any cranelift bugs affecting Wasmtime is something we have to take care of in Wasmtime (patch releases, etc).
If there's a bug in Cranelift, though (security or not), which we probably know cannot affect Wasmtime (e.g. in an architecture not supported or something like that) I think the process is a bit more ambiguous. Most of this RFC is about LTS/point releases for Wasmtime, and I don't think it would apply in the case that the bug from Cranelift isn't exposed through Wasmtime's usage.
Not to say that point releases won't happen, just that I don't think it would always necessitate a Wasmtime point release as well. Ideally the release process for these two would be diverged so that Cranelift could be released without the other (and in theory Wasmtime as well if Cranelift didn't have changes, but that seems unlikely), but that may take some more infrastructure.
RFC notifications bot (Aug 20 2021 at 09:06):bnjbvr submitted PR review.
RFC notifications bot (Aug 20 2021 at 09:06):bnjbvr created PR review comment:
whynotboth.jpg: Could have a moving tag for
current-ltsandprev-lts+ fixed tags with version numbers. But I'll stop with the bikeshedding here, having tags with the version number would be already quite good too!
RFC notifications bot (Aug 20 2021 at 09:33):tschneidereit submitted PR review.
RFC notifications bot (Aug 20 2021 at 09:33):tschneidereit created PR review comment:
If there's a bug in Cranelift, though (security or not), which we probably know cannot affect Wasmtime (e.g. in an architecture not supported or something like that) I think the process is a bit more ambiguous. Most of this RFC is about LTS/point releases for Wasmtime, and I don't think it would apply in the case that the bug from Cranelift isn't exposed through Wasmtime's usage.
I think this is the key point. The way Cranelift is tested right now in particular doesn't really allow us to make strong claims about its stability and quality properties when used to compile non-Wasm content, and perhaps even when used in other embeddings in general.
I think that will change, but this RFC is about Wasmtime, and in that context it makes sense to treat Cranelift purely as a dependency to Wasmtime, and look at its stability and quality properties through that lens.
it's hard for me to imagine at the moment what would cause Cranelift to diverge to another release process
I also agree with this, though! When Cranelift gains a stability story of its own, that explicitly encompasses other uses, I don't think that'd be a reason for it to also gain a completely separate release process. It seems much better to broaden the scope of guarantees we give for it by introducing the required testing strategies, etc, but still let it benefit from being included in the same release train schedule, etc.
RFC notifications bot (Sep 09 2021 at 10:32):PiotrSikora submitted PR review.
RFC notifications bot (Sep 09 2021 at 10:32):PiotrSikora created PR review comment:
Monthly releases (e.g. releasing Wasmtime on the first Tuesday of every month) might be easier to reason about, seeing as even this document uses "once a month" and "every 4 weeks" interchangebly.
RFC notifications bot (Sep 09 2021 at 10:32):PiotrSikora submitted PR review.
RFC notifications bot (Sep 09 2021 at 10:32):PiotrSikora created PR review comment:
This means, for example, that Wasmtime 2.0 will be released 4 weeks after
RFC notifications bot (Sep 09 2021 at 10:32):PiotrSikora created PR review comment:
to worry about how the version number of your language's Wasmtime embedding
RFC notifications bot (Sep 09 2021 at 10:32):PiotrSikora created PR review comment:
An LTS version is considered "supported" for 10 release cycles, or 40 weeks (~9
RFC notifications bot (Sep 09 2021 at 10:32):PiotrSikora created PR review comment:
time, each supported for 9 months at a time**.
RFC notifications bot (Sep 09 2021 at 10:32):PiotrSikora created PR review comment:
Wasmtime 1.0. After one year Wasmtime will be at 14.0. At this time it is not
RFC notifications bot (Sep 09 2021 at 10:32):PiotrSikora created PR review comment:
Since you've included
witx-bindgenhere, is this expected to be used by all Bytecode Alliance projects or only by Wasmtime (with sub-crates) and its language bindings?
RFC notifications bot (Sep 09 2021 at 10:32):PiotrSikora created PR review comment:
it's guaranteed to be in the next release, unless reverted. Note that bug fixes and such for
RFC notifications bot (Sep 09 2021 at 10:32):PiotrSikora created PR review comment:
Maintaining LTS releases is a costly process, so starting with two when it's not clear that even one is necessary seems like an overkill. I'd suggest starting with one, perhaps with a slightly longer cycle (~6 months), and adding another one only if there is a clear demand for it.
RFC notifications bot (Sep 09 2021 at 10:32):PiotrSikora created PR review comment:
be expected to upgrade Wasmtime at least every 9 months, likely every 5 months.
RFC notifications bot (Sep 09 2021 at 10:32):PiotrSikora created PR review comment:
lines up with that of Wasmtime itself. E.g., if you use `wasmtime-py 7.0`, you
RFC notifications bot (Sep 09 2021 at 11:27):tschneidereit submitted PR review.
RFC notifications bot (Sep 09 2021 at 11:27):tschneidereit created PR review comment:
Great question! We do expect
witx-bindgento be used by other projects (both in and outside the BA), so in some sense it wouldn't need to have a release schedule aligned with Wasmtime's, and there's nothing tyingwitx-bindgento Wasmtime in some fundamental way.At the same time, I think there are practical reasons for doing so, at least for the time being. As of now, simply as a matter of fact we're making heavy use of Wasmtime in the development process of witx-bindgen. And increasingly that'll be the case the other way around as well. Aligning the release processes means we can rely on available feature sets in these development processes.
Should over time other uses of
witx-bindgenbecome dominant, it'd make sense to reevaluate this alignment.
RFC notifications bot (Sep 09 2021 at 11:37):tschneidereit submitted PR review.
RFC notifications bot (Sep 09 2021 at 11:37):tschneidereit created PR review comment:
The worry we have about only having a single LTS release is that it basically means LTS users have a zero-length upgrade window when a new LTS release is published. That seems counter to what someone in need of an LTS release would want :)
Plus, our hope is that we'll be able to automate our processes around releases to a very high degree, and to make use of that same automation for LTS releases as well—which ideally will address some of the cost concerns.
At the very least, I'd hope that maintaining two LTS releases wouldn't be that much more burdensome than maintaining a single one, in particular given the tightly circumscribed scope of changes we intend to make available to LTS releases.
I think it's worth at least thinking through what it'd mean to start without an LTS release at all: you're right that the need for one is more derived from precedent in other projects than from immediate requests from Wasmtime users.
An option could be to split the LTS part out into a separate proposal that could simmer for a bit longer. The one worry I'd have about that is that it's probably easier to establish the required processes for all of this in one go, so getting up the energy to do the LTS stuff later might simply be harder. But @alexcrichton would much better be able to comment on that part than me :)
RFC notifications bot (Sep 09 2021 at 12:46):PiotrSikora submitted PR review.
RFC notifications bot (Sep 09 2021 at 12:46):PiotrSikora created PR review comment:
I meant the cost of backporting fixes, which often means rewriting the change from scratch, since trivial cherry-picks to a branch from a few months ago rarely work in an actively developed codebase... Having said that, I'm not very familiar with the codebase or the development pace of Wasmtime, so perhaps that won't be an issue.
RFC notifications bot (Sep 09 2021 at 12:50):PiotrSikora submitted PR review.
RFC notifications bot (Sep 09 2021 at 12:50):PiotrSikora created PR review comment:
Right, at the same time, for someone using
witx-bindgenonly to generate guest bindings, seeing "random" major version bumps without any significant changes inwitx-bindgenmight be a weird experience.
RFC notifications bot (Sep 09 2021 at 13:04):tschneidereit submitted PR review.
RFC notifications bot (Sep 09 2021 at 13:04):tschneidereit created PR review comment:
That's true, yes, and to some degree more so than for the Wasmtime embeddings.
Our processes in developing
witx-bindgenitself would become increasingly more complicated if we didn't align version numbers like this that I think the benefits outweigh these costs, though. Which seems even more true given that, when using it as a guest bindings generator, one should only need to updatewitx-bindgento pull in significant changes. And even then, the major version upgrade should be trivial—except ifwitx-bindgenhas significant API changes which would necessitate a major version bump anyway.
RFC notifications bot (Sep 09 2021 at 14:38):alexcrichton updated PR #14 from wasmtime-1.0 to main.
RFC notifications bot (Sep 09 2021 at 14:42):alexcrichton created PR review comment:
My main goal is to make sure that the releases are as automate-able as possible to make the work of a release on us in theory "make sure the changelog is up-to-date and merge a PR". In that sense I don't think that the precise cadence really matters too too much. I'm expecting users will have some sort of regular process to update
wasmtimeand/or notifications through some bot like dependabot.All that to say I think the precise number doesn't really matter too too much and I think that whatever is the easiest and most reliable to automate is probably the way to go. I chose a weekly-based cadence since that's what rust-lang/rust does (although 6 weeks instead of 4) and I know there's a fair bit of tooling for things like "when's the next release?" which I believe is easier with a machine-predictable cadence rather than a human-predictable cadence. I don't think it's really onerous one way or another though.
RFC notifications bot (Sep 09 2021 at 14:42):alexcrichton submitted PR review.
RFC notifications bot (Sep 09 2021 at 14:50):alexcrichton submitted PR review.
RFC notifications bot (Sep 09 2021 at 14:50):alexcrichton created PR review comment:
AFAIK most of us working on Wasmtime haven't dealt much historically with maintaining LTS releases of software, so in that sense that may be why we're being a bit overly ambitiuos here. Given how different the LTS process is I don't forsee it being easier to add now vs later if we decide to do it, so in that sense I don't think there's urgency to add it earlier while the release process is all in someone's head.
An alternative model could be one where the LTS windows overlap for 1 release instead of 5 releases. For example 5.0 would be an LTS release for the 5.0-11.0 releases, but when 12.0 is released only 10.0 is a supported LTS release. Something like that means that our window for supporting multiple LTS releases is much smaller and because 11.0 is probably pretty similar to 10.0 any work for backports would primarily be from 11.0 to 5.0 rather than 11.0 to 10.0 (which is likely cherry-pick-able). This would still have the property where there's a window where users can upgrade but still get bug fixes if necessary.
Naturally though I'd expect that anything here can change at any time with sufficient consensus. It's probably a lot easier to add an LTS process than it is to remove one as well. Given that we don't have anyone specifically asking for LTS it may not be a bad idea to not support it initially. We could have some initial releases and see if users find the update-every-4-week cadence to be too onerous. We'd probably, in that case, update the bugfix policy to backport to the current release at the time of the bugfix and perhaps the last 1-2 releases prior to that as well.
Overall I don't personally know the best thing for Wasmtime at this time. We made an initial guess with 2 LTS releases at this cadence, but it's mostly a guess that it'll work out.
RFC notifications bot (Sep 09 2021 at 14:52):alexcrichton submitted PR review.
RFC notifications bot (Sep 09 2021 at 14:52):alexcrichton created PR review comment:
Currently
witx-bindgenhas a lot of various components which I suspect is going to end up like thewasm-toolsrepo where each component is separately managed and versioned. I expect anything wasmtime-related to be tied to wasmtime's own version, but things like the JS code generator or Rust generator for compiled-to-wasm-code probably don't need to be tied to weasmtime's own release process.
RFC notifications bot (Sep 10 2021 at 08:18):sparker-arm submitted PR review.
RFC notifications bot (Sep 10 2021 at 08:18):sparker-arm created PR review comment:
If customers aren't requesting an LTS, it might be worth postponing until the release process has had some time to mature. I completely agree that backporting is very costly, even in relatively mature code bases, so it could be too much work for little gain.
And sorry if I missed it, but is there a plan for stabilization and testing before a release drops, or are we going for an "always releasable trunk"? Particularly if we go for LTS releases, it would be really useful to give developers and users a reasonable amount of time to test before hand, otherwise we'd probably just get a bunch of issues a couple of weeks after the release and it wouldn't be great for trust to push out a buggy LTS.
RFC notifications bot (Sep 14 2021 at 21:16):alexcrichton submitted PR review.
RFC notifications bot (Sep 14 2021 at 21:16):alexcrichton created PR review comment:
I posted a comment at the bottom about this, but personally I agree with y'all's warnings and conclusions that maybe we don't need LTS at all.
For releases yeah the plan is that
mainis always green with CI and always release-able.
RFC notifications bot (Sep 16 2021 at 14:45):alexcrichton updated PR #14 from wasmtime-1.0 to main.
RFC notifications bot (Sep 17 2021 at 06:47):sparker-arm submitted PR review.
RFC notifications bot (Sep 27 2021 at 15:57):alexcrichton merged PR #14.
Last updated: Nov 22 2024 at 17:03 UTC