Lann Martin (Oct 12 2023 at 19:30): Lann Martin (Oct 12 2023 at 19:30):If there are any wit-bindgen repo maintainers around, main is currently broken due to incompatible deps: https://github.com/bytecodealliance/wit-bindgen/pull/698
Peter Huene (Oct 12 2023 at 19:48):merged
Joel Dice (Oct 12 2023 at 20:19):One more tweak needed, apparently: https://github.com/bytecodealliance/wit-bindgen/pull/700
Peter Huene (Oct 12 2023 at 23:22):Merged
Lann Martin (Oct 16 2023 at 12:43):Should it be surprising to us that these incompatible versions made it through CI?
Alex Crichton (Oct 16 2023 at 13:41):Unfortunately no it's not surprising because the wasm-tools repo itself was always "self consistent" since everything is tested against itself. The problem is that the wit-component released bumped its dependency on wit-parser from 0.11 to 0.12, and so downstream applications that used wit-component 0.14 and wit-parser 0.11, the prior latest, all of a sudden would no longer compile due to the version mismatch in wit-parser. Effectively downstream applications had no way of depending on the latest correctly.
One possible way to catch this on CI would be to have automatic validation that a minor version bump is indeed a minor bump, but I'm not sure if such tooling in Rust has progressed to that point.
Lann Martin (Oct 16 2023 at 14:17):Right, but I'm thinking about wit-bindgen's CI. Based on https://github.com/bytecodealliance/wit-bindgen/issues/697 it appears that wit-bindgen-rust became unbuildable when the wit-component dep was bumped.
Alex Crichton (Oct 16 2023 at 14:32):oh wit-bindgen would have caught it I think if there were a PR or something, but this is a publication of wit-component that broke preexisting wit-bindgen so I'm not sure there's much that could be done about that
Lann Martin (Oct 16 2023 at 14:36):ah thanks; I had been under the vague impression that the breakage was caused by a dep change in wit-bindgen, but that makes sense
Victor Adossi (Oct 17 2023 at 23:51):Would it be worth it to write some E2E tests that run from places like wit-component -- I'd love to take this on if so.
The simplest version, adding tests to wit-component (since that's where things actually broke) that build it locally and use it in concert with downstream stuff. We could limit these tests to release PRs only (or even PRs with semver minor changes).
A slightly more complex version of this would be creating an e2e-tests/ecosystem-tests/ecosystem-stability repo that is triggerable from other actions -- some upstream dep can provide a version/git SHA/etc and be compiled in/used from a supported use case/build process.
I personally like the second approach (I could also create it without much upstreaming initially until the prototype is ready/complete), because I think it would be a good place to point people for examples and a (possibly generated!) support matrix.
Lann Martin (Oct 18 2023 at 12:42):I believe @Kyle Brown has been working on some sort of similar test suite.
Alex Crichton (Oct 18 2023 at 14:40):I think it'd be reasonable to have such tests, yeah, but I'm not sure how they would be integrated in to the release process for wasm-tools
Robin Brown (Oct 18 2023 at 17:54):I've done some design and prototyping for a Guest Language test suite but haven't had time to work on it lately. The design document can be found here: https://hackmd.io/qeVkBYYWQ1C1uiKYk-HzZw
Victor Adossi (Oct 19 2023 at 08:40):Thanks @Kyle Brown! Taking a look at what you have, I think it's slightly different from what I want to build -- I want tests that are guest language independent -- basically testing the tooling that is being produced. It's less of a conformance suite (forgive me if I misread your intent there!) and more of a have-a-machine-make-sure-everything-works repo, along with a place we can point new people for setups that work (ex. exactly which adapter version, with exactly which release of wasmtime, etc.)
Likely these tests will have a very large matrix, so we can know which combinations of versions of which tooling work together, essentially.
The way I'm envisioning it is that it's a repo with as much up-to-date information as possible on what works together, possibly automatically updated, and other repos could trigger workflows on it to test downstream usability.
Alternativley, the primary output of the crate might even be a Github Action, that's callable from other projects -- but the basic point is to start from near nothing (as a downstream user with) and use the tooling in the ecosystem at various versions.
Victor Adossi (Oct 19 2023 at 08:43):It's probably easier to show rather than tell, so I'll try to put something together and show that
Alex Crichton (Oct 19 2023 at 15:37):ah yeah if you've got a proof-of-concept that'd be great to look at!
Last updated: Nov 22 2024 at 16:03 UTC