@fitzgen (he/him) currently we're building z3 and peepmatic on all the test builders, was that intentional?
it looks like CI time for windows has spiked to 45m
yes, it is intentional
I don't know why windows is so slow to compile z3
we can't use system z3 because it is missing symbols (at least on ubuntu) :-/
I guess we could look into fetching from https://github.com/Z3Prover/z3/releases and using those prebuilt Z3s
Hm why do we want to build peepmatic and such on all builders?
It seems like architecturally it's so expensive we wouldn't really want to do that
but instead we should focus it on one builder, but no need to test all channels on all platforms in all modes
I'm not sure if it's z3 specifically or something else that's slow
well, since getting windows builds working was somewhat difficult, I wouldn't want to remove its testing
I'm not sure if it's z3 specifically or something else that's slow
it is building z3 that is slow here
was that something we did?
or was that upstream in the z3 crate?
It just feels weird to blow 20 minutes testing a thing that's off-by-default for everyone
we statically link our own build of z3 because system packages are unreliable and missing symbols and such
that's a long time for a PR to be sitting on CI
but that logic is in z3, right?
not in wasmtime?
like it seems like z3's windows support shouldn't be tested by wastime
wasmtime*
it is a feature of the z3 crate that I added and turned on
but why would we be responsible for testing it?
in the wasmtime ci that is
the z3 crate's ci does test it; we aren't responsible for testing that, its just that we test everything on all of our platforms
but why would we test specifically on windows?
like why isn't one builder sufficient?
I thought we were just going to have one builder verify that the peepmatic blob is up-to-date
why do we test anything on windows?
hm that feels like a disingenuous question
there is only one builder that tests that the generated peephole optimizations are up to date
like we obviously have tons of windows-specific code in wasmtime
but why would anything in peepmatic be windows-specific?
right what I mean is that i thought we were only going to build peepmatic on that one builder
lets back up a bit
when peepmatic is on by default, do you want to still only test it on ubuntu?
that's what I would expect, yeah
like, it seems we want to test everything we ship on all platforms we are shipping them on
b/c it's development model is "we don't care how slow it is to build
I think you're conflating this though
we're not shipping peepmatic anywhere
we ship wasmtime everywhere
we only ship the peepmatic-generated-blob
so I don't think there's much benefit to getting so much coverage
like it's not like it's wrong but it is costly
much of the test coverage for the peepmatic-generated blob is in crates that depend transitively on z3
and apparently very costly
but we test the blob in all configurations if we enable it by default
I agree we need to make the cost lower, and I suggested a potential solution for investigation: fetch pre-built libz3.so
s from the z3 github releases page
z3 is only needed generating the blob, and I'm saying I don't think there's any need to test blob-generation on all platforms
we are not testing blob generation on all platforms
I mean yeah we can make it faster but I still don't see the point in testing it on all platforms
beyond making us feel good about ourselves
aren't we?
like we're running cargo test -p peepmatic
basically
no, only the rebuild peephole optimizers builder is regenerating the blob
yes, we are running cargo test -p peepmatic
something about cargo test -p peepmatic
is bringing in z3?
like something is building z3 on the "test everything" builders
peepmatic
depends on z3
and I thought we were not going to do that
we aren't building z3 on the dist builders
which is what we talked about before
My understanding from reading the patch is that we don't link the z3 engine into wasmtime. Is that the case?
sorry I should have clarified, but dist/test is what I was talking about there
My understanding from reading the patch is that we don't link the z3 engine into wasmtime. Is that the case?
this is correct
I think we should build z3 on precisely one builder
the one that does peepmatic/blob testing
Offhand, that sounds reasonable to me. In theory z3 or the peepmatic code could differ depending on the host, but that should be very rare and arguably not something we need to be testing for every CI run
I don't disagree with the conclusion per se, but I guess I am confused where the line is drawn. when peepmatic is enabled by default, do we test it on all the testers?
I think it depends on what you mean by testing peepmatic, I personally think that no matter what at all times we should only build z3 in one place one itme
and that's possible b/c wasmtime doesn't depend on z3
also, if testing peepmatic is fast, does it change the calculus? because all the time currently is spent in building z3, and if we don't do that, then it will be pretty fast
("don't build z3" == "fetch built release from Z3 github releases")
It would from a practical point of view, but I would personally say we still shouldn't do it
like it'd change to the point that I still not want to build the z3 crate everywhere but I wouldn't care enough to pursue it
I hold out hope that someday CI machines will have better ways to cache builds of dependencies between runs, but until then, I agree.
aside: it might be nice to set up sccache for our ci
Normal users building from source on Windows won't regenerate the bytecode, because that's behind a feature flag disabled by default, so even if z3/peepmatic would generate different bytecode on Windows, it won't affect people's actual experience by default
I guess I am fearful of it breaking on windows and no one noticing, but I guess that is pretty unlikely given that the z3 repo's CI is testing windows builds now
I have no opinion on most of this conversation, but Adam recently setup Lucet with CircleCI caching the target directory and it made a significant improvement on build times. I’m not sure if GH CI can do that sort of caching.
(We need CircleCI to test our userfaultfd memory management backend, that syscall isn’t available on GH CI)
now there's a reason for picking a CI system I haven't heard of before
https://github.com/bytecodealliance/wasmtime/pull/1714
Would it make sense to only run this at all if the peepmatic input changed in a PR?
or any configuration affecting it, I guess
figuring out all the things that could affect it could be a game a wack a mole, I think
e.g. changing a clif operator could require changes to the glue between peepmatic and cranelift
ah, that's a good point!
Last updated: Dec 23 2024 at 13:07 UTC