Andrew Brown (Mar 14 2020 at 02:54): Andrew Brown (Mar 14 2020 at 02:54):I've been seeing some weird CI failures (https://github.com/bytecodealliance/wasmtime/pull/1263, https://github.com/bytecodealliance/wasmtime/pull/1318, https://github.com/bytecodealliance/wasmtime/pull/1319); @Andy Wortman has a theory that it may disk space related. The error is error: linking with 'cc'. Can anyone with Azure permissions investigate?
Andrew Brown (Mar 14 2020 at 02:55):Here's an example: https://github.com/bytecodealliance/wasmtime/pull/1319/checks?check_run_id=506868460#step:16:43
Andrew Brown (Mar 16 2020 at 16:58):@Alex Crichton, your name came up in the Cranelift meeting on this issue; @fitzgen (he/him) thought you might have insight into this. Also, @Andy Wortman has a separate issue that might also be demonstrating the same behavior
Alex Crichton (Mar 16 2020 at 18:25):unfortunately I don't really know what's goign on with CI
Alex Crichton (Mar 16 2020 at 18:25):my best guess is we should roll back nightlies (e.g. use like nightly-YYYY-MM-DD) until one works
Alex Crichton (Mar 16 2020 at 18:27):I can investigate more tmw when I'm back at work
iximeow (Mar 16 2020 at 20:08):as a datapoint: building and running wasmtime tests by the github actions actions results in ~17gb of Stuff after running all the steps listed in a nightly build. measured in a fresh clone of the repo, so that's not with any pre-existing target content.
i notice there are a few steps that are _only_ run for nightly tests, so if it is in fact a disk space issue that might explain why it's only seen on nightly test runs
iximeow (Mar 16 2020 at 22:25):datapoint 2: building with non-nightly steps results in a build that rounds up to 16gb but is in fact 15.63gb! so we at least cross the 16gb threshold on nightly vs nonnightly
Alex Crichton (Mar 17 2020 at 14:40):I'm going to try to continue investigating this error today and see if I can get to the bottom of it
Alex Crichton (Mar 17 2020 at 15:29):ok bisecting a bunch of nightlies at https://github.com/alexcrichton/wasmtime/actions/runs/57523147 and https://github.com/alexcrichton/wasmtime/actions/runs/57538657 shows this is pretty unlikely to be a regression
Alex Crichton (Mar 17 2020 at 15:29):testing the last good commit -- https://github.com/alexcrichton/wasmtime/runs/514044968?check_suite_focus=true on todays' nightly however shows that this is still failing
Alex Crichton (Mar 17 2020 at 15:29):gonna try some disk space savings next
Alex Crichton (Mar 17 2020 at 16:33):ok looks like disk space was the issue -- https://github.com/bytecodealliance/wasmtime/pull/1341, I'll continue to send a few PRs to save disk space on CI so we buy ourselves lots of time before hitting this again
Alex Crichton (Mar 17 2020 at 16:33):I'm sad the linker didn't print anything more useful, I thought it actually printed out if it ran out of disk space...
Alex Crichton (Mar 17 2020 at 16:35):that PR saved about 3GB, and https://github.com/bytecodealliance/wasmtime/pull/1343 should save another 7 or so
Alex Crichton (Mar 17 2020 at 16:35):er, 4*
Last updated: Nov 22 2024 at 16:03 UTC