Andrew Brown (Aug 09 2021 at 17:45): Andrew Brown (Aug 09 2021 at 17:45):As discussed here, depending on the Wasm spec interpreter as a submodule would dramatically increase the checkout size. I propose we create https://github.com/bytecodealliance/wasm-spec-interpreter as a mirror of the interpreter directory found under https://github.com/WebAssembly/spec. The additional benefit of this would be to allow for a place to merge in changes from various proposals and stage bug fixes. Thoughts?
Andrew Brown (Aug 09 2021 at 17:46):cc: @Till Schneidereit, @Chris Fallin, @Alex Crichton
Chris Fallin (Aug 09 2021 at 17:48):This seems like the most reasonable approach to me; alternatives are vendoring into the wasmtime repo or depending on the whole WebAssembly/spec repo, which as pointed out is quite large. As long as we're going to pull out the subdirectory it makes sense to have a separate repo IMHO
Till Schneidereit (Aug 10 2021 at 09:32):that might be the best option, though I find it a bit unfortunate to have to keep a mirror around just because of this one use :frown:
A potential alternative would be manually do the clone as part of the build script for our use of the interpreter. I'm doing roughly that with SpiderMonkey here: https://github.com/tschneidereit/spidermonkey-wasi-embedding/blob/main/build-engine.sh
The goal here was to make it easy to do local development by just cloning the repo very cheaply (or including it as a submodule), but also enabling building from source in situations where you don't want to depend on external binary blobs.
Not the same situation for sure, but the approach of storing ~ the same information as .gitmodules in a slightly different form, and then manually cloning just that revision (and doing a shallow clone in the case of the interpreter) might work?
Andrew Brown (Aug 10 2021 at 16:05):I'm fine with either the mirror or pulling it in the build script. I know @Alex Crichton has mentioned that a mirror would be nice since we expect to need a place to patch the spec interpreter temporarily as well as a place to merge in various proposals. Does that change the equation?
Till Schneidereit (Aug 10 2021 at 16:11):hmm, yeah, that's a good point. Let me think about this a bit and perhaps check in with a couple of folks, but yeah, that might be a solid reason for having our own full mirror
Till Schneidereit (Aug 10 2021 at 16:14):oh, but if we do the mirror thing, then I think it'd be valuable for that mirror to be for the entire repository, not just the interpreter. That'd enable us to do direct PRs against the spec instead of having to hand-craft patches all the time. So perhaps we do both—have a mirror and do the manual checkout in the build script?
Till Schneidereit (Aug 10 2021 at 17:07):ok, I checked in with some folks, and have firmed up my take on this :smile:
I do think having the mirror is good, but also that it should be a full-repo mirror for the above-mentioned reasons. I hope the manual checkout in the build script is viable?
The one thing we should make sure is to give the repo a name/description that make clear what they are and aren't. Lin pointed out that "spec" would make it sound like a BA spec, for example :smile: Perhaps something sort of annoying like "spec-mirror-for-fuzzing" would make sense?
Andrew Brown (Aug 10 2021 at 17:17):How about wasm-spec-mirror with a description like: "A mirror of https://github.com/WebAssembly/spec for fuzzing"
Andrew Brown (Aug 10 2021 at 17:19):Actually, wait: we don't need the mirror just yet if we are going to manually checkout the repo... Why not delay creating the repository until it is needed? (E.g. we need to patch it or merge in proposals)
Andrew Brown (Aug 10 2021 at 17:21):The full mirror is not as helpful as a "just the interpreter" mirror because it contains all the extra weight that @bjorn3 was worried about... another reason to delay making it
Chris Fallin (Aug 10 2021 at 17:21):@Andrew Brown do you have any insights about the timeline of merging the SIMD spec into the main WebAssembly/spec repo? I wasn't able to find anything (open PR or issue for example)
Chris Fallin (Aug 10 2021 at 17:22):that could be a reason to create our "merged for-fuzzing mirror" soon
Andrew Brown (Aug 10 2021 at 17:25):I should have asked that in the latest SIMD meeting but instead I asked when Safari will implement Wasm SIMD (answer: no one knows)
Andrew Brown (Aug 10 2021 at 17:26):so, not really sure
Andrew Brown (Aug 10 2021 at 17:26):I'll just create the mirror and we can figure this out later; sort of want to pop this PR off the stack
Andrew Brown (Aug 10 2021 at 18:16):Ok, https://github.com/bytecodealliance/wasm-spec-mirror is created and the latest commit on https://github.com/bytecodealliance/wasmtime/pull/3124 retrieves the repository, shallowly, in the build.rs
Last updated: Oct 23 2024 at 20:03 UTC