As discussed here, depending on the Wasm spec interpreter as a submodule would dramatically increase the checkout size. I propose we create https://github.com/bytecodealliance/wasm-spec-interpreter as a mirror of the interpreter directory found under https://github.com/WebAssembly/spec. The additional benefit of this would be to allow for a place to merge in changes from various proposals and stage bug fixes. Thoughts?
cc: @Till Schneidereit, @Chris Fallin, @Alex Crichton
This seems like the most reasonable approach to me; alternatives are vendoring into the wasmtime repo or depending on the whole WebAssembly/spec repo, which as pointed out is quite large. As long as we're going to pull out the subdirectory it makes sense to have a separate repo IMHO
that might be the best option, though I find it a bit unfortunate to have to keep a mirror around just because of this one use :frown:
A potential alternative would be manually do the clone as part of the build script for our use of the interpreter. I'm doing roughly that with SpiderMonkey here: https://github.com/tschneidereit/spidermonkey-wasi-embedding/blob/main/build-engine.sh
The goal here was to make it easy to do local development by just cloning the repo very cheaply (or including it as a submodule), but also enabling building from source in situations where you don't want to depend on external binary blobs.
Not the same situation for sure, but the approach of storing ~ the same information as .gitmodules
in a slightly different form, and then manually cloning just that revision (and doing a shallow clone in the case of the interpreter) might work?
I'm fine with either the mirror or pulling it in the build script. I know @Alex Crichton has mentioned that a mirror would be nice since we expect to need a place to patch the spec interpreter temporarily as well as a place to merge in various proposals. Does that change the equation?
hmm, yeah, that's a good point. Let me think about this a bit and perhaps check in with a couple of folks, but yeah, that might be a solid reason for having our own full mirror
oh, but if we do the mirror thing, then I think it'd be valuable for that mirror to be for the entire repository, not just the interpreter. That'd enable us to do direct PRs against the spec instead of having to hand-craft patches all the time. So perhaps we do both—have a mirror and do the manual checkout in the build script?
ok, I checked in with some folks, and have firmed up my take on this :smile:
I do think having the mirror is good, but also that it should be a full-repo mirror for the above-mentioned reasons. I hope the manual checkout in the build script is viable?
The one thing we should make sure is to give the repo a name/description that make clear what they are and aren't. Lin pointed out that "spec" would make it sound like a BA spec, for example :smile: Perhaps something sort of annoying like "spec-mirror-for-fuzzing" would make sense?
How about wasm-spec-mirror
with a description like: "A mirror of https://github.com/WebAssembly/spec for fuzzing"
Actually, wait: we don't need the mirror just yet if we are going to manually checkout the repo... Why not delay creating the repository until it is needed? (E.g. we need to patch it or merge in proposals)
The full mirror is not as helpful as a "just the interpreter" mirror because it contains all the extra weight that @bjorn3 was worried about... another reason to delay making it
@Andrew Brown do you have any insights about the timeline of merging the SIMD spec into the main WebAssembly/spec repo? I wasn't able to find anything (open PR or issue for example)
that could be a reason to create our "merged for-fuzzing mirror" soon
I should have asked that in the latest SIMD meeting but instead I asked when Safari will implement Wasm SIMD (answer: no one knows)
so, not really sure
I'll just create the mirror and we can figure this out later; sort of want to pop this PR off the stack
Ok, https://github.com/bytecodealliance/wasm-spec-mirror is created and the latest commit on https://github.com/bytecodealliance/wasmtime/pull/3124 retrieves the repository, shallowly, in the build.rs
Last updated: Dec 23 2024 at 12:05 UTC