Stream: git-wasmtime

Topic: wasmtime / issue #5802 Add `Engine::serialization_compati...


view this post on Zulip Wasmtime GitHub notifications bot (Feb 16 2023 at 16:00):

lann opened issue #5802:

Feature

Add a new Engine::serialization_compatibility_hash method that returns a hash of the "compiler info" appended to precompiled Wasm binaries.

Benefit

This would allow precompiled Wasm from multiple configurations/versions of Wasmtime to be safely and efficiently stored together. This may still be a little pessimistic compared with Wasmtime's actual compatibility checks but would be better than the conservative approach of only reusing precompiled binaries on a single host.

Implementation

Extract the following section from append_compiler_info and reuse it to compute a sha-256 digest of the "compiler info": https://github.com/bytecodealliance/wasmtime/blob/c3c16eb207eccd895f5fbbc4b771bd74ea36d071/crates/wasmtime/src/engine/serialization.rs#L50-L64

Alternatives

view this post on Zulip Wasmtime GitHub notifications bot (Feb 16 2023 at 16:02):

lann edited issue #5802:

Feature

Add a new Engine::serialization_compatibility_hash method that returns a hash of the "compiler info" appended to precompiled Wasm binaries.

Benefit

This would allow precompiled Wasm from multiple configurations/versions of Wasmtime to be safely and efficiently stored together by, for example, keying its storage on (Hash(wasm), serialization_compat_hash). This may still be a little pessimistic compared with Wasmtime's actual compatibility checks but would be better than the conservative approach of only reusing precompiled binaries on a single host.

Implementation

Extract the following section from append_compiler_info and reuse it to compute a sha-256 digest of the "compiler info": https://github.com/bytecodealliance/wasmtime/blob/c3c16eb207eccd895f5fbbc4b771bd74ea36d071/crates/wasmtime/src/engine/serialization.rs#L50-L64

Alternatives

view this post on Zulip Wasmtime GitHub notifications bot (Feb 16 2023 at 16:02):

lann edited issue #5802:

Feature

Add a new Engine::serialization_compatibility_hash method that returns a hash of the "compiler info" appended to precompiled Wasm binaries.

Benefit

This would allow precompiled Wasm from multiple configurations/versions of Wasmtime to be safely and efficiently stored together by, for example, keying its storage on (Hash(wasm), serialization_compat_hash). This may still be a little pessimistic compared with Wasmtime's actual compatibility checks but would be better than the conservative approach of only reusing precompiled binaries on a single host.

Implementation

Extract the following section from append_compiler_info and reuse it to compute a sha-256 digest of the "compiler info": https://github.com/bytecodealliance/wasmtime/blob/c3c16eb207eccd895f5fbbc4b771bd74ea36d071/crates/wasmtime/src/engine/serialization.rs#L50-L64

Alternatives

view this post on Zulip Wasmtime GitHub notifications bot (Feb 17 2023 at 15:15):

alexcrichton commented on issue #5802:

One alternative idea for this would be to implement Hash and Eq for the Config type perhaps. That's more-or-less what the blob here is but would be a bit less specific to the current serialization strategy.

view this post on Zulip Wasmtime GitHub notifications bot (Feb 17 2023 at 15:51):

lann commented on issue #5802:

implement Hash and Eq for the Config

That seems like it could work, though it would be a little more pessimistic compatibility-wise for settings that don't affect compilation or if a rustc update changes Hash (rare I assume).

One notable oddity is that we'd need to special-case Config::module_version's WasmtimeVersion to mix in CARGO_PKG_VERSION somewhere.

Would this be documented behavior of impl Hash for Config? Is it reasonable to assume that Config will continue to cover everything used in compatibility checks?

view this post on Zulip Wasmtime GitHub notifications bot (Feb 17 2023 at 17:24):

alexcrichton commented on issue #5802:

That's true, yeah. I'm mostly thinking that I'd prefee to not bake in sha256 or something like that but to expose this through Hash and Eq if possible (in case anyone wants strict-equality as well instead of just hash-equality). One possibility would be something like Engine::cache_key(&self) -> impl Hash + Eq instead of returning the sha256 hash itself as well. We could then change the serialization.rs to use this directly to ensure nothing regresses as well perhaps.

view this post on Zulip Wasmtime GitHub notifications bot (Feb 17 2023 at 17:29):

lann commented on issue #5802:

What about Engine::cache_key(&self) -> impl AsRef<[u8]>?

view this post on Zulip Wasmtime GitHub notifications bot (Feb 17 2023 at 17:33):

lann edited a comment on issue #5802:

The semantics of Hash just don't seem quite right for this use. You can get a u64 out of it for indexing but you wouldn't really use this key in a HashMap.

What about Engine::cache_key(&self) -> impl AsRef<[u8]>?

view this post on Zulip Wasmtime GitHub notifications bot (Feb 17 2023 at 17:33):

lann edited a comment on issue #5802:

The semantics of Hash just don't seem quite right for this. You can get a u64 out of it for indexing but you wouldn't really use this key in a HashMap.

What about Engine::cache_key(&self) -> impl AsRef<[u8]>?

view this post on Zulip Wasmtime GitHub notifications bot (Feb 17 2023 at 17:34):

alexcrichton commented on issue #5802:

I suppose that's probably the only reasonable thing to implement in the short term. I'm hesitant to have something that feels so specific here, but if it solves a use case for y'all and it's already used elsewhere it shouldn't really increase maintenance burden at all.

view this post on Zulip Wasmtime GitHub notifications bot (Feb 17 2023 at 17:39):

lann commented on issue #5802:

Yeah, I actually think https://github.com/bytecodealliance/wasmtime/issues/3900 would be a more generally-useful approach but I don't think I have the context/time to tackle it myself. I'm not actually opposed to the "build a dummy module and read out the compiler info" hack as a short term workaround but I wouldn't want to rely on it for very long.

view this post on Zulip Wasmtime GitHub notifications bot (Feb 17 2023 at 17:39):

lann edited a comment on issue #5802:

Yeah, I actually think https://github.com/bytecodealliance/wasmtime/issues/3900 (if exposed publicly) would be a more generally-useful approach but I don't think I have the context/time to tackle it myself. I'm not actually opposed to the "build a dummy module and read out the compiler info" hack as a short term workaround but I wouldn't want to rely on it for very long.

view this post on Zulip Wasmtime GitHub notifications bot (Feb 17 2023 at 20:33):

alexcrichton commented on issue #5802:

Nah it's much easier to "just" add a method to Engine to expose that section rather than harvesting it yourself, so I think that's ok to add.

view this post on Zulip Wasmtime GitHub notifications bot (Apr 18 2023 at 22:08):

jameysharp closed issue #5802:

Feature

Add a new Engine::serialization_compatibility_hash method that returns a hash of the "compiler info" appended to precompiled Wasm binaries.

Benefit

This would allow precompiled Wasm from multiple configurations/versions of Wasmtime to be safely and efficiently stored together by, for example, keying its storage on (Hash(wasm), serialization_compat_hash). This may still be a little pessimistic compared with Wasmtime's actual compatibility checks but would be better than the conservative approach of only reusing precompiled binaries on a single host.

Implementation

Extract the following section from append_compiler_info and reuse it to compute a sha-256 digest of the "compiler info": https://github.com/bytecodealliance/wasmtime/blob/c3c16eb207eccd895f5fbbc4b771bd74ea36d071/crates/wasmtime/src/engine/serialization.rs#L50-L64

Alternatives


Last updated: Dec 23 2024 at 13:07 UTC