Stream: wasmtime

Topic: calling `_initialize`


view this post on Zulip Roman Volosatovs (Oct 02 2024 at 14:29):

I've found out that TinyGo 0.33 wasip2 reactor components are currently incompatible with Wasmtime, due to the fact that _initialize is never called before, for example, wasi:http/incoming-handler.handle is invoked.

I've found https://github.com/bytecodealliance/wasmtime/issues/7592 related to the topic and here's the TinyGo PR for more context https://github.com/tinygo-org/tinygo/pull/4482

Do I understand correctly that the suggested solution is to build a custom adapter, which would be _initialize-aware and call it as part of e.g. wasi:http/incoming-handler.handle call?

Is there some way to make this part of instantiation of the component directly to be done ahead of time?
Or is there some other way this is supposed to be handled?

i have a preview1 reactor module, which exports _initialize. when adapting it with wasi_snapshot_preview1.reactor.wasm and loading the resulted component with wasmtime, i expected the _initialize e...
Using init() to initialize stdio or environment is prone to race conditions for reactor-style components, e.g. wasi:http/incoming-handler interface implementations due to the fact that init() is no...

view this post on Zulip Alex Crichton (Oct 02 2024 at 14:38):

The wit-component crate creating components was updated in https://github.com/bytecodealliance/wasm-tools/pull/1747 to recognize _initialize and handle that, so this may not have propagated to the TinyGo toolchain yet?

(e.g. if they're using a wasi-sdk release then it's not been tagged in wasi-sdk yet)

This commit implements recognition of the _initialize function from WASIp1 in the componentization process of wasm-tools component new. This additionally corresponds to the same function in the pro...

view this post on Zulip Roman Volosatovs (Oct 02 2024 at 14:39):

Oh, cool, thanks! I'll follow up with them

view this post on Zulip Roman Volosatovs (Oct 02 2024 at 14:53):

FWIW, they just shell-out to wasm-tools component new https://github.com/tinygo-org/tinygo/blob/ef4f46f1d1550beb62324d750c496b2b4a7f76d0/builder/build.go#L890-L896, but even after updating to wasm-tools 0.218 seems that TinyGo init() is still ignored :thinking:

Go compiler for small places. Microcontrollers, WebAssembly (WASM/WASI), and command-line tools. Based on LLVM. - tinygo-org/tinygo

view this post on Zulip Alex Crichton (Oct 02 2024 at 14:55):

Are you able to double-check if _initialize is called with the updated version of wasm-tools? If that doesn't fix the issue then it may be something else at play?

view this post on Zulip Richard Backhouse (Oct 02 2024 at 14:57):

This might be related to the fact that the tinygo linked wasi-libc is still SDK 20

view this post on Zulip Richard Backhouse (Oct 02 2024 at 14:57):

https://github.com/tinygo-org/tinygo/tree/release/lib

Go compiler for small places. Microcontrollers, WebAssembly (WASM/WASI), and command-line tools. Based on LLVM. - tinygo-org/tinygo

view this post on Zulip Roman Volosatovs (Oct 02 2024 at 14:59):

Good catch, I'll try updating wasi-libc and test!

view this post on Zulip Alex Crichton (Oct 02 2024 at 15:03):

Oh wasi-libc hasn't had a release yet with an updated version of wasm-component-ld so you'd have to build from source currently (or overwrite the wasm-component-ld binary)

view this post on Zulip Roman Volosatovs (Oct 02 2024 at 15:13):

"simply" updating the wasi-libc to latest tag or just main did not seem to help, but I'm not too familiar with tinygo build setup. Also not completely sure where wasm-component-ld binary fits in here.

view this post on Zulip Roman Volosatovs (Oct 02 2024 at 15:19):

I've filed an issue here https://github.com/tinygo-org/tinygo/issues/4488, btw

Currently, init() functions do not run in wasip2 reactor components run in Wasmtime 25. A recent change to wasm-tools introduced support for _initialize in component new step bytecodealliance/wasm-...

view this post on Zulip Alex Crichton (Oct 02 2024 at 15:21):

I'll also admit I don't know the particulars of TinyGo's linking process. Using wasi-sdk with rust/c/c++ components are created with the wasm-component-ld linker which is executed in lieu of wasm-ld. The wasm-component-ld linker links to wit-component, the Rust crate to create components, and the workhorse of wasm-tools component new. The code you linked though looks like it's executing wasm-tools directly, however, so the support may be entirely unrelated to the version of wasi-sdk in use.

The latest version of wasi-sdk, the wasi-sdk-24 tag, does not have a wasm-component-ld with support for _initialize. The latest wasm-component-ld, if you install via cargo install, does however. Support for _initialize first showed up in 217 for wasm-tools.

view this post on Zulip Roman Volosatovs (Oct 02 2024 at 15:22):

if that helps, it seems that they use wasm-ld https://github.com/tinygo-org/tinygo/blob/ef4f46f1d1550beb62324d750c496b2b4a7f76d0/targets/wasip2.json#L8

Go compiler for small places. Microcontrollers, WebAssembly (WASM/WASI), and command-line tools. Based on LLVM. - tinygo-org/tinygo

view this post on Zulip Roman Volosatovs (Oct 02 2024 at 15:22):

so perhaps the fix is to change the linker to wasm-component-ld?

view this post on Zulip Roman Volosatovs (Oct 02 2024 at 15:23):

unexpected: linker wasm-component-ld should be ld.lld or wasm-ld

view this post on Zulip Alex Crichton (Oct 02 2024 at 15:23):

I don't think so, it looks like TinyGo is doing some wasm transforms like asyncify which wouldn't work with a component. They're also executing wasm-tools component new which means that wasm-component-ld isn't needed.

view this post on Zulip Alex Crichton (Oct 02 2024 at 15:24):

If you're sure that your newer version of wasm-tools was used during linking for the component then at least the way I might be able to help would be to capture the wasm-tools component new command. For example what was the input module and what was the output?

view this post on Zulip Roman Volosatovs (Oct 02 2024 at 15:33):

in.wasm
out.wasm

here's the captured input and output Wasm used for wasm-tools component new

really appreciate the help!

view this post on Zulip Alex Crichton (Oct 02 2024 at 15:39):

It looks like in.wasm doesn't export _initialize?

view this post on Zulip Roman Volosatovs (Oct 02 2024 at 15:50):

well, yeah, it looks like they wouldn't https://github.com/tinygo-org/tinygo/blob/ef4f46f1d1550beb62324d750c496b2b4a7f76d0/src/runtime/runtime_wasm_wasip2.go#L14-L26 :D

looks like this could be the actual issue here (they simply never exported the function)

Go compiler for small places. Microcontrollers, WebAssembly (WASM/WASI), and command-line tools. Based on LLVM. - tinygo-org/tinygo

view this post on Zulip Roman Volosatovs (Oct 02 2024 at 16:01):

yep, that's it, it works now! Thanks @Alex Crichton

view this post on Zulip Roman Volosatovs (Oct 02 2024 at 16:02):

up-to-date wasm-tools and a small fix to tinygo to actually export the _initialize is what's necessary here

view this post on Zulip Roman Volosatovs (Oct 02 2024 at 16:07):

https://github.com/tinygo-org/tinygo/pull/4490

Closes #4488 Refs #4482 Turns out _initialize was not exported by wasip2 components and so init() was never called in reactors. This PR fixes that, by initializing the heap and calling all initiali...

view this post on Zulip Roman Volosatovs (Oct 02 2024 at 16:36):

according to https://github.com/WebAssembly/WASI/blob/main/legacy/application-abi.md#current-unstable-abi, it's either _initialize or _start that's called, but not both.
For wasip2 command components, is _initialize meant to be ignored or is it meant to be called the same as for reactors?

WebAssembly System Interface. Contribute to WebAssembly/WASI development by creating an account on GitHub.

view this post on Zulip Alex Crichton (Oct 02 2024 at 16:46):

There's less of a distinction of sorts in components about commands/reactors. Currently though the _start function is not handled at all by wasm-tools component new. It's not recognized or called at all. All wasm-tools component new does is ensure that if _initialize is exported it's called before all exports.

view this post on Zulip Alex Crichton (Oct 02 2024 at 16:46):

In that sense TinyGo will have to protect against this internally where if _start is accessible via wasi:cli/run then it'll have to be prepared for _initialize to already have run.


Last updated: Dec 23 2024 at 13:07 UTC