Stream: wasmtime

Topic: Wasmtime Android arm64

view this post on Zulip Gareth (Apr 17 2024 at 14:25):

Hi, I have been part of a project working on using wasm with Wasmtime to perform ML inference using wasi-nn with an onnx backend. This has been successful and we have an early mvp working. Thanks also for the support to my questions from the community here, it has been very helpful.

An extension of this project requires us to run this module via a Kotlin Android App, to be called via JNI, Java Native Interface in C. 

I've searched through threads here and can see that currently there doesn't seem to be a plan to have Wasmtime built for android arm64 architecture. I found a few threads here: https://github.com/bytecodealliance/wasmtime/pull/6480, https://github.com/bytecodealliance/wasmtime/issues/3921

Currently we are in the process of adapting our wasi-http and wasi-nn interfaces to  be able to use both Wasmtime and WasmEdge (which can be compiled for android). However we have found that the wasi-nn implementations between Wasmtime and WasmEdge are subtly different and require a sizeable amount of code to adapt between them. Going forward, managing compatibility with multiple wasm runtimes and wasi interfaces adds a lot of complexity and ideally we would like to stick to just using Wasmtime.

Does anybody know if maintaining a Wasmtime build for android is in the roadmap and if so when it might be expected? Also are there any recommendations for managing wasi interface differences between runtimes if we need to be able to use multiple runtimes, any insights are much appreciated :smile:

Cheers,
Gareth

Android release binaries by martindevans · Pull Request #6480 · bytecodealliance/wasmtime
As discussed here. This adds Android targets to the build matrix (Android aarch64 and x86_64) and adds Android to the docs in "tier3". This will make wasmtime more convenient to use on Android devi...
Android NDK link arm64-v8a/libwasmtime.a failed? · Issue #3921 · bytecodealliance/wasmtime
My project link arm64-v8a/libwasmtime.a error rustix.dcbac7fd-cgu.5.rcgu.o): In function rustix::imp::io::syscalls::pwritev::hdad5057263d83a9f': rustix.dcbac7fd-cgu.5:(.text._ZN6rustix3imp2io8sysca...

view this post on Zulip Alex Crichton (Apr 17 2024 at 15:08):

Wasmtime doesn't currently have a roadmap per-se but adding Android builds and keeping them building is pretty reasonable. Current maintainers don't have a lot of Android experience and there's nothing on CI right now -- not because we don't want but rather it just hasn't happened yet. If you're interested I think #6480 is a great starting point and we can try to help resolve build errors as they come up.

view this post on Zulip Andrew Brown (Apr 18 2024 at 10:02):

@Gareth, just a heads up from the wasi-nn spec side: the wasi-nn spec now uses preview 2 concepts like resources that rely on the component model. In the near term, I will be updating Wasmtime to implement those new spec changes; I'm not sure where WasmEdge is at in terms of component model support, but I suspect further behind than Wasmtime. If you're interested in using wasi-nn and have any questions/suggestions, feel free to drop in to the ML working group where we regularly meet to discuss this kind of stuff (meeting).

meetings/machine-learning at main · bytecodealliance/meetings
Contribute to bytecodealliance/meetings development by creating an account on GitHub.

view this post on Zulip Gareth (Apr 18 2024 at 14:09):

Alex Crichton said:

Wasmtime doesn't currently have a roadmap per-se but adding Android builds and keeping them building is pretty reasonable. Current maintainers don't have a lot of Android experience and there's nothing on CI right now -- not because we don't want but rather it just hasn't happened yet. If you're interested I think #6480 is a great starting point and we can try to help resolve build errors as they come up.

Thanks Alex, we will need to look at available resources, its something we are interested in and would be great to contribute, however we also are not Android developers. I'll feedback and pose the idea though.

view this post on Zulip Gareth (Apr 18 2024 at 14:13):

Andrew Brown said:

Gareth, just a heads up from the wasi-nn spec side: the wasi-nn spec now uses preview 2 concepts like resources that rely on the component model. In the near term, I will be updating Wasmtime to implement those new spec changes; I'm not sure where WasmEdge is at in terms of component model support, but I suspect further behind than Wasmtime. If you're interested in using wasi-nn and have any questions/suggestions, feel free to drop in to the ML working group where we regularly meet to discuss this kind of stuff (meeting).

Thanks Andrew. Yes at the moment we are using the component model with wasi-nn in Wasmtime, we based it off the example code from David Justice. With WasmEdge we are not currently using the component model, although I think they are working on it. I will try and check out the wasi-nn meeting sometime :) We are also interested in model training, do you think that would be a long way off?

view this post on Zulip Andrew Brown (Apr 18 2024 at 18:34):

So far we've avoided model training because, well, it's a lot to bite off. But we've discussed it as a possibility before; extending the spec to cover that probably depends more on the motivation of those willing to push it forward than on any technical obstacle.

Last updated: Nov 22 2024 at 16:03 UTC