Stream: general

Topic: AI / wasi-nn building blocks suitable for evaluation / poc

view this post on Zulip Adrian Ibanez (Mar 20 2025 at 03:52):

HI. I'm looking for information about wasi-nn compliant building blocks and "working" configurations to implement some proof of concepts on ios wasmtime runtime ( wasmex )

view this post on Zulip Adrian Ibanez (Mar 20 2025 at 03:57):

For anyone interested I started working on an "experimental" integration of edge impulse models https://forum.edgeimpulse.com/t/wasm-standalone-inference-targeting-wasmtime-runtime/13574/4

Far from something useful but a starting point. To be honest I'm still trying to understand what the options are to interface with wasm modules. Haven't understood yet what the best options are to exchange data. Is sharing stdin / stdout between host and wasm still a thing? Or are there better options. eg. wasmex supports components but I'm still figuring out how the apis are supposed to be used.

Wasm standalone inference targeting wasmtime runtime
Have you considered providing a rust version of the model deployments? The ecosystem regarding wasm support seems quite good as far as I understand. And I’m specifically asking because I already use rust resp. rustler to interface with elixir and provide multiplatform support for things like BLE sensors. Having rust code would be quite interesting in all those applications. eg. as wasm and integrated in elixir or as native library on different platforms

view this post on Zulip Adrian Ibanez (Mar 20 2025 at 12:51):

Also is there something like a repository with information about prebuilt wasi components suitable for tests? Not particularly related to my question of wasi-nn

view this post on Zulip Adrian Ibanez (Mar 20 2025 at 15:46):

With wasi components I meant any wasi compliant projects that could be tested / shown as part of a poc. Not pariticularly wasi "components" api ... but I'm definitely interested in that api as wasmex exposes it in elixir

view this post on Zulip Ralph (Mar 20 2025 at 18:15):

I'm not really clear if this is something you'd wanna look at, but https://github.com/microsoft/aici

GitHub - microsoft/aici: AICI: Prompts as (Wasm) Programs
AICI: Prompts as (Wasm) Programs. Contribute to microsoft/aici development by creating an account on GitHub.

view this post on Zulip Ralph (Mar 20 2025 at 18:15):

it might NOT be what you're after, but it works nicely for p1 execution; there's a PR for components, but it's not yet merged.

view this post on Zulip Adrian Ibanez (Mar 20 2025 at 18:42):

@Ralph sounds mighty ... but complex. Is this something that has been running on wasmtime before? I'm basically looking for the MVP regarding wasi-nn integration or wasmtime compatible wasm packages in general. Something to validate that it can run successfully on the iOS / android platforms I'm targeting. There are quite a few moving pieces so I wouldn't mind keeping the complexity in check when it comes to wasmtime / wasm; specially as I'm quite a greenhorn when it comes to wasm

view this post on Zulip Adrian Ibanez (Mar 20 2025 at 18:44):

@Ralph p1 execution = wasi preview1?

view this post on Zulip Andrew Brown (Mar 20 2025 at 19:37):

I posted this example elsewhere but I'll drop it here as well for future readers who may be interested: https://github.com/bytecodealliance/wasmtime/blob/2b9da02f/crates/wasi-nn/examples/classification-example/README.md. That quite minimal example should give you an idea of the pieces required for running wasi-nn on Wasmtime. There's much more to learn after that, but I'm afraid the documentation is not great for someone completely new to the project (but I would love help improving that!).

wasmtime/crates/wasi-nn/examples/classification-example/README.md at 2b9da02f09d81dbcfb7cc615f67ffdf206394c19 · bytecodealliance/wasmtime
A lightweight WebAssembly runtime that is fast, secure, and standards-compliant - bytecodealliance/wasmtime

Last updated: Apr 09 2025 at 21:03 UTC