Stream: wasi-nn

Topic: Getting started

view this post on Zulip Richard Zak (Jun 28 2022 at 23:16):

I tried to follow along with the Bytecode Alliance's blog post on Wasi-nn, and wasn't able to get it to run. One of the Wasi-nn features wasn't available to Wasmtime, despite compiling Wasmtime with --features nn. Code: https://github.com/rjzak/wasi_benchmarks. Is there any advice? Have things changed since Dec 2020 https://bytecodealliance.org/articles/using-wasi-nn-in-wasmtime ?

GitHub - rjzak/wasi_benchmarks: Benchmarks for Wasi runtime environments
Benchmarks for Wasi runtime environments. Contribute to rjzak/wasi_benchmarks development by creating an account on GitHub.
Machine Learning in WebAssembly: Using wasi-nn in Wasmtime
The wasi-nn proposal allows WebAssembly programs to access host-provided machine learning(ML) functions. This post will explain the motivation for wasi-nn, a brief look at thespecification, and how to use it in Wasmtime to do machine lea...

view this post on Zulip Andrew Brown (Jun 28 2022 at 23:28):

Things have changed! I don't think you even have to use that feature flag any more since that feature should now be a default one. Maybe what you're missing is the --wasi-modules experimental-wasi-nn flag at runtime?

view this post on Zulip Andrew Brown (Jun 28 2022 at 23:29):

cc: @Brian Jones who also works on this

view this post on Zulip Andrew Brown (Jun 28 2022 at 23:34):

This might be helpful: https://github.com/bytecodealliance/wasmtime/blob/main/ci/run-wasi-nn-example.sh (and maybe we should document somewhere that that script is a good one to look at for this kind of thing)

wasmtime/run-wasi-nn-example.sh at main · bytecodealliance/wasmtime
A standalone runtime for WebAssembly. Contribute to bytecodealliance/wasmtime development by creating an account on GitHub.

view this post on Zulip Richard Zak (Jun 29 2022 at 13:55):

I used the --wasi-modules command, and now get a new error, despite running the OpenVINO setupvars.sh script:

Host error: BackendError(BackendAccess(library loading error

Caused by:
    system failed to load shared libraries (see https://github.com/intel/openvino-rs/blob/main/crates/openvino-finder): Unable to find the `inference_engine_c_api` library to load))
thread 'main' panicked at 'not implemented', crates/wasi-nn/src/witx.rs:17:45
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I had thought that wasmtime could link against OpenVINO libraries, but I suppose that isn't the case, as ldd doesn't show anything related to OpenVINO.

view this post on Zulip Andrew Brown (Jun 29 2022 at 14:16):

The libraries are loaded at runtime and it looks like the openvino-finder just cannot find them. That link in the error message explains how it looks for the OpenVINO libraries--could you take a look at that and see if your environment somehow does not include one of those methods for locating the libraries?

view this post on Zulip Brian Jones (Jun 29 2022 at 17:41):

Did you source /opt/intel/openvino/bin/setupvars.sh?

view this post on Zulip Richard Zak (Jun 29 2022 at 23:13):

I'll look more at openvino-finder. And I did run OpenVINO's setupvars.sh script.

view this post on Zulip Andrew Brown (Jun 29 2022 at 23:55):

Let us know how it goes and, if things don't go well, one of us can screen share with you and figure out what's not right in the environment

view this post on Zulip Richard Zak (Jun 30 2022 at 02:02):

What's the connection of openvino-finder to all of this? Seems that Wasmtime doesn't use it and maybe it should?

view this post on Zulip Andrew Brown (Jun 30 2022 at 16:04):

Wasmtime depends on the openvino crate which depends on openvino-sys and that depends on openvino-finder

Last updated: Nov 22 2024 at 17:03 UTC