Stream: git-wasmtime

Topic: wasmtime / issue #9379 WASI-NN OpenVINO backend error: "s...

view this post on Zulip Wasmtime GitHub notifications bot (Oct 06 2024 at 00:47):

danbugs added the bug label to Issue #9379.

view this post on Zulip Wasmtime GitHub notifications bot (Oct 06 2024 at 00:47):

danbugs opened issue #9379:

Test Case

Zipped Wasm file: wasi_nn_example.zip

Steps to Reproduce

(1) cargo component new wasi_nn_example

(2) add dependency to wit-bindgen in Cargo.toml. Here's the full Cargo.toml:

[package]
name = "wasi_nn_example"
version = "0.1.0"
edition = "2021"

[dependencies]
wit-bindgen = { version = "0.33.0",  default-features = false, features = ["macros"] }
image2tensor = "0.3.1"

[profile.release]
codegen-units = 1
opt-level = "s"
debug = false
strip = true
lto = true

[package.metadata.component]
package = "component:wasi-nn-example"

[package.metadata.component.dependencies]

(3) populate main.rs w/:

use std::fs;
use crate::wasi::nn::graph;
use crate::wasi::nn::graph::{ExecutionTarget, GraphEncoding};

wit_bindgen::generate!({
    path: "wit",
    world: "ml",
});

fn main() {
    let xml = fs::read_to_string("fixture/model.xml").unwrap();
    println!("Read graph XML, first 50 characters: {}", &xml[..50]);

    let weights = fs::read("fixture/model.bin").unwrap();
    println!("Read graph weights, size in bytes: {}", weights.len());

    let graph = graph::load(
        &[xml.into_bytes(), weights],
        GraphEncoding::Openvino,
        ExecutionTarget::Cpu,
    ).unwrap();
    println!("Loaded graph into wasi-nn with ID: {:#?}", graph);
}

(4) cargo component build --release

(5) wasmtime -Snn --dir .\fixture\::fixture ..\..\target\wasm32-wasip1\debug\wasi_nn_example.wasm

Expected Results

I expected the graph::load function to return a proper Graph and not panic. It's failing in the copy_from_slice here:

https://github.com/bytecodealliance/wasmtime/blob/8096068c80bc2b1a934d608acb775016ee5a69cf/crates/wasi-nn/src/backend/openvino.rs#L41

Actual Results

wasmtime -Snn --dir .\fixture\::fixture ..\..\target\wasm32-wasip1\debug\wasi_nn_example.wasm
Read graph XML, first 50 characters: <?xml version="1.0" ?>
<net name="mobilenet_v2_1.0
Read graph weights, size in bytes: 13956476
thread 'main' panicked at C:\Users\danil\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wasmtime-wasi-nn-25.0.1\src\backend\openvino.rs:41:16:
source slice length (13956476) does not match destination slice length (5233680)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Versions and Environment

Wasmtime version or commit: v25.0.1

Operating system: Microsoft Windows 11 Pro

Architecture: x86_64

Extra Info

cargo-component version used: v0.16.0

cc: @abrown

view this post on Zulip Wasmtime GitHub notifications bot (Oct 06 2024 at 01:13):

danbugs edited issue #9379:

Test Case

Zipped Wasm file: wasi_nn_example.zip

Steps to Reproduce

(1) cargo component new wasi_nn_example

(2) add dependency to wit-bindgen in Cargo.toml. Here's the full Cargo.toml:

[package]
name = "wasi_nn_example"
version = "0.1.0"
edition = "2021"

[dependencies]
wit-bindgen = { version = "0.33.0",  default-features = false, features = ["macros"] }
image2tensor = "0.3.1"

[profile.release]
codegen-units = 1
opt-level = "s"
debug = false
strip = true
lto = true

[package.metadata.component]
package = "component:wasi-nn-example"

[package.metadata.component.dependencies]

(3) populate main.rs w/:

use std::fs;
use crate::wasi::nn::graph;
use crate::wasi::nn::graph::{ExecutionTarget, GraphEncoding};

wit_bindgen::generate!({
    path: "wit",
    world: "ml",
});

fn main() {
    let xml = fs::read_to_string("fixture/model.xml").unwrap();
    println!("Read graph XML, first 50 characters: {}", &xml[..50]);

    let weights = fs::read("fixture/model.bin").unwrap();
    println!("Read graph weights, size in bytes: {}", weights.len());

    let graph = graph::load(
        &[xml.into_bytes(), weights],
        GraphEncoding::Openvino,
        ExecutionTarget::Cpu,
    ).unwrap();
    println!("Loaded graph into wasi-nn with ID: {:#?}", graph);
}

(4) cargo component build --release

(5) wasmtime -Snn --dir .\fixture\::fixture ..\..\target\wasm32-wasip1\debug\wasi_nn_example.wasm

Expected Results

I expected the graph::load function to return a proper Graph and not panic. It's failing in the copy_from_slice here:

https://github.com/bytecodealliance/wasmtime/blob/8096068c80bc2b1a934d608acb775016ee5a69cf/crates/wasi-nn/src/backend/openvino.rs#L41

Actual Results

wasmtime -Snn --dir .\fixture\::fixture ..\..\target\wasm32-wasip1\debug\wasi_nn_example.wasm
Read graph XML, first 50 characters: <?xml version="1.0" ?>
<net name="mobilenet_v2_1.0
Read graph weights, size in bytes: 13956476
thread 'main' panicked at C:\Users\danil\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wasmtime-wasi-nn-25.0.1\src\backend\openvino.rs:41:16:
source slice length (13956476) does not match destination slice length (5233680)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Versions and Environment

Wasmtime version or commit: v25.0.1

Operating system: Microsoft Windows 11 Pro

Architecture: x86_64

Extra Info

cargo-component version used: v0.16.0

I got the model.xml and model.bin from here: https://download.01.org/openvinotoolkit/fixtures/mobilenet/

cc: @abrown

view this post on Zulip Wasmtime GitHub notifications bot (Oct 07 2024 at 17:36):

abrown commented on issue #9379:

@danbugs, take a look at https://github.com/intel/openvino-rs/issues/143: the issue is a breaking change in how OpenVINO v2024.2 laid out its element type enum so if one is using using pre-2024.2 bindings with a post-2024.2 library this issue pops up. The fix is in https://github.com/intel/openvino-rs/pull/144. Can you confirm which version of OpenVINO you installed?

view this post on Zulip Wasmtime GitHub notifications bot (Oct 07 2024 at 17:37):

abrown edited a comment on issue #9379:

@danbugs, take a look at https://github.com/intel/openvino-rs/issues/143: the issue is a breaking change in how OpenVINO v2024.2 laid out its element type enum so if one is using pre-2024.2 bindings with a post-2024.2 library this issue pops up. The fix is in https://github.com/intel/openvino-rs/pull/144. Can you confirm which version of OpenVINO you installed?

view this post on Zulip Wasmtime GitHub notifications bot (Oct 07 2024 at 18:36):

danbugs commented on issue #9379:

@danbugs, take a look at intel/openvino-rs#143: the issue is a breaking change in how OpenVINO v2024.2 laid out its element type enum so if one is using pre-2024.2 bindings with a post-2024.2 library this issue pops up. The fix is in intel/openvino-rs#144. Can you confirm which version of OpenVINO you installed?

@abrown ~ understood. I do haveOpenVINO 2024.4 installed!

view this post on Zulip Wasmtime GitHub notifications bot (Oct 16 2024 at 04:07):

abrown closed issue #9379:

Test Case

Zipped Wasm file: wasi_nn_example.zip

Steps to Reproduce

(1) cargo component new wasi_nn_example

(2) add dependency to wit-bindgen in Cargo.toml. Here's the full Cargo.toml:

[package]
name = "wasi_nn_example"
version = "0.1.0"
edition = "2021"

[dependencies]
wit-bindgen = { version = "0.33.0",  default-features = false, features = ["macros"] }
image2tensor = "0.3.1"

[profile.release]
codegen-units = 1
opt-level = "s"
debug = false
strip = true
lto = true

[package.metadata.component]
package = "component:wasi-nn-example"

[package.metadata.component.dependencies]

(3) populate main.rs w/:

use std::fs;
use crate::wasi::nn::graph;
use crate::wasi::nn::graph::{ExecutionTarget, GraphEncoding};

wit_bindgen::generate!({
    path: "wit",
    world: "ml",
});

fn main() {
    let xml = fs::read_to_string("fixture/model.xml").unwrap();
    println!("Read graph XML, first 50 characters: {}", &xml[..50]);

    let weights = fs::read("fixture/model.bin").unwrap();
    println!("Read graph weights, size in bytes: {}", weights.len());

    let graph = graph::load(
        &[xml.into_bytes(), weights],
        GraphEncoding::Openvino,
        ExecutionTarget::Cpu,
    ).unwrap();
    println!("Loaded graph into wasi-nn with ID: {:#?}", graph);
}

(4) cargo component build --release

(5) wasmtime -Snn --dir .\fixture\::fixture ..\..\target\wasm32-wasip1\debug\wasi_nn_example.wasm

Expected Results

I expected the graph::load function to return a proper Graph and not panic. It's failing in the copy_from_slice here:

https://github.com/bytecodealliance/wasmtime/blob/8096068c80bc2b1a934d608acb775016ee5a69cf/crates/wasi-nn/src/backend/openvino.rs#L41

Actual Results

wasmtime -Snn --dir .\fixture\::fixture ..\..\target\wasm32-wasip1\debug\wasi_nn_example.wasm
Read graph XML, first 50 characters: <?xml version="1.0" ?>
<net name="mobilenet_v2_1.0
Read graph weights, size in bytes: 13956476
thread 'main' panicked at C:\Users\danil\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wasmtime-wasi-nn-25.0.1\src\backend\openvino.rs:41:16:
source slice length (13956476) does not match destination slice length (5233680)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Versions and Environment

Wasmtime version or commit: v25.0.1

Operating system: Microsoft Windows 11 Pro

Architecture: x86_64

Extra Info

cargo-component version used: v0.16.0

I got the model.xml and model.bin from here: https://download.01.org/openvinotoolkit/fixtures/mobilenet/

cc: @abrown

Last updated: Oct 23 2024 at 20:03 UTC