Bailey Hayes (Nov 03 2023 at 01:29): Bailey Hayes (Nov 03 2023 at 01:29):Is there overlap between wasi-nn and web-nn? Apologies if this has been asked before.
It would be very interesting if wasi-nn components could be backed by web-nn using a tool like JCO.
Andrew Brown (Nov 03 2023 at 01:45):There is one key difference between the APIs: WebNN is a model builder API in which one "builds up" a graph of nodes, the ML operations to perform. wasi-nn is a model loader API in which one loads an encoded version of a model in a single step. In the model builder paradigm, the user can introspect the graph operations--low-level control. In the model loader paradigm, the model contents are essentially opaque to the user--high-level ease of use.
Andrew Brown (Nov 03 2023 at 01:47):When wasi-nn was first proposed, we discussed this difference with the Web ML working group: at the time they were also considering enabling the model loader paradigm for WebNN. That didn't happen (though maybe at some point it still could?).
Bailey Hayes (Nov 03 2023 at 01:49):Ah OK that makes sense. While WebNN may adopt the loader paradigm, are there plans for creating a builder API for wasi?
Andrew Brown (Nov 03 2023 at 01:50):Many of the key WebNN contributors from Intel work in our larger group so we have discussed how one might implement wasi-nn in terms of WebNN. No one has vetted this idea with implementation (so take it for the idea that it is), but it could be possible to write code to read a model in, e.g., the ONNX format and rebuild that as a WebNN graph.
Andrew Brown (Nov 03 2023 at 01:51):Bailey Hayes said:
Ah OK that makes sense. While WebNN may adopt the loader paradigm, are there plans for creating a builder API for wasi?
I think that idea has been floated, but my sense is that the easiest thing here is to try to build an "encoded graph to WebNN graph" converter.
Andrew Brown (Nov 03 2023 at 01:56):Convincing WebNN to adopt the loader paradigm -- lots of work and a long shot. Defining a WebNN clone in wasi-nn and implementing it in several engines -- even more work! But creating a conversion from a well-defined model encoding (e.g., ONNX, OpenVINO) to the also well-defined WebNN builder API seems... workable. Ningxin (one of the leads from our group) showed us that in some phase for a certain ML backend, WebNN already does this kind of conversion. My thought is that this converter could be compiled to a combination of WebAssembly and JS, which seems crucial for using it in JCO.
Andrew Brown (Nov 03 2023 at 01:57):I suspect there may be certain parts of graph encodings that cannot yet be modeled in WebNN's builder API, but that is the part that would need to be vetted.
Bailey Hayes (Nov 03 2023 at 02:00):I don't have a use-case (yet) for needing this in a js runtime, but it's certainly something I'm curious about. Thank you for the info
Mingqiu Sun (Nov 09 2023 at 18:05):I think it is a good idea to add support of wasi-nn in JCO at some point, via an ML framework layer such as onnx.js or tensorflow.js.
zslayton (Feb 08 2025 at 19:20):Forgive me, I'm a newcomer to this space and I'm sure this is a perennial topic.
The 2020 blog post that introduced wasi-nn said that the primary motivation for using a graph loader API (vs builder) was due to the growth/instability of the WebNN API, which was rapidly adding new operations at the time. 5 years later, is that still the case or has it mostly stabilized?
I was just watching the talk the wasi-gfx project folks gave at Wasmcon. They've created a tool, webidl2wit, for mechanically translating WebIDL to WIT. They used it to automatically map the entire WebGPU API into wasi (!), minimizing the amount of manual adaptation required to keep the APIs in sync. They demo running GPU code in the browser and on a host running a customized wasmtime. I'd be curious whether a similar approach might be workable for bringing the graph loader API to wasi (via wasi-nn or a separate initiative.)
Andrew Brown (Feb 12 2025 at 16:44):I'd be curious whether a similar approach might be workable for bringing the graph loader API to wasi (via
wasi-nnor a separate initiative.)
I think you mean WebNN's _graph builder_ API, right? But, yes, I think it could make sense as an experiment. It would be cool to be able to use more web APIs from within a standalone WebAssembly runtime!
Andrew Brown (Feb 12 2025 at 16:49):@zslayton, here are challenges that I feel that experiment might run into: are we able to link to a WebNN implementation from whatever engine the experiment uses? IIRC, WebNN had a way to use JS code as operators--how do we map that in the experiment?
zslayton (Feb 12 2025 at 17:53):I think you mean WebNN's _graph builder_ API, right?
:man_facepalming: D'oh! Yes, that's what I meant, thanks.
are we able to link to a WebNN implementation from whatever engine the experiment uses?
I'm confident this is possible, but less confident about how to actually do it. :sweat_smile: The only cross-platform ML framework that implements WebNN is LiteRT (formerly TensorFlow-Lite). There are several Rust crates for TF-Lite (in various states of repair), so hopefully one of those would be sufficient to start on a proof-of-concept that forwarded the appropriate method calls. (I'm very open to others' thoughts here!)
IIRC, WebNN had a way to use JS code as operators--how do we map that in the experiment?
I took a look through the W3C spec but wasn't able to find something like this. Could you point me to an example?
Andrew Brown (Feb 12 2025 at 23:38):I may have mis-remembered about JS operators or perhaps that went away over time. And I also am not completely sure which implementation to start with; what you suggested sounds reasonable, though. Perhaps take a look at ONNX as well, since I know the ort crate works (it's used currently in wasmtime-wasi-nn).
Andrew Brown (Feb 12 2025 at 23:38):Let me contact some people I know here who work on WebNN and see if they have any suggestions.
Last updated: Feb 27 2025 at 23:03 UTC