Bailey Hayes (Nov 03 2023 at 01:29): Bailey Hayes (Nov 03 2023 at 01:29):Is there overlap between wasi-nn and web-nn? Apologies if this has been asked before.
It would be very interesting if wasi-nn components could be backed by web-nn using a tool like JCO.
Andrew Brown (Nov 03 2023 at 01:45):There is one key difference between the APIs: WebNN is a model builder API in which one "builds up" a graph of nodes, the ML operations to perform. wasi-nn is a model loader API in which one loads an encoded version of a model in a single step. In the model builder paradigm, the user can introspect the graph operations--low-level control. In the model loader paradigm, the model contents are essentially opaque to the user--high-level ease of use.
Andrew Brown (Nov 03 2023 at 01:47):When wasi-nn was first proposed, we discussed this difference with the Web ML working group: at the time they were also considering enabling the model loader paradigm for WebNN. That didn't happen (though maybe at some point it still could?).
Bailey Hayes (Nov 03 2023 at 01:49):Ah OK that makes sense. While WebNN may adopt the loader paradigm, are there plans for creating a builder API for wasi?
Andrew Brown (Nov 03 2023 at 01:50):Many of the key WebNN contributors from Intel work in our larger group so we have discussed how one might implement wasi-nn in terms of WebNN. No one has vetted this idea with implementation (so take it for the idea that it is), but it could be possible to write code to read a model in, e.g., the ONNX format and rebuild that as a WebNN graph.
Andrew Brown (Nov 03 2023 at 01:51):Bailey Hayes said:
Ah OK that makes sense. While WebNN may adopt the loader paradigm, are there plans for creating a builder API for wasi?
I think that idea has been floated, but my sense is that the easiest thing here is to try to build an "encoded graph to WebNN graph" converter.
Andrew Brown (Nov 03 2023 at 01:56):Convincing WebNN to adopt the loader paradigm -- lots of work and a long shot. Defining a WebNN clone in wasi-nn and implementing it in several engines -- even more work! But creating a conversion from a well-defined model encoding (e.g., ONNX, OpenVINO) to the also well-defined WebNN builder API seems... workable. Ningxin (one of the leads from our group) showed us that in some phase for a certain ML backend, WebNN already does this kind of conversion. My thought is that this converter could be compiled to a combination of WebAssembly and JS, which seems crucial for using it in JCO.
Andrew Brown (Nov 03 2023 at 01:57):I suspect there may be certain parts of graph encodings that cannot yet be modeled in WebNN's builder API, but that is the part that would need to be vetted.
Bailey Hayes (Nov 03 2023 at 02:00):I don't have a use-case (yet) for needing this in a js runtime, but it's certainly something I'm curious about. Thank you for the info
Mingqiu Sun (Nov 09 2023 at 18:05):I think it is a good idea to add support of wasi-nn in JCO at some point, via an ML framework layer such as onnx.js or tensorflow.js.
Last updated: Nov 22 2024 at 17:03 UTC