Shannon Duncan (shadowcodex) (May 04 2023 at 16:02): Shannon Duncan (shadowcodex) (May 04 2023 at 16:02):Hey all! I'm looking at the wasmtime-py repo and wondering a few things related to contributing. I've been looking to do some open source contributions for a while and the wasm community seems like a great place to do so. I'm a python developer mostly with no Rust experience so looks like I could help with the wasmtime-py repo if there are opportunities there.
A few questions:
Shannon Duncan (shadowcodex) (May 04 2023 at 16:03):CC @Alex Crichton who seems to be the most active contributor?
Alex Crichton (May 04 2023 at 16:13):Hello! The wasmtime-py repo could definitely do with some love, so help is appreciated! Currently there aren't a ton of docs and I think most of the design is just sitting in my head, so in terms of how best to start that may be it, helping document what you're learning as you come up to speed. I should also clarify that I myself am no Pythonista and I would consider myself as barely knowing Python. In that sense if you're a Python expert I'd be more than happy to have you weigh in on what's going on around wasmtime-py.
I see you've also found the guest side of things which is good to be aware of as well. The intersection between the two projects doesn't currently exist in any concrete form, but in the hypothetical the intersection is the component model and how types are represented in Python.
Anyway, for your questions:
Are there any prior discussions of the repo? Looking for information on what files are autogen what aren't, how it's setup etc.
Not currently. Maybe some bits and pieces in the issues, but nothing "formal". I'd recommend following what CI does from .github/workflows/*.yml. The ci/cbindgen.py script generates typed cffi bindings from the Wasmtime c-api header file, and otherwise the ci/build-rust.py will build component-related bindgen files.
What are the high priority things needed / to be fixed for the repo?
I don't have anything off the top of my head, but some possible ideas are:
In general I find this sort of use is best driven by your own personal needs, but you can also peruse the issue tracker as well! (I'm not sure if there's a ton there though)
How intertwined is this with the CPython work and the #wasm > Python guest runtime and bindings convo?
Not a ton, but definitely some with respect to how component model types are represented. Ideally bindgen sources would be shared as well, but I don't think things are advanced to quite that point yet.
Joel Dice (May 04 2023 at 16:26):Regarding the intersection of host and guest binding generation for Python: as I briefly discussed with Alex, I'm planning to split the wasmtime-py type binding generator into its own crate so it can be reused by componentize-py on the guest side, probably tomorrow.
Shannon Duncan (shadowcodex) (May 04 2023 at 17:30):Thanks Alex & Joel!
I'll take some time this weekend to dig in and get things running locally / fiddle with the ci stuff.
Shannon Duncan (shadowcodex) (May 04 2023 at 17:34):@Joel Dice how is componentize related to the podcast Brett did on RealPython? Is this the main effort to do WASM compiling from Python?
Joel Dice (May 04 2023 at 17:41):Brett's been attending our meetings, but I haven't had a chance to listen to that podcast. CPython already has upstream WASI support, and there's also Pyodide for targeting the browser and Node.js. So I wouldn't necessarily call this the "main" Python->Wasm effort. This one is primarily focused on targeting the Wasm Component Model with Python.
Shannon Duncan (shadowcodex) (May 06 2023 at 23:05):Took a look around and at the PR open for speed improvements. I’m going to look into that more this week and the comments.
Def outside my comfort zone but no better way to learn than to do.
Shannon Duncan (shadowcodex) (May 16 2023 at 02:24):Is it possible to use a .wasm file with wasmtime-py instead of a .wat file?
Shannon Duncan (shadowcodex) (May 16 2023 at 02:39):It looks like currently no, but we could extend this here to skip this step if the bytes passed in are already a wasmbinary format. https://github.com/bytecodealliance/wasmtime-py/blob/main/wasmtime/_module.py#L28
However we would probably have to put a flag in or something to allow it to skip compiling and it would pass only if the instance passed in is bytes or bytearray.
Alex Crichton (May 16 2023 at 03:05):Currently it should support both binary and text format, where the text format is automatically converted to binary
Shannon Duncan (shadowcodex) (May 16 2023 at 03:13):Looks like bytes is automatically taken to wat2wasm. If wat2wasm recielves a wasm bytes does it do nothing?
Looks like I can pass in a bytearray and it will skip the wat2wasm.
Alex Crichton (May 16 2023 at 05:27):Yes wat2wasm is a noop if binary wasm is passed to it
Alex Crichton (May 16 2023 at 05:27):Note that it's also checking the first byte, which if it's 0 indicates binary wasm
Shannon Duncan (shadowcodex) (May 16 2023 at 13:11):Ok a couple other questions as I'm entering and not finding helpful documentation on. Is there a way to generate a *.wit file from a*.wasm file? or a *.wat file?
I'm trying to mount arbitrary wasm/wat in a python process but want to create a definition of that mounted module such that someone could type check it and see what the expected params to pass in are and expected output is.
Seems to only be possible if they pass me a *.wit file as well.
Shannon Duncan (shadowcodex) (May 16 2023 at 13:17):Looks like someone was asking same question over in wasi land: https://bytecodealliance.zulipchat.com/#narrow/stream/219900-wasi/topic/Create.20WIT.20from.20wasm.20binary/near/358635878
Lann Martin (May 16 2023 at 13:21):The wasm-tools component wit command parses a component .wasm/.wat and prints its interface as WIT
Lann Martin (May 16 2023 at 13:22):I imagine doing something similar programmatically in Python would require writing bindings for various crates in wasm-tools
Joel Dice (May 16 2023 at 13:29):wasmtime-py generates bindings based on WIT data extracted from the binary form of the component type in a .wasm file: https://github.com/bytecodealliance/wasmtime-py/blob/95998108fd9c72cb8ffc2779b018576f9b75a332/rust/bindgen/src/bindgen.rs#L80
Shannon Duncan (shadowcodex) (May 16 2023 at 13:42):Joel Dice said:
wasmtime-pygenerates bindings based on WIT data extracted from the binary form of the component type in a .wasm file: https://github.com/bytecodealliance/wasmtime-py/blob/95998108fd9c72cb8ffc2779b018576f9b75a332/rust/bindgen/src/bindgen.rs#L80
Oh I missed this. This will be great for ahead of time bindgen.
We will need a JIT something that allows us to have a .wasm or .wat file in a variable and extract the imports / exports with typing information so we can at runtime evaluate a wasm file before execution with some params provided.
It looks like we'll need to add bindings for the python to be able to hook into the wasm-tools component wit similar to what bindgen.rs is doing.
Shannon Duncan (shadowcodex) (May 16 2023 at 13:42):Lann Martin said:
I imagine doing something similar programmatically in Python would require writing bindings for various crates in
wasm-tools
Thanks Lann! I imagine you are right. We will need bindings for this.
Shannon Duncan (shadowcodex) (May 20 2023 at 20:23):Was trying to get it going locally but running to issues on ci/build-rust.py :
error[E0425]: cannot find function, tuple struct or tuple variant `Err` in this scope
--> /Users/<user>/.cargo/registry/src/github.com-1ecc6299db9ec823/wasi-0.11.0+wasi-snapshot-preview1/src/lib_generated.rs:2147:14
|
2147 | _ => Err(Errno(ret as u16)),
| ^^^ not found in this scope
error[E0425]: cannot find function, tuple struct or tuple variant `Ok` in this scope
--> /Users/<user>/.cargo/registry/src/github.com-1ecc6299db9ec823/wasi-0.11.0+wasi-snapshot-preview1/src/lib_generated.rs:2160:14
|
2160 | 0 => Ok(()),
| ^^ not found in this scope
error[E0425]: cannot find function, tuple struct or tuple variant `Err` in this scope
--> /Users/<user>/.cargo/registry/src/github.com-1ecc6299db9ec823/wasi-0.11.0+wasi-snapshot-preview1/src/lib_generated.rs:2161:14
|
2161 | _ => Err(Errno(ret as u16)),
| ^^^ not found in this scope
Lots of these cannot find function in this scope.
Alex Crichton (May 22 2023 at 14:33):I think this probably means you need the wasm32-wasi rust target installed
Alex Crichton (May 22 2023 at 14:33):I think that should be amongst the first error messages
Shannon Duncan (shadowcodex) (May 22 2023 at 19:01):Is that different than rustup target add wasm32-wasi ? Cause that was added. I'll have to see if I can capture those errors into a file to get to beginning. So many of them it's beyond history of terminal haha.
Alex Crichton (May 22 2023 at 19:03):oh yeah that should have done it, in that case I'd take a look at the first errors
Joel Dice (May 22 2023 at 19:37):If you're doing anything involving the WASI preview 1 -> 2 adapter, you'll also need the wasm32-unknown-unknown target
hax10 (Jun 17 2023 at 13:51):Hi! I'm trying to get WASM components working with wasmtime-py.
I'm following the tutorial below, but it seems to be broken:
https://github.com/bytecodealliance/wasmtime-py
Specifically, when I try to embed the WIT file in my WAT code --
wasm-tools component new demo.wat --wit demo.wit -o demo.wasm
-- I'm told that wasm-tools doesn't accept a --wit argument. I'm running the latest release of wasm-tools (1.0.35), so I guessed that this argument has been removed. I then tried:
wasm-tools component embed demo.wit --world demo demo.wat -o demo.wasm
but the provided *.wit file doesn't work. When I tweak it to more recent syntax --
package local:demo
world demo {
import python: interface {
print: func(s: string)
}
export run: func()
}
-- I can embed the *.wit file successfully and produce a demo.wasm file.
Now, when I try the next command in the tutorial, which is:
python -m wasmtime.bindgen demo.wasm --out-dir demo
I get the following error:
``
RuntimeError: failed to parse the input component
Caused by:
attempted to parse a wasm module with a component parser
Can anyone give me any pointers as to what I'm doing wrong?
Last updated: Nov 22 2024 at 16:03 UTC