Stream: git-wasmtime

Topic: wasmtime / Issue #2430 out of bounds memory access and/or...

view this post on Zulip Wasmtime GitHub notifications bot (Nov 20 2020 at 04:12):

ZW007 edited Issue #2430:

I am running a standalone wasm file by using wasmtime --dir=. mainWasi.wasm
The wasm needs to read some local files, the size of local files in total is 10MB

Error: failed to run main module mainWasi.wasm

Caused by:
0: failed to invoke command default
1: wasm trap: out of bounds memory access
wasm backtrace:
0: 0x29cb - <unknown>!Predict
1: 0x346d - <unknown>!testing
2: 0x38ef - <unknown>!foo
3: 0x39eb - <unknown>!__original_main
4: 0x2c1 - <unknown>!_start

I wonder if I can allocate more memory for wasmtime? In lucet, i can use --min-reserved-size, but haven't found similar in wasmtime, but even when i use lucetc-wasi --min-reserved-size 6442450944 -o mainLucet.so mainWasi.wasm to compile in lucet, I still get the error in lucet run: thread 'main' panicked at 'instance can be created: ModuleError(IncorrectModule("heap spec sizes would overflow: HeapSpec { reserved_size: 6442450944, guard_size: 4194304, initial_size: 131072, max_size: None }"))', lucet-wasi/src/main.rs:239:24

What I am running is a standalone lenet inference task. I dont think it can be called a large app. Maybe I should try to read just one image from the file system, rather than the whole t10k-images-idx3-ubyte which is 7.8MB.

Thank you!

view this post on Zulip Wasmtime GitHub notifications bot (Nov 20 2020 at 20:42):

alexcrichton commented on Issue #2430:

Sorry it sounds like there's a lot of moving parts here and there's perhaps more than a few scenarios in play. Would you be able to share a set of instructions along with code of how to reproduce the issue you're currently having?

view this post on Zulip Wasmtime GitHub notifications bot (Nov 20 2020 at 21:15):

ZW007 commented on Issue #2430:

Sorry it sounds like there's a lot of moving parts here and there's perhaps more than a few scenarios in play. Would you be able to share a set of instructions along with code of how to reproduce the issue you're currently having?

@alexcrichton Thank you. Does wasm support struct, I have tested by printf, before this line, it is OK
https://github.com/ZW007/lenet5/blob/6b60eb9fd8233e4503c5ff9b637229810ec4b719/lenet.c#L271
Where Feature is a struct defined here:
https://github.com/ZW007/lenet5/blob/245d95cdc141eb46fc5e58193e239f8cb8245e15/lenet.h#L60

To reproduce the error, In readme, you can find the error I had in runing inside wasmtime.
https://github.com/ZW007/lenet5

view this post on Zulip Wasmtime GitHub notifications bot (Nov 20 2020 at 21:24):

ZW007 deleted a comment on Issue #2430:

Sorry it sounds like there's a lot of moving parts here and there's perhaps more than a few scenarios in play. Would you be able to share a set of instructions along with code of how to reproduce the issue you're currently having?

@alexcrichton Thank you. Does wasm support struct, I have tested by printf, before this line, it is OK
https://github.com/ZW007/lenet5/blob/6b60eb9fd8233e4503c5ff9b637229810ec4b719/lenet.c#L271
Where Feature is a struct defined here:
https://github.com/ZW007/lenet5/blob/245d95cdc141eb46fc5e58193e239f8cb8245e15/lenet.h#L60

To reproduce the error, In readme, you can find the error I had in runing inside wasmtime.
https://github.com/ZW007/lenet5

view this post on Zulip Wasmtime GitHub notifications bot (Nov 20 2020 at 21:26):

ZW007 commented on Issue #2430:

Sorry it sounds like there's a lot of moving parts here and there's perhaps more than a few scenarios in play. Would you be able to share a set of instructions along with code of how to reproduce the issue you're currently having?

@alexcrichton Thank you. Does wasm support struct, I have tested by printf, before this line, it is OK
https://github.com/ZW007/lenet5/blob/6b60eb9fd8233e4503c5ff9b637229810ec4b719/lenet.c#L271
Where Feature is a struct defined here:
https://github.com/ZW007/lenet5/blob/245d95cdc141eb46fc5e58193e239f8cb8245e15/lenet.h#L60

To reproduce the error, In readme, you can find the error I had in runing inside wasmtime.
https://github.com/ZW007/lenet5

view this post on Zulip Wasmtime GitHub notifications bot (Nov 25 2020 at 17:33):

alexcrichton commented on Issue #2430:

Sorry for taking a bit to get back, but running that wasm file I get "Dataset File Not Find!Please Copy Dataset to the Floder Included the exe", so I think I'm missing some files?

Also have you seen this file work in other wasm environments? Is there anything in particular that makes you think wasmtime is at fault?

view this post on Zulip Wasmtime GitHub notifications bot (Nov 25 2020 at 18:45):

ZW007 commented on Issue #2430:

Sorry for taking a bit to get back, but running that wasm file I get "Dataset File Not Find!Please Copy Dataset to the Floder Included the exe", so I think I'm missing some files?

@alexcrichton Hi Alex! Sorry for the mess. I have deleted the wasm file in that repo. Please refer to the new repo.
You could clone that repo, get the wasm file, and run the wasm (need to give --dir=. permission) using :
wasm32-wasi-clang -I ./ lenet.c main.c -lm -o mainWasi.wasm
wasmtime --dir=. mainWasi.wasm

It should work if you have downloaded the mnist dataset, extra it and put them in the same dir: http://yann.lecun.com/exdb/mnist/

The extracted files need to named by:
t10k-images.idx3-ubyte
t10k-labels.idx1-ubyte
train-images.idx3-ubyte
train-labels.idx1-ubyte

Also have you seen this file work in other wasm environments? Is there anything in particular that makes you think wasmtime is at fault?

Yes, I ran it in Nodejs and worked fine. That is how I ran it, first generate wasm file and glue code,

emcc -s ALLOW_MEMORY_GROWTH=1 -s NODERAWFS=1 -I ./ lenet.c main.c -lm -o

After that, there is an a.out.index.js file create by me, https://github.com/ZW007/lenet5/blob/main/a.out.index.js

node a.out.index.js

You should find node works fine then,

view this post on Zulip Wasmtime GitHub notifications bot (Nov 25 2020 at 18:50):

ZW007 edited a comment on Issue #2430:

Sorry for taking a bit to get back, but running that wasm file I get "Dataset File Not Find!Please Copy Dataset to the Floder Included the exe", so I think I'm missing some files?

@alexcrichton Hi Alex! Sorry for the mess. I have deleted the wasm file in that repo. Please refer to the new repo.
You could clone that repo, get the wasm file, and run the wasm (need to give --dir=. permission) using :
wasm32-wasi-clang -I ./ lenet.c main.c -lm -o mainWasi.wasm
wasmtime --dir=. mainWasi.wasm

It should work if you have downloaded the mnist dataset, extra it and put them in the same dir: http://yann.lecun.com/exdb/mnist/

The extracted files need to named by:
t10k-images.idx3-ubyte
t10k-labels.idx1-ubyte
train-images.idx3-ubyte
train-labels.idx1-ubyte

Also have you seen this file work in other wasm environments? Is there anything in particular that makes you think wasmtime is at fault?

Yes, I ran it in Nodejs and worked fine. That is how I ran it, first generate wasm file and glue code, using NODERAWFS=1 for file mapping

emcc -s ALLOW_MEMORY_GROWTH=1 -s NODERAWFS=1 -I ./ lenet.c main.c -lm -o

After that, there is an a.out.index.js file create by me, https://github.com/ZW007/lenet5/blob/main/a.out.index.js run this js file by

node a.out.index.js

You should find node works fine then,

view this post on Zulip Wasmtime GitHub notifications bot (Nov 25 2020 at 18:53):

ZW007 edited a comment on Issue #2430:

Sorry for taking a bit to get back, but running that wasm file I get "Dataset File Not Find!Please Copy Dataset to the Floder Included the exe", so I think I'm missing some files?

@alexcrichton Hi Alex! Sorry for the mess. I have deleted the wasm file in that repo. Please refer to the new repo.
You could clone that repo, get the wasm file, and run the wasm (need to give --dir=. permission) using :
wasm32-wasi-clang -I ./ lenet.c main.c -lm -o mainWasi.wasm
wasmtime --dir=. mainWasi.wasm

It should work if you have downloaded the mnist dataset, extra it and put them in the same dir: http://yann.lecun.com/exdb/mnist/

The extracted files need to named by:
t10k-images.idx3-ubyte
t10k-labels.idx1-ubyte
train-images.idx3-ubyte
train-labels.idx1-ubyte

Also have you seen this file work in other wasm environments? Is there anything in particular that makes you think wasmtime is at fault?

Yes, I ran it in Nodejs and worked fine. That is how I ran it, first generate wasm file and glue code, using NODERAWFS=1 for file mapping

emcc -s ALLOW_MEMORY_GROWTH=1 -s NODERAWFS=1 -I ./ lenet.c main.c -lm -o

After that, there is an a.out.index.js file create by me, https://github.com/ZW007/lenet5/blob/main/a.out.index.js run this js file by

node a.out.index.js

You should find node works fine then. Please let me know if there are still problems, thank you!

view this post on Zulip Wasmtime GitHub notifications bot (Nov 25 2020 at 21:55):

alexcrichton commented on Issue #2430:

I believe the issue you're running into is stack overflow. If during the linking phase you tell LLD to create a larger stack then it makes more progress. I believe the reason why it seems emscripten works and C doesn't is that presumably emscripten allocates a larger stack by default. You can allocate a larger stack by passing -Wl,-z,stack-size=1048576 to the compilation.

view this post on Zulip Wasmtime GitHub notifications bot (Nov 25 2020 at 23:13):

ZW007 commented on Issue #2430:

I believe the issue you're running into is stack overflow. If during the linking phase you tell LLD to create a larger stack then it makes more progress. I believe the reason why it seems emscripten works and C doesn't is that presumably emscripten allocates a larger stack by default. You can allocate a larger stack by passing -Wl,-z,stack-size=1048576 to the compilation.

@alexcrichton Thank you Alex! I asked a silly question again, sorry (

I suspected the problem is from memory and read the wasmtime tutorial http://docs.wasmtime.dev/introduction.html
but i did not find anything about stack allocation. I know it is clang usage, not wasmtime, maybe you could add some about clang, to make it nicer to the newbie like me )

view this post on Zulip Wasmtime GitHub notifications bot (Nov 26 2020 at 01:28):

ZW007 closed Issue #2430:

I am running a standalone wasm file by using wasmtime --dir=. mainWasi.wasm
The wasm needs to read some local files, the size of local files in total is 10MB

Error: failed to run main module mainWasi.wasm

Caused by:
0: failed to invoke command default
1: wasm trap: out of bounds memory access
wasm backtrace:
0: 0x29cb - <unknown>!Predict
1: 0x346d - <unknown>!testing
2: 0x38ef - <unknown>!foo
3: 0x39eb - <unknown>!__original_main
4: 0x2c1 - <unknown>!_start

I wonder if I can allocate more memory for wasmtime? In lucet, i can use --min-reserved-size, but haven't found similar in wasmtime, but even when i use lucetc-wasi --min-reserved-size 6442450944 -o mainLucet.so mainWasi.wasm to compile in lucet, I still get the error in lucet run: thread 'main' panicked at 'instance can be created: ModuleError(IncorrectModule("heap spec sizes would overflow: HeapSpec { reserved_size: 6442450944, guard_size: 4194304, initial_size: 131072, max_size: None }"))', lucet-wasi/src/main.rs:239:24

What I am running is a standalone lenet inference task. I dont think it can be called a large app. Maybe I should try to read just one image from the file system, rather than the whole t10k-images-idx3-ubyte which is 7.8MB.

Thank you!

view this post on Zulip Wasmtime GitHub notifications bot (Nov 30 2020 at 15:08):

alexcrichton commented on Issue #2430:

No worries, and makes sense that we should add it to the tutorial!

Last updated: Nov 22 2024 at 17:03 UTC