wasmtime / issue #7706 Unable to read data > 2^31 - 4096 ... · git-wasmtime

Stream: git-wasmtime

Topic: wasmtime / issue #7706 Unable to read data > 2^31 - 4096 ...

Wasmtime GitHub notifications bot (Dec 19 2023 at 17:24):

HI,

I have a wasi-http component written in C that stores an uploaded file to the filesystem. (it runs under this).

This seems to work fine up to a file size of 2^31 - 4096 (2147479552), i.e 4KiB shy of 2GiB, which seems suspicious ;).

I'm basically doing

do {
        ok = wasi_io_streams_method_input_stream_blocking_read(
                                                        b_in_stream,
                                                        8 * 1024*1024,
                                                        &data,
                                                        &in_stream_err);
        if (!ok)
                break;

        bytes_wrote = write(fd, data.ptr, data.len);
        printf("Read : %ld bytes, Wrote : %ld bytes\n",
               data.len, bytes_wrote);

        total_bytes_wrote += bytes_wrote;
} while (total_bytes_wrote < content_len);

With a 2GiB file I see

...
Read : 8388608 bytes, Wrote : 8388608 bytes
Read : 8388608 bytes, Wrote : 8388608 bytes
Read : 8384512 bytes, Wrote : 8384512 bytes

The last read was 4KiB short. The next call to wasi_io_streams_method_input_stream_blocking_read() returns false.

(Note: I've stripped the version information from the function/type names for usability)

If I try and read the error from in_stream_err I get

thread 'tokio-runtime-worker' panicked at src/lib.rs:221:45:
failed to handle request: failed to invoke wasm `handle`

Caused by:
    0: error while executing at wasm backtrace:
           0: 0x1d441 - wit-component:shim!indirect-wasi:io/streams@0.2.0-rc-2023-10-18-[method]error.to-debug-string
           1: 0x1772 - wasi_io_streams_method_error_to_debug_string
                           at /home/andrew/src/project_blackbird/c/wit-spin/large-upload/proxy.c:738:3
           2: 0x26e3 - exports_wasi_http_incoming_handler_handle
                           at /home/andrew/src/project_blackbird/c/wit-spin/large-upload/component.c:90:4
           3: 0x2299 - __wasm_export_exports_wasi_http_incoming_handler_handle
                           at /home/andrew/src/project_blackbird/c/wit-spin/large-upload/proxy.c:2098:3
    1: unknown handle index 0

_Maybe_ because in_stream_err.tag is 1 which is _maybe_ WASI_IO_STREAMS_STREAM_ERROR_CLOSED

Is something using a s32 somewhere it shouldn't?

Wasmtime GitHub notifications bot (Dec 19 2023 at 18:20):

alexcrichton commented on issue #7706:

Thanks for the report! Would you be able to create a standalone wasm file that works in the Wasmtime CLI to reproduce this? I tried locally and a Rust-compiled component which uses std::fs::copy copied a 5G file without problems for me locally, so I wasn't able to trivially reproduce

Wasmtime GitHub notifications bot (Dec 19 2023 at 20:34):

ac000 commented on issue #7706:

Hi Alex, I was able to modify my component to run under wasmtime serve minus the writing of the data as I can't do --dir with serve, but that bit's not important here anyway.

For some reason my printf(3)'s don't work under wasmtime serve 14.0.4 (this is the version this needs to currently work with) but do with 15.0.1, so I had to send the important info back to curl, but long story short
$ curl -T 2G localhost:8080
Content-Length : 2147483648, total_bytes_read : 2147483648
so it seems my problem lies elsewhere...

Wasmtime GitHub notifications bot (Dec 19 2023 at 23:34):

ac000 closed issue #7706:

HI,