Stream: git-wasmtime

Topic: wasmtime / issue #8821 Directory read bug.

view this post on Zulip Wasmtime GitHub notifications bot (Jun 17 2024 at 12:33):

Userzxcvbvnm added the bug label to Issue #8821.

view this post on Zulip Wasmtime GitHub notifications bot (Jun 17 2024 at 12:33):

Userzxcvbvnm opened issue #8821:

Test Case

The c test case is:

#include <stdio.h>
#include <dirent.h>
#include <fcntl.h>
#include <unistd.h>


#include <errno.h>
#include <string.h>

void fd_readdir_00001_sYQM0(int fd) {
    printf("Enter function fd_readdir_00001_sYQM0\n");

    DIR *directory;
    struct dirent *entry;
    directory = fdopendir(fd);

    if (directory == NULL) {
        printf("fdopendir failed.\n");
        return;
    }

    while ((entry = readdir(directory)) != NULL) {
        printf("Get dir content:%s\n", entry->d_name);
    }

    printf("Print dir content finished.\n");
}

int get_fd(const char *filename, int flags) {
    int fd = open(filename, flags);

    if (fd == -1) {
        printf("Get file descriptor of file %s failed!\n", filename);
        return -1;
    } else {
        printf("Get file descriptor of file %s succeed!\n", filename);
        return fd;
    }
}

void closebyfd(int fd) {
    if (close(fd) == -1) {
        printf("Close the file %d by descriptor failed!\n", fd);
    }
}

int main() {
    int fd = get_fd("subdir_2/subdir_5", O_RDONLY | O_DIRECTORY);
    if (fd == -1) {
        return 1;
    }

    fd_readdir_00001_sYQM0(fd);

    closebyfd(fd);

    return 0;
}

Steps to Reproduce

(1)compile to wasm:./wasi-sdk-21.0/bin/clang --target=wasm32-unkown-wasi --sysroot=./wasi-sdk-21.0/share/wasi-sysroot test.c -o test.wasm

(2)Running wasm:
(Before run the Wasm file, directory subdir_2/subdir_5 exists, and contains sub files subfile_1, subfile_2, subfile_3 in it.
Before run the Wasm file, change the permission of subdir_2/subdir_5 into 0400(r--------) or 0600(rw-------). )
wasmtime run --dir=. test.wasm

Expected Results

Print:

Get file descriptor of file subdir_2/subdir_5 succeed!
Enter function fd_readdir_00001_sYQM0
Get dir content:subfile_2
Get dir content:subfile_3
Get dir content:.
Get dir content:..
Get dir content:subfile_1
Print dir content finished.

This is what WAMR and WasmEdge do.

Actual Results

wasmtime prints:

Get file descriptor of file subdir_2/subdir_5 succeed!
Enter function fd_readdir_00001_sYQM0
fdopendir failed.

wasmtime fails to read the file items in the directory although both permission 0400 and 0600 has the read permission.

Versions and Environment

Wasmtime version or commit: 19.0.2
Operating system: Ubuntu 20.04
Architecture: x86_64

view this post on Zulip Wasmtime GitHub notifications bot (Jun 17 2024 at 17:52):

alexcrichton commented on issue #8821:

This is an under-specified corner of this function in WASI. Right now Wasmtime will reopen the directory each time a directory iterator is requested to ensure that state on the file descriptor is not carried over from one read to the next. I don't know if other runtimes will have the same behavior if the same file descriptor is opened as a directory and read twice. This directory cannot be directly opened in this manner I believe because the execute permission bit is missing.

I think it would be best to open a bug with WASI to figure out the correct behavior here.

view this post on Zulip Wasmtime GitHub notifications bot (Jun 18 2024 at 02:44):

Userzxcvbvnm commented on issue #8821:

Thanks for your reply! I reproduce this with Linux native code (gcc test.c -o test, ./test) and Linux native has the same behavior with WAMR and WasmEdge. I'm not sure whether wasmtime should behave like this.

view this post on Zulip Wasmtime GitHub notifications bot (Jun 18 2024 at 02:44):

Userzxcvbvnm edited issue #8821:

Test Case

The c test case is:

#include <stdio.h>
#include <dirent.h>
#include <fcntl.h>
#include <unistd.h>


#include <errno.h>
#include <string.h>

void fd_readdir_00001_sYQM0(int fd) {
    printf("Enter function fd_readdir_00001_sYQM0\n");

    DIR *directory;
    struct dirent *entry;
    directory = fdopendir(fd);

    if (directory == NULL) {
        printf("fdopendir failed.\n");
        return;
    }

    while ((entry = readdir(directory)) != NULL) {
        printf("Get dir content:%s\n", entry->d_name);
    }

    printf("Print dir content finished.\n");
}

int get_fd(const char *filename, int flags) {
    int fd = open(filename, flags);

    if (fd == -1) {
        printf("Get file descriptor of file %s failed!\n", filename);
        return -1;
    } else {
        printf("Get file descriptor of file %s succeed!\n", filename);
        return fd;
    }
}

void closebyfd(int fd) {
    if (close(fd) == -1) {
        printf("Close the file %d by descriptor failed!\n", fd);
    }
}

int main() {
    int fd = get_fd("subdir_2/subdir_5", O_RDONLY | O_DIRECTORY);
    if (fd == -1) {
        return 1;
    }

    fd_readdir_00001_sYQM0(fd);

    closebyfd(fd);

    return 0;
}

Steps to Reproduce

(1)compile to wasm:./wasi-sdk-21.0/bin/clang --target=wasm32-unkown-wasi --sysroot=./wasi-sdk-21.0/share/wasi-sysroot test.c -o test.wasm

(2)Running wasm:
(Before run the Wasm file, directory subdir_2/subdir_5 exists, and contains sub files subfile_1, subfile_2, subfile_3 in it.
Before run the Wasm file, change the permission of subdir_2/subdir_5 into 0400(r--------) or 0600(rw-------). )
wasmtime run --dir=. test.wasm

Expected Results

Print:

Get file descriptor of file subdir_2/subdir_5 succeed!
Enter function fd_readdir_00001_sYQM0
Get dir content:subfile_2
Get dir content:subfile_3
Get dir content:.
Get dir content:..
Get dir content:subfile_1
Print dir content finished.

This is what WAMR, WasmEdge and Linux native code do.

Actual Results

wasmtime prints:

Get file descriptor of file subdir_2/subdir_5 succeed!
Enter function fd_readdir_00001_sYQM0
fdopendir failed.

wasmtime fails to read the file items in the directory although both permission 0400 and 0600 has the read permission.

Versions and Environment

Wasmtime version or commit: 19.0.2
Operating system: Ubuntu 20.04
Architecture: x86_64

view this post on Zulip Wasmtime GitHub notifications bot (Jun 18 2024 at 15:35):

alexcrichton commented on issue #8821:

Yes I understand that Linux enables this, but WASI isn't just a copy wholesale of Linux. This is specifically grappling with a concept not in WASI, namely file permissions. Given that I'm hesitant to commit any particular behavior in Wasmtime as it's not clear to me which is the way to go.

In general most of the cases you're running into are "holes" in the WASI specification, but WASI (in my opinion) is going to take a very long time if ever to completely and fully specify every possible interaction between filesystems. Given that we can try to patch things up as they come up but it's not really feasible to get 100% determinism across all possible runtimes across all possible platforms. For example I would be surprised if other runtimes has consistent behavior on https://github.com/bytecodealliance/wasmtime/issues/8817 across all platforms. Just because Linux does something doesn't mean that it's what should be codified in WASI.

I don't think that this issue has a solid resolution in WASI itself, but I still think it's worthwhile to open an issue there and get the discussion going. Either that or someone else can also chime in and say "no, Alex, behavior X is obviously the correct one to implement here" and Wasmtime can implement that.

view this post on Zulip Wasmtime GitHub notifications bot (Jun 19 2024 at 02:16):

Userzxcvbvnm commented on issue #8821:

Thanks for your reply! As you suggested, I'll open a bug with WASI to figure out the correct behavior.

view this post on Zulip Wasmtime GitHub notifications bot (Jun 25 2024 at 08:34):

Userzxcvbvnm edited issue #8821:

Test Case

The c test case is:

#include <stdio.h>
#include <dirent.h>
#include <fcntl.h>
#include <unistd.h>


#include <errno.h>
#include <string.h>

void fd_readdir_00001_sYQM0(int fd) {
    printf("Enter function fd_readdir_00001_sYQM0\n");

    DIR *directory;
    struct dirent *entry;
    directory = fdopendir(fd);

    if (directory == NULL) {
        printf("fdopendir failed.\n");
        return;
    }

    while ((entry = readdir(directory)) != NULL) {
        printf("Get dir content:%s\n", entry->d_name);
    }

    printf("Print dir content finished.\n");
}

int get_fd(const char *filename, int flags) {
    int fd = open(filename, flags);

    if (fd == -1) {
        printf("Get file descriptor of file %s failed!\n", filename);
        return -1;
    } else {
        printf("Get file descriptor of file %s succeed!\n", filename);
        return fd;
    }
}

void closebyfd(int fd) {
    if (close(fd) == -1) {
        printf("Close the file %d by descriptor failed!\n", fd);
    }
}

int main() {
    int fd = get_fd("subdir_2/subdir_5", O_RDONLY | O_DIRECTORY);
    if (fd == -1) {
        return 1;
    }

    fd_readdir_00001_sYQM0(fd);

    closebyfd(fd);

    return 0;
}

Steps to Reproduce

(1)compile to wasm:./wasi-sdk-21.0/bin/clang --target=wasm32-unkown-wasi --sysroot=./wasi-sdk-21.0/share/wasi-sysroot test.c -o test.wasm

(2)Running wasm:
(Before run the Wasm file, directory subdir_2/subdir_5 exists, and contains sub files subfile_1, subfile_2, subfile_3 in it.
Before run the Wasm file, change the permission of subdir_2/subdir_5 into 0400(r--------) or 0600(rw-------). )
wasmtime run --dir=. test.wasm

Expected Results

Print:

Get file descriptor of file subdir_2/subdir_5 succeed!
Enter function fd_readdir_00001_sYQM0
Get dir content:subfile_2
Get dir content:subfile_3
Get dir content:.
Get dir content:..
Get dir content:subfile_1
Print dir content finished.

This is what WAMR, WasmEdge and Linux native code do.

Actual Results

wasmtime prints:

Get file descriptor of file subdir_2/subdir_5 succeed!
Enter function fd_readdir_00001_sYQM0
fdopendir failed.

wasmtime fails to read the file items in the directory although both permission 0400 and 0600 has the read permission.

Versions and Environment

Wasmtime version or commit: 19.0.2 and 13.0.1
Operating system: Ubuntu 20.04
Architecture: x86_64

view this post on Zulip Wasmtime GitHub notifications bot (Sep 07 2024 at 12:46):

tschneidereit closed issue #8821:

Test Case

The c test case is:

#include <stdio.h>
#include <dirent.h>
#include <fcntl.h>
#include <unistd.h>


#include <errno.h>
#include <string.h>

void fd_readdir_00001_sYQM0(int fd) {
    printf("Enter function fd_readdir_00001_sYQM0\n");

    DIR *directory;
    struct dirent *entry;
    directory = fdopendir(fd);

    if (directory == NULL) {
        printf("fdopendir failed.\n");
        return;
    }

    while ((entry = readdir(directory)) != NULL) {
        printf("Get dir content:%s\n", entry->d_name);
    }

    printf("Print dir content finished.\n");
}

int get_fd(const char *filename, int flags) {
    int fd = open(filename, flags);

    if (fd == -1) {
        printf("Get file descriptor of file %s failed!\n", filename);
        return -1;
    } else {
        printf("Get file descriptor of file %s succeed!\n", filename);
        return fd;
    }
}

void closebyfd(int fd) {
    if (close(fd) == -1) {
        printf("Close the file %d by descriptor failed!\n", fd);
    }
}

int main() {
    int fd = get_fd("subdir_2/subdir_5", O_RDONLY | O_DIRECTORY);
    if (fd == -1) {
        return 1;
    }

    fd_readdir_00001_sYQM0(fd);

    closebyfd(fd);

    return 0;
}

Steps to Reproduce

(1)compile to wasm:./wasi-sdk-21.0/bin/clang --target=wasm32-unkown-wasi --sysroot=./wasi-sdk-21.0/share/wasi-sysroot test.c -o test.wasm

(2)Running wasm:
(Before run the Wasm file, directory subdir_2/subdir_5 exists, and contains sub files subfile_1, subfile_2, subfile_3 in it.
Before run the Wasm file, change the permission of subdir_2/subdir_5 into 0400(r--------) or 0600(rw-------). )
wasmtime run --dir=. test.wasm

Expected Results

Print:

Get file descriptor of file subdir_2/subdir_5 succeed!
Enter function fd_readdir_00001_sYQM0
Get dir content:subfile_2
Get dir content:subfile_3
Get dir content:.
Get dir content:..
Get dir content:subfile_1
Print dir content finished.

This is what WAMR, WasmEdge and Linux native code do.

Actual Results

wasmtime prints:

Get file descriptor of file subdir_2/subdir_5 succeed!
Enter function fd_readdir_00001_sYQM0
fdopendir failed.

wasmtime fails to read the file items in the directory although both permission 0400 and 0600 has the read permission.

Versions and Environment

Wasmtime version or commit: 19.0.2 and 13.0.1
Operating system: Ubuntu 20.04
Architecture: x86_64

view this post on Zulip Wasmtime GitHub notifications bot (Sep 07 2024 at 12:46):

tschneidereit commented on issue #8821:

Closing this as it's more of a WASI issue than a Wasmtime one—at least for now, as Wasmtime's behavior would have to change in case the spec is clarified in a way that makes the current behavior invalid. @Userzxcvbvnm if you ended up filing that issue, would you mind linking it here?

Last updated: Dec 23 2024 at 12:05 UTC