Stream: git-wasmtime

Topic: wasmtime / issue #7730 WASI conformance - HTTP test progr...

Wasmtime GitHub notifications bot (Dec 28 2023 at 09:23):

guybedford opened issue #7730:

Feature

As part of the JCO implementation of wasi:http, I found it useful to rewrite the test runners as WASI commands themselves, which reference the expected server details through environment variables. As a result, the tests can then be run fully independently of the Wasmtime test suite, since if all the test programs pass as commands, that means there is conformance.

Benefit

The benefit of this approach is simplifying WASI conformance testing to always be defined in terms of test program completion, given the expected environment for the given test program is configured correctly.

Implementation

I went ahead and worked on this refactoring for the JCO HTTP tests in https://github.com/bytecodealliance/wasmtime/compare/main...guybedford:wasmtime:http-incoming-tests-refactor, which reimplements all the client tests as WASI HTTP client tests instead of running through the Rust tests, introducing three new test programs to perform these tests: proxy_handler, proxy_streaming and proxy_echo. These tests are fully migrated and should comprehensively cover the existing tests.

For now this branch is enough to support JCO conformance testing only, not Wasmtime as well. So this will only be suitable to PR when it removes the existing Rust tests in Wasmtime and replaces them with a harness to do the calls through client test programs.

Alternatives

There are of course many alternatives - it would be great to get feedback on if this approach seems good to upstream, and if so it we can continue working towards getting the PR ready. Any further feedback or suggestions is very much welcome.

Wasmtime GitHub notifications bot (Jan 02 2024 at 21:01):

alexcrichton commented on issue #7730:

I think this'd be a great idea myself. To confirm I understand the result here, the thinking is to take all the "spin up a server tests" and where today we use reqwest to send HTTP requests to the server this would instead run a wasm client which would itself then make HTTP requests and assert properties of the responses? If so :+1: from me. If not probably still :+1: I may just need clarification!

Wasmtime GitHub notifications bot (Jan 02 2024 at 22:46):

guybedford commented on issue #7730:

Yes that's correct - so the test programs use environment variables to know where to send the requests, and are implemented as commands which use WASI HTTP outgoing handler as a client to verify the correct responses. As mentioned this refactoring of the tests into commands is already done on https://github.com/bytecodealliance/wasmtime/compare/main...guybedford:wasmtime:http-incoming-tests-refactor, just not the update to Wasmtime's own test suite yet to move off the current model.

Last updated: Nov 22 2024 at 16:03 UTC