Stream: git-wasmtime

Topic: wasmtime / PR #11064 Add `native-tls` backend for wasi-tls

Wasmtime GitHub notifications bot (Jun 17 2025 at 18:30):

badeend requested wasmtime-wasi-reviewers for a review on PR #11064.

Wasmtime GitHub notifications bot (Jun 17 2025 at 18:30):

badeend opened PR #11064 from badeend:native_tls to bytecodealliance:main:

During the initial wasi-tls implementation discussion, the suggestion was floated to support both rustls and native-tls. The initial/current implementation only supports rustls. This PR adds native-tls as an alternative TLS backend.

The default TLS provider remains rustls. Consumers can switch to the native-tls backend by enabling the corresponding feature flags and then use the newly created WasiTlsCtxBuilder::provider method. On the CLI, this is exposed as -S tls-provider=nativetls.

Adding this new backend immediately surfaced two bugs:

1. Improved compatibility of test_tls_sample_application

The test used to perform a "half-close" after sending the HTTP request. This is a TLS1.3+ feature, though Rustls & OpenSSL already supported it for TLS1.2 and lower. Technically, that makes them non-spec compliant, but they chose to align with the semantics of the underlying TCP connection. I suspect the TLS1.3 spec was updated to match what was already happening in reality.

Anyhow, SChannel does not support half-closed connections and so the read call after the close_output+shutdown failed. I've reordered the test to first perform the HTTP conversation and _then_ do the TLS+TCP teardown.

2. Implement flushing on the AsyncWriteStream.

The AsyncWriteStream implementation was copied from the TCP equivalent, which doesn't need flushing. The TLS implementations _do_ maintain an internal buffer, so the flush call needs to be hooked up. Without it, the test would hang after making the change mentioned above.

Misc: Reorganized wasi-tls folder

To make life easier for myself, I've shuffled some code around. I've isolated the rustls-specific bits from the rest of the implementation. The folder structure now looks like:

• providers/*: The rustls & native-tls specific parts.
• bindings.rs: The generated bindings.
• host.rs: The host types + impls.
• io.rs: WASI I/O conversion utility types.

Wasmtime GitHub notifications bot (Jun 17 2025 at 18:30):

badeend requested wasmtime-core-reviewers for a review on PR #11064.

Wasmtime GitHub notifications bot (Jun 17 2025 at 18:30):

badeend requested alexcrichton for a review on PR #11064.

Wasmtime GitHub notifications bot (Jun 17 2025 at 18:30):

badeend requested wasmtime-default-reviewers for a review on PR #11064.

Wasmtime GitHub notifications bot (Jun 17 2025 at 18:40):

badeend updated PR #11064.

Wasmtime GitHub notifications bot (Jun 18 2025 at 14:59):

alexcrichton commented on PR #11064:

This all looks great to me, thanks again for working on this!

My only comment would be could this update CI checks to perform a build of the wasmtime-wasi-tls crate with some feature combinations? I'd be interested to see --no-default-features --features rustls and one for native-tls to ensure the crate can built with only one of the two enabled.

I'll file a separate PR to perform the vets as well.

Wasmtime GitHub notifications bot (Jun 19 2025 at 08:35):

badeend edited PR #11064:

During the initial wasi-tls implementation discussion, the suggestion was floated to support both rustls and native-tls. The initial/current implementation only supports rustls. This PR adds native-tls as an alternative TLS backend.

The default TLS provider remains rustls. Consumers can switch to the native-tls backend by enabling the corresponding feature flags and then use the newly created WasiTlsCtxBuilder::provider method. On the CLI, this is exposed as -S tls-provider=nativetls.

Adding this new backend immediately surfaced two bugs:

1. Improved compatibility of test_tls_sample_application

The test used to perform a "half-close" after sending the HTTP request. This is a TLS1.3+ feature, though Rustls & OpenSSL already supported it for TLS1.2 and lower. Technically, that makes them non-spec compliant, but they chose to align with the semantics of the underlying TCP connection. I suspect the TLS1.3 spec was updated to match what was already happening in reality.

Anyhow, SChannel does not support half-closed connections and so the read call after the close_output+shutdown failed. I've reordered the test to first perform the HTTP conversation and _then_ do the TLS+TCP teardown.

2. Implement flushing on the AsyncWriteStream.

The AsyncWriteStream implementation was copied from the TCP equivalent, which doesn't need flushing. The TLS implementations _do_ maintain an internal buffer, so the flush call needs to be hooked up. Without it, the test would hang after making the change mentioned above.

Misc: Reorganized wasi-tls folder

To make life easier for myself, I've shuffled some code around. I've isolated the rustls-specific bits from the rest of the implementation. The folder structure now looks like:

• providers/*: The rustls & native-tls specific parts.
• bindings.rs: The generated bindings.
• host.rs: The host types + impls.
• io.rs: WASI I/O conversion utility types.

CC: @jsturtevant

Wasmtime GitHub notifications bot (Jun 19 2025 at 08:51):

badeend updated PR #11064.

Wasmtime GitHub notifications bot (Jun 19 2025 at 08:51):

badeend updated PR #11064.

Wasmtime GitHub notifications bot (Jun 19 2025 at 08:56):

badeend commented on PR #11064:

Is this the right place to do that?

Wasmtime GitHub notifications bot (Jun 19 2025 at 17:00):

jsturtevant created PR review comment:

does it make sense to have to set two feature flags to enable the feature or should we provide a default to enable the feature or should we provide a default?

Also a bit confused in the way to set this up. Do I need to configure both feature flags (wasi-tls and nativetls or rusttls) and then also set the provider with wasi_tls_ctx: WasiTlsCtxBuilder::new().provider(provider).build(),?

Wasmtime GitHub notifications bot (Jun 19 2025 at 17:00):

jsturtevant submitted PR review:

Great finds in the bugs. LGTM me overall

Wasmtime GitHub notifications bot (Jun 19 2025 at 19:08):

badeend updated PR #11064.

Wasmtime GitHub notifications bot (Jun 19 2025 at 19:09):

badeend submitted PR review.

Wasmtime GitHub notifications bot (Jun 19 2025 at 19:09):

badeend created PR review comment:

rustls is part of the default feature. So by default, DefaultProvider will point to the rustls provider and there's no need to configure anything on the WasiTlsCtxBuilder.

Similarly, if you build with --no-default-features --features nativetls, then DefaultProvider will point to the native_tls provider and there's again no need to configure anything on the WasiTlsCtxBuilder.

The only reason to call WasiTlsCtxBuilder::provider is when the crate has been built with multiple providers and you want to switch between them at run time. This is what happens in e.g. the test suite and in the CLI code.

I've added some docs to clarify this.

Wasmtime GitHub notifications bot (Jun 20 2025 at 14:15):

alexcrichton submitted PR review:

Yep that works :+1:

Wasmtime GitHub notifications bot (Jun 20 2025 at 18:57):

alexcrichton commented on PR #11064:

Hm ok so this is actually going to be kind of tricky. We test crates in CI with --all-features but we also test cross-compilations to various architectures, and native-tls on Linux wants OpenSSL which wants a system copy by default which isn't present for cross-compiled variants.

One option might be to have a feature such as openssl-vendored on the wasmtime-wasi-tls crate which builds a vendored copy of openssl-src. That way --all-features of that would be activated an OpenSSL would be built from source?

That's not an ideal option though IMO as ideally we wouldn't test native-tls by default and we'd instead have a dedicated job for just testin native-tls, but with --all-features that's not easy :(

Wasmtime GitHub notifications bot (Jun 22 2025 at 09:45):

badeend commented on PR #11064:

How about moving /crates/wasi-tls/src/providers/native_tls.rs into a dedicated wasi-tls-nativetls crate and excluding that entire crate from the default test runs?

Wasmtime GitHub notifications bot (Jun 22 2025 at 09:56):

badeend edited a comment on PR #11064:

How about moving /crates/wasi-tls/src/providers/native_tls.rs into a dedicated wasi-tls-nativetls crate and excluding that entire crate from the default test runs?

Edit: And set up a separate CI job to test wasi-tls-nativetls in isolation on a Linux, MacOS and Windows host

Wasmtime GitHub notifications bot (Jun 22 2025 at 09:57):

badeend edited a comment on PR #11064:

How about moving /crates/wasi-tls/src/providers/native_tls.rs into a dedicated wasi-tls-nativetls crate and exclude that entire crate from the default test runs?

Edit: And set up a separate CI job to test wasi-tls-nativetls in isolation on a Linux, MacOS and Windows host

Wasmtime GitHub notifications bot (Jun 22 2025 at 10:15):

badeend edited a comment on PR #11064:

How about moving /crates/wasi-tls/src/providers/native_tls.rs into a dedicated wasi-tls-nativetls crate and exclude that entire crate from the default test runs?

Edit: And set up a separate CI job to test wasi-tls-nativetls in isolation on a Linux, MacOS and Windows host.

Edit2: A downside of this approach is that the wasmtime CLI may not reference the wasi-tls-nativetls package anymore either.

Wasmtime GitHub notifications bot (Jun 23 2025 at 16:45):

alexcrichton commented on PR #11064:

I think what you're proposing is probably the best option for now. You're right that it means that the CLI won't be easily buildable with native-tls, but that seems like a reasonable state of affairs for the time being and we can address that later if need be

Wasmtime GitHub notifications bot (Jun 26 2025 at 19:43):

badeend updated PR #11064.

Wasmtime GitHub notifications bot (Jun 26 2025 at 20:00):

badeend updated PR #11064.

Wasmtime GitHub notifications bot (Jun 26 2025 at 20:34):

badeend updated PR #11064.

Wasmtime GitHub notifications bot (Jun 26 2025 at 20:39):

badeend commented on PR #11064:

Could you take a look at my latest changes? I've ran it with prtest:full and everything (except cargo vet) now succeeds.

Wasmtime GitHub notifications bot (Jun 27 2025 at 14:31):

alexcrichton updated PR #11064.

Wasmtime GitHub notifications bot (Jun 27 2025 at 14:31):

alexcrichton has enabled auto merge for PR #11064.

Wasmtime GitHub notifications bot (Jun 27 2025 at 15:15):

alexcrichton merged PR #11064.

Last updated: Dec 06 2025 at 07:03 UTC