Joel Dice (Sep 12 2024 at 16:26): Joel Dice (Sep 12 2024 at 16:26):@James Sturtevant These instructions for build a NativeAOT-LLVM runtime and ILC should be up to date: https://github.com/dotnet/runtimelab/blob/feature/NativeAOT-LLVM/docs/workflow/building/coreclr/nativeaot.md. You'll probably want to build using my PR branch: https://github.com/dicej/runtimelab/tree/system-net-http-wasi
And here's what I have so far for the SslStream experiment: https://github.com/dicej/dotnet-wasi-tls. I haven't made many changes to the .NET runtime code yet, but I'll push those changes to a branch when I've made significant progress. cc @Pavel Šavara
Pavel Šavara (Sep 12 2024 at 17:00):I just learned that Mono/WASI Net10 SDK and workload is not going to be available soon. So using it from componentize-dotnet is blocked. I don't really understand the details of why, but it's some versioning hell in the product construction.
@Larry Ewing could maybe share more.
That should not prevent me from working on sockets upstream or James from work on SslStream upstream either, but we would have to merge it to NAOT_LLVM first before that would be visible to broader audience. And maybe working on NAOT_LLVM branch directly would be way to go. We will see.
Mono/WASI in Net9 is broken for other reasons and we are not going to try to fix it, because many of the fixes are only on main branch, which is Net10 now.
James Sturtevant (Sep 16 2024 at 17:45):this was the sql library we want to use ssltream in https://github.com/dotnet/SqlClient?
Joel Dice (Sep 16 2024 at 17:52):yes. I'll grab that Windows machine I made the build changes to and send you the patch.
Joel Dice (Sep 16 2024 at 17:55):diff --git a/src/Microsoft.Data.SqlClient/netcore/src/Microsoft.Data.SqlClient.csproj b/src/Microsoft.Data.SqlClient/netcore/src/Microsoft.Data.SqlClient.csproj
index 494bd0b9..69f908e1 100644
--- a/src/Microsoft.Data.SqlClient/netcore/src/Microsoft.Data.SqlClient.csproj
+++ b/src/Microsoft.Data.SqlClient/netcore/src/Microsoft.Data.SqlClient.csproj
@@ -2,10 +2,9 @@
<PropertyGroup>
<AssemblyName>Microsoft.Data.SqlClient</AssemblyName>
<TargetFrameworks>net8.0;net6.0</TargetFrameworks>
- <GeneratePlatformNotSupportedAssemblyMessage Condition="'$(OSGroup)' == 'AnyOS'">Microsoft.Data.SqlClient is not supported on this platform.</GeneratePlatformNotSupportedAssemblyMessage>
<OSGroup Condition="'$(OSGroup)' == ''">$(OS)</OSGroup>
+ <TargetsUnix Condition="'$(OSGroup)'!='Unix'">true</TargetsUnix>
<TargetsWindows Condition="'$(OSGroup)'=='Windows_NT'">true</TargetsWindows>
- <TargetsUnix Condition="'$(OSGroup)'=='Unix'">true</TargetsUnix>
<!-- Allow explicit addition of the Compile files instead of all project files to be included by Default -->
<EnableDefaultCompileItems>false</EnableDefaultCompileItems>
<TargetGroup Condition="'$([MSBuild]::GetTargetFrameworkIdentifier($(TargetFramework)))'=='.NETCoreApp'">netcoreapp</TargetGroup>
@@ -37,7 +36,7 @@
<EmbeddedFiles Include="$(TargetFrameworkMonikerAssemblyAttributesPath)" />
</ItemGroup>
<!-- Contains common items shared between NetFx and NetCore -->
- <ItemGroup Condition="'$(OSGroup)' != 'AnyOS'">
+ <ItemGroup>
<Compile Include="$(CommonPath)\CoreLib\System\Threading\Tasks\TaskToApm.cs">
<Link>Common\CoreLib\System\Threading\Tasks\TaskToApm.cs</Link>
</Compile>
@@ -874,7 +873,7 @@
</ItemGroup>
<!-- Unix only -->
- <ItemGroup Condition="'$(TargetsUnix)' == 'true'">
+ <ItemGroup Condition="'$(TargetsWindows)' != 'true'">
<Compile Include="$(CommonPath)\Interop\Unix\Interop.Libraries.cs">
<Link>Common\Interop\Unix\Interop.Libraries.cs</Link>
</Compile>
There's probably a cleaner, better way to do it, but that's what I used.
James Sturtevant (Sep 16 2024 at 19:55):seem on windows I didn't need these changes... was able to pull the library in and compile a small app. I think these might be needed for building on linux?
Joel Dice (Sep 16 2024 at 19:55):They are for building for WASI
James Sturtevant (Sep 16 2024 at 19:56):yea, I see now app was building a wasi binary, then when I pulled in the library it stopped for some reason...
James Sturtevant (Sep 17 2024 at 00:12):what was the break point you set for seeing the stack trace? I am getting Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object. I guess this is were I would use that to see where it is throwing
James Sturtevant (Sep 17 2024 at 00:14):btw I am able to build and run with without the patch, unless I am missing something.
Joel Dice (Sep 17 2024 at 00:47):break S_P_CoreLib_System_Runtime_EH__RhpThrowEx
James Sturtevant (Sep 17 2024 at 15:45):What was the command you were using to build SQL client and the errors that lead to changing those lines in the csproj? I seem to be making some progress without those changes. When you get on later today maybe its worth 30 mins to show you what I have to make sure I am on track?
Joel Dice (Sep 17 2024 at 17:03):I was using cmd.exe, so I don't think I have shell history, but we can pair up and retrace those steps. I do remember I had to iterate on the patch before I got it to run all the way to SocketAsyncEngine and fail when trying to populate a thread pool. If you're seeing failures before it gets to that point, it probably means you need part or all of my patch.
Want to pair up at 2pm PDT today? Or else let me know your availability. I'll be free 2-4pm PDT today.
James Sturtevant (Sep 17 2024 at 17:11):2pm works for me. running wasmtime on windows is really slow :-/. I think I am making the same progress you are but should see. I just set up on linux and getting a whole different set of errors and they are way more expressive
Joel Dice (Sep 17 2024 at 17:15):I think what I did was build the SqlClient nupkg on Windows, copy it over to my Linux machine, and build and test my app there. It all should work on Windows, but Linux is just more comfortable and familiar for me.
Joel Dice (Sep 17 2024 at 17:56):BTW, I'm thoroughly refactoring my dotnet-tls repo so it mirrors (a subset of) the System.Net.Sockets and System.Net.Security APIs, with the goal of making it a drop-in replacement for those assemblies (e.g. for use in Microsoft.Data.SqlClient).
Joel Dice (Sep 17 2024 at 17:56):So hopefully we won't need to change much code in SqlClient -- just some using statements.
Joel Dice (Sep 17 2024 at 21:00):just sent you a Teams meeting invite
Larry Ewing (Sep 17 2024 at 21:03):Pavel Šavara said:
I just learned that Mono/WASI Net10 SDK and workload is not going to be available soon. So using it from componentize-dotnet is blocked. I don't really understand the details of why, but it's some versioning hell in the product construction.
Larry Ewing could maybe share more.That should not prevent me from working on sockets upstream or James from work on SslStream upstream either, but we would have to merge it to NAOT_LLVM first before that would be visible to broader audience. And maybe working on NAOT_LLVM branch directly would be way to go. We will see.
Mono/WASI in Net9 is broken for other reasons and we are not going to try to fix it, because many of the fixes are only on main branch, which is Net10 now.
The challenge here is that adding a new framework (net10) while in the middle of a release has proved challenging in the past. I'm feeling better about the potential timeline today than I was last week. I will keep @Pavel Šavara updated with the status.
Larry Ewing (Sep 17 2024 at 22:33):Joel Dice said:
just sent you a Teams meeting invite
If you could extract a public test case around how the connection is initiated and processed It sounds like there might be alternate solutions?
Joel Dice (Sep 17 2024 at 22:45):If you could extract a public test case around how the connection is initiated and processed It sounds like there might be alternate solutions?
For context: our goal is to get Microsoft.Data.SqlClient working (with few or no changes) on WASI. As a stepping stone towards that goal (and as a way of proving the concept), we're temporarily replacing its use of System.Net.Sockets and System.Net.Security with our own minimal alternatives. That will allow us to prove that WASI (when augmented with the proposed wasi:sockets/tls draft) is capable of supporting this use case. Once we've done that, we can work on proper WASI support for System.Net.Sockets and System.Net.Security in the .NET runtime (Pavel is making awesome progress on the former, but the latter hasn't been started). And once that's done, we should be able to make SqlClient work more-or-less unmodified.
With all that in mind, can you clarify what you mean by your question? What kind of public test case are you looking for, and what do you mean by "alternate solutions"? Do you mean a different way of approaching the problem of socket and TLS support on WASI?
Joel Dice (Sep 17 2024 at 23:12):@James Sturtevant I just pushed a bit update to https://github.com/dicej/dotnet-wasi-tls, including minimal Dns, TcpClient, and SslStream implementations. It all compiles fine but won't run until we have a host-side wasi:sockets/tls implementation, which I'll work on tomorrow.
Larry Ewing (Sep 17 2024 at 23:16):I meant I was curious about how the api was being used outside the wasi case because it sounded like there was some potential to adjust the api usage that wasn't directly related to the wasi work
Joel Dice (Sep 17 2024 at 23:20):You mean adjust the API usage in the SqlClient code? Yes, thanks for reminding me. At minimum, the "Async open requests are handled serially" part of https://github.com/dotnet/SqlClient/discussions/2612 will need to be addressed.
Joel Dice (Sep 17 2024 at 23:22):In the meantime, we can hack around that by adding a synchronous Connect method to my fake TcpClient. @Pavel Šavara won't let that leak into the real implementation, but I can do what I want in my fake one :)
Larry Ewing (Sep 17 2024 at 23:26):Yes, no need to block on the ideal solution but it sounds like something upstream would like to fix so lets add weight to that.
James Sturtevant (Sep 18 2024 at 00:56):@Joel Dice Upgrading 24.04 and using gdb 15.0.50 and now able to set breakpoints plus get much more info in output
Joel Dice (Sep 18 2024 at 21:13):As promised, I've added a Wasmtime-based host implementation of wasi:sockets/tls to https://github.com/dicej/dotnet-wasi-tls. I'm able to send HTTPS GET requests to arbitrary websites and print the responses.
@James Sturtevant maybe we could pair up tomorrow on getting SqlClient to use this?
James Sturtevant (Sep 18 2024 at 21:47):sure, I've got the sqlclient mostly wired in with wasi-tls. working on the getting tcp in fully
James Sturtevant (Sep 18 2024 at 21:47):Just got #2 0x00007ffeaad500e3 in S_P_CoreLib_Internal_Runtime_CompilerHelpers_ThrowHelpers__ThrowMissingMethodException (id=MissingMethod, methodName=...)
#3 0x00007ffeaf903410 in Microsoft_Data_SqlClient_WasiEventLoop___Register_g__CallRegister_1_0 (var0=<optimized out>, var1=<optimized out>, var2=<optimized out>, var3=<optimized out>) at .tmpgxmt8A.wasm:79957694
Joel Dice (Sep 18 2024 at 21:53):Hmm, looks like UnsafeAccessor can't find the Thread.RegisterWasiPollableHandle method in the runtime. Which branch of runtimelab did you build from?
James Sturtevant (Sep 18 2024 at 21:54):actually looks like I dropped tcp client before done connecting...
James Sturtevant (Sep 18 2024 at 21:55):I am using the released version right now, as you provided DNS the impl
Joel Dice (Sep 18 2024 at 21:56):ok, I'm still using my https://github.com/dotnet/runtimelab/pull/2614 branch; I haven't kept track of which changes are where, but my branch is probably the safest one to use right now since that's what I'm testing.
James Sturtevant (Sep 18 2024 at 22:01):fair enough, I didn't have it building this morning when I started so was going till I ran into something but it does seem that I am missing that register. Will try it with the packages I built from your branch
Joel Dice (Sep 18 2024 at 22:03):FYI, I've added instructions to the dotnet-wasi-tls repo README.md that tells you how to download pre-built packages for various platforms, if that's easier.
James Sturtevant (Sep 18 2024 at 22:27):using those packages did the trick. Made it alittle further!
James Sturtevant (Sep 19 2024 at 13:57):Had to make a few changes to wasi-tls host world but I've got it connecting. The TLS handshake is failing with EOF, going to try a few things this morning but if you want to pair that would be helpful!
---> Microsoft.Data.SqlClient.SqlException (0x80131904): A connection was successfully established with the server, but then an error occurred during the pre-login handshake. (provider: SSL Provider, error: 31 - Encryption(ssl/tls) handshake failed)
---> System.AggregateException: One or more errors occurred. (TLS handshake failed on hostside: ImportsWorld.None)
Awesome! I've got meetings until 9am PDT, but free after that. When works best for you?
James Sturtevant (Sep 19 2024 at 15:34):That time works for me
James Sturtevant (Sep 20 2024 at 19:19):cargo run --release --manifest-path host/Cargo.toml ../wasi-sql/bin/Debug/net9.0/wasi-wasm/native/sqlclient-wasi.wasm
Finished release profile [optimized] target(s) in 0.22s
Running host/target/release/host ../wasi-sql/bin/Debug/net9.0/wasi-wasm/native/sqlclient-wasi.wasm
Hello from WASI!
Stream creation
Stream creation
ServerVersion: 12.00.5688
State: Open
1
2
goodbye
James Sturtevant (Sep 20 2024 at 19:20):Connected :tada:
Joel Dice (Sep 20 2024 at 20:05):That's awesome! Did you have to do anything else besides set up a server that could accept "Encrypt=strict" connections?
James Sturtevant (Sep 20 2024 at 20:17):and fix the DNS look up :-) but other than that it worked
James Sturtevant (Sep 21 2024 at 00:11):saw your comments and will address Monday. Do we want to sync up early next week to talk next steps too?
Joel Dice (Sep 21 2024 at 01:35):Yup; just sent you and Pavel an invite.
Ralph (Sep 23 2024 at 10:25):effing brilliant here, all of you
Pavel Šavara (Sep 23 2024 at 13:33):
Pavel Šavara (Sep 23 2024 at 13:33):I had to disable many, many tests ... but this will unblock us for further adventure
James Sturtevant (Sep 23 2024 at 16:29):I am back and around for rest of the day, if want to have another quick sync
Last updated: Dec 13 2025 at 20:04 UTC