Stream: jco

Topic: A @bytecodealliance/wstd for JS

Victor Adossi (May 15 2025 at 02:37):

I've been wanting a library under the jco project wherein we can put tooling (essentially an SDK) that would make the developer experience of JS components more convenient. After a bit of discussion in the JS subgroup meeting yesterday since there were no large objections, I'd like to discuss it here as well.

An example use case for this is enabling an express-like facade for users that want to build wasi:http/incoming-handler based servers:

import { express } from '@bytecodealliance/wstd/http/servers';

const app = express();

app.get("/hello/:name", (req, res) => {
    res.send(`hello ${req.params.name}`);
})

app.listen();

The above code should produce a component that exports wasi:http-incoming-handler, and be fairly easy to modify and most importantly be familiar to server-side JS devs today.

The idea is that before Node builtins land in StarlingMonkey (or anywhere below jco), we want to give users a good experience while they build JS components.

Ideally, this project completely goes away when all of NodeJS and it's ecosystem works smoothly in WebAssembly, but as we've seen with other runtimes (Bun, Deno, etc) -- that can take quite a while, and corner-cases abound.

Any feedback on this before I make the proposal would be welcome.

One thing I'm struggling with (assuming this is a good idea) whether to call it wstd (following in the footsteps of the similarly named Rust crate by @Yosh Wuyts ), or jco-sdk (since it may not necessarily be only standard library functions), or even jco-std.

Victor Adossi (May 15 2025 at 02:46):

Tagging some people who might be interested to make sure this doesn't get lost in the ether:
@Till Schneidereit @Tomasz Andrzejak @Calvin Prewitt @Guy Bedford

Milan (rajsite) (May 15 2025 at 02:51):

Are you already familiar with the hono examples shared last JS working group meeting? We already have access to a production quality web framework with sophisticated middleware.

I also went in depth at the wasmCloud community meeting:
https://www.youtube.com/live/wSG_ypHAiiM?t=136

It works really well and workflows like oauth, oidc, SSR JSX components, etc are all there.

What I'd really like in something like std is a filesystem api readily available. Would be great to align on node:fs for wide familiarity but deno's FS api is really well aligned on platform features like leveraging WHATWG streams while less familiar is easy to pick up.

Milan (rajsite) (May 15 2025 at 02:52):

The dream is already here :)
https://github.com/rajsite/wasm-component-examples/blob/main/examples/hono-features-app/src/lib/app.ts

Victor Adossi (May 15 2025 at 03:00):

You Betcha! I work with @Lachlan Heywood :)

The problem is that most developers actually don't know Hono -- they're familiar with Express:

• express has 166M downloads a week
• hono has 3M
• fastify has 10M

It's not even close, for better or for worse. And what makes hono work (it's been built wonderfully) is it's closeness to standards and built in flexibility -- the reason it was able to be adapted so well is because of how Hono was built, and unfortunately other older frameworksa ren't quite that way.

I'm interested in meeting developers where they are today, and making a good experience for them and I think a facade for Express would do that -- hopefully I'm not mistaken here.

Also, just a side note, it's absolutely intended that we have the hono implementation (and others!) available as part of this new SDK!

What I'd really like in something like std is a filesystem api readily available. Would be great to align on node:fs for wide familiarity but deno's FS api is really well aligned on platform features like leveraging WHATWG streams while less familiar is easy to pick up.

I may be biased, but I also think this is a good fit for this model :)

Having a fs module (maybe next to http) that looks a lot like fs/promises from Node (with a little fs-extra thrown in!) but is not bound to strict Node compat would be great for developers.

Victor Adossi (May 15 2025 at 03:06):

We of course want to be standards forward, but adoption is usually quite tied with familiarity -- Hono is a great example of doing both -- their API looks like Fastify but is incredibly flexible due to their adoption of standards.

We want something similar :)

Also, another example that people are going to want to do is be able to access functionality like random/crypto that they would normally just import. Right now there are two ways they can do that:

• Add & use wasi:random/random themselves
• Bundle a pure-JS implementation of sha256/simple hashing functions

(I didn't pull that usecase out of thin air, one of our customers actually asked for this/tried to do this almost immediately after starting w/ the jco ecosystem)

One cool thing that we might be forced to deal with in this case is a system for noting the dependencies of various projects in NodeJS packages -- if you import a random helper, it would be great if in the package.json we could either find the wit file (if it was included in the package), or just know that wasi:cli/random is used (so we can update the resulting component accordingly.

Milan (rajsite) (May 15 2025 at 03:06):

I see the argument, but my concern is that what makes express valuable is the ecosystem and plugins. I think folks can handle the hello world / simple router example difference to hono and I'm skeptical of how a look alike express will be taken if the ecosystem around it is incompatible. I feel like that was a big lesson learned from the Deno folks and them / bun have done massive efforts for deep node compatibility to capture that mindshare. Getting express to work is nice but getting, i.e. next.js working is useful and a huge leap.

Milan (rajsite) (May 15 2025 at 03:11):

Express-ed my concern (:drum:) and won't hijack the convo, and I certainly know users / AI agents that could use express-like syntax to get started quicker!

Victor Adossi (May 15 2025 at 03:14):

I see the argument, but my concern is that what makes
express valuable is the ecosystem and plugins.

I agree here, but I think the problem is fractal in that way -- the ecosystem and plugins will be in a some-supported-some-not until we get Node builtins at the StarlingMonkey level (and likely even after, because that obviously doesn't solve natively built packages).

I think the question is, is an express-like DX possible before then, or not.

I think folks can handle the hello world / simple router example difference to hono and I'm
skeptical of how a look alike express will be taken if the ecosystem
around it is incompatible.

I see the question here -- my reasoning is pretty simple:

• If Hono compat is good, Express compat is even better
• Hono suffers from the same ecosystem, as soon as people try to use libraries that depend on platform functionality (i.e. those that are a bit further from Hono's principles, or just anything in the NodeJS ecosystem). We don't have an official platform name w/ Hono (yet?), so there's no way for libraries to build for WASI.

I feel like that was a big lesson learned from the Deno folks and them / bun have done massive efforts for deep node compatibility to capture that mindshare. Getting express to work is nice but getting, i.e. next.js working is useful and a huge leap.

I agree, but it took them literally years to get there, and they're not all the way there yet :)

The work at the StarlingMonkey level is certainly crucial (NodeJS support at that level), but there's probably a bunch of time between now and then.

The goal is for large swaths of this new library to be obsoleted by StarlingMonkey level NodeJS support (and eventually, someday, maybe, fully upstreamed NodeJS WASI support), but until then, just like the Hono adapter delivers value, it would be great to give other (sometimes larger) parts of the ecosystem the same treatment.

Victor Adossi (May 15 2025 at 03:20):

Oh also @Milan (rajsite) If you'd like to contribute an example to jco of a Hono example that looks like the example component you have, I'd love to have it in there -- it would be a great help

Victor Adossi (May 15 2025 at 04:19):

Sorry @Milan (rajsite), I got too excited (it's day time where I am):

https://github.com/bytecodealliance/jco/pull/699

Would appreciate a review! :bow:

Milan (rajsite) (May 15 2025 at 05:32):

Gave some initial feedback but off to bed :sleeping:

Victor Adossi (May 15 2025 at 06:50):

A huge amount of thoughtful feedback, thank you so much! Just got through the first round of them, will wait for you to see & resolve the remaining unresolved ones then merge! :rocket:

Tomasz Andrzejak (May 15 2025 at 07:32):

Also, another example that people are going to want to do is be able to access functionality like random/crypto that they would normally just import.

random/crypto are part of the WinterTC specification and are implemented in StarlingMonkey:
https://w3c.github.io/webcrypto/#crypto-interface

Things like this should just work:

const hashBuffer = await crypto.subtle.digest('SHA-256', data);

Or did you mean something else?

Victor Adossi (May 15 2025 at 08:54):

Thanks for the pointer -- I did mean the Crypto API! I stand corrected, the min common api spec certainly has that use case covered, I guess the gap then is just documentation, and eventual Node API support so code "just works" when it's imported.

That's not a great example, but a better one is fs which was already mentioned, bearing some similarity to the node-like APIs that people generally reach for today.

As far as I'm concerned the most compelling examples are widely used libraries like express or fs-extra, -- things that don't "just work" yet, but are critical for building a non-trivial app. These are the first things someone will have imported, and until we have Node API support at the StarlingMonkey level (the ultimate solution here), people will have a hard time adopting the Wasm JS ecosystem.

Victor Adossi (May 15 2025 at 08:57):

Oh and the WinterTC Crypto interface is exactly WebCrypto, so that's probably worth noting, it's more of an endorsement of the existing browser standard

Victor Adossi (May 15 2025 at 08:59):

Come to think of it, we could definitely use an example of this. In fact, an example of all of the WinterTC APIs in use would be fantastic

Victor Adossi (May 15 2025 at 09:05):

https://github.com/bytecodealliance/jco/issues/701

Tomasz Andrzejak (May 15 2025 at 09:08):

Yes, also when https://github.com/bytecodealliance/StarlingMonkey/pull/238 is merged I would add a section that outlines current status of wintertc compatibility, similar to this:

https://runtime-compat.unjs.io/

then we could link to it to give users an idea of what is supported.

Victor Adossi (May 15 2025 at 09:15):

It would great to have StarlingMonkey on that list!

Tomasz Andrzejak (May 15 2025 at 09:47):

I guess it is there indirectly through fastly runtime :)

Ralph (Jun 18 2025 at 13:11):

where would we be at with respect to this at the moment? No complaints! Just doing a survey....

Victor Adossi (Jun 18 2025 at 13:14):

Hey @Ralph so I'd love to get this going but I need a few administrative things taken care of to be able to do this... jco is waiting on componentize-js for a release, I can't create new JS projects under the @bytecodealliance org (so neither this project nor jco-transpile) can take off.

Basically we're at a bit of a loggerheads until someone can essentially add more maintainers to the JS ecosystem projects on the NPM projects/etc.

Ralph (Jun 18 2025 at 13:15):

totally understood. what do you mean by, "I can't create new JS projects under the org..."?

Victor Adossi (Jun 18 2025 at 13:15):

There's at least a tsc PR for jco-transpile: https://github.com/bytecodealliance/governance/pull/127

Victor Adossi (Jun 18 2025 at 13:15):

Ralph said:

totally understood. what do you mean by, "I can't create new JS projects under the org..."?

Ah as in I don't have NPM permissions for the @bytecodealliance-js org (IIRC the name was something like that)

Ralph (Jun 18 2025 at 13:17):

OK, so....

1. componentize-js needs a release for jco
2. You or someone else needs to create a repo for this or jco-transpile
3. something about jco-transpile taking off..... :-)

Victor Adossi (Jun 18 2025 at 13:18):

also would be great to be able to release preview2-shim and preview3-shim

Victor Adossi (Jun 18 2025 at 13:19):

Likely this project may end up being @bytecodealliance/jco-std to match jco-transpile but either way -- we're not held up by the code here, just admin stuff

Ralph (Jun 18 2025 at 13:22):

OK, so LOW hanging fruit is a new jco-std/jco-transpile in the org, yes?

Victor Adossi (Jun 18 2025 at 13:33):

Yeah! Would appreciate any help :bow: both those projects plus preview3-shim would be good (we don't have that one yet but we should have it soon, so making the project would be good)

Till Schneidereit (Jun 18 2025 at 13:34):

hello! I'll try to revive myself from an overly long hiatus and ramp back up on things

Ralph (Jun 18 2025 at 13:35):

a wild @Till Schneidereit BTW, there is no such thing as an overly long hiatus. I want one, too, dammit.

Till Schneidereit (Jun 18 2025 at 13:35):

oh, if only it had been something like a sabbatical, instead of overwhelm with both other work and sadness ...

Ralph (Jun 18 2025 at 13:36):

actually, a sabbatical would in fact be better.

Till Schneidereit (Jun 18 2025 at 13:39):

very quickly on the jco-transpile project: I'm still not really convinced that it makes sense to split this out into its own repository. In my mind, that would only make sense if both of these conditions held:

1. there is a fundamentally separate set of maintainers with no real overlap
2. there are no dependencies between the two

I don't think either of these holds, and in particular worry about the second one. To give a completely different example, we moved Cranelift into the Wasmtime repository years ago because we realized that we'd either have to do that, or over time would end up writing a second embedding replicating much of Wasmtime, simply because to fully and properly test Cranelift we'd need a pretty comprehensive testing embedding. And then the question is, why isn't that just Wasmtime. And since you really don't want circular (developer workflow) dependencies across multiple repositories, merging them became obvious.

Till Schneidereit (Jun 18 2025 at 13:41):

What I'd instead strongly favor is restructuring the jco repository to contain both of these as separate, independently usable packages, but still being able to lean on jco for fully exercising the transpile API surface

Till Schneidereit (Jun 18 2025 at 13:41):

(this is also the reason why I strongly favor having the next iteration of / replacement for componentizeJS live inside the StarlingMonkey repo)

Victor Adossi (Jun 18 2025 at 13:42):

In this case, the request is actually driven by user demand -- I think we risk fragmenting the ecosystem by not allowing users (folks at Arcjet in this case) to use jco's transpilation stuff only.

To be clear this is to add a jco-transpile project, which can be used from jco -- so I think we're agreed actually? Right now we don't have independently usable packages, we have just jco but it pulls in stuff not needed in particular by transpile.

Victor Adossi (Jun 18 2025 at 13:42):

The goal here was to have jco-transpile that is used by jco similarly to how jco uses componentize-js -- this leaves people who only want transpilation with a package they can pull in directly

Till Schneidereit (Jun 18 2025 at 13:43):

I'm not saying that we should keep the two strongly tied together: I'm saying we shouldn't split them into separate repositories

Victor Adossi (Jun 18 2025 at 13:43):

OH also a key misunderstanding here -- there's no new repo! jco has been monorepo'd at this point:

https://github.com/bytecodealliance/jco/tree/main/packages

It would go in there

Till Schneidereit (Jun 18 2025 at 13:44):

ah, good! And right, I knew that at some point, but it slipped my mind, apologies

Victor Adossi (Jun 18 2025 at 13:44):

Both this idea, jco-std, would all live under the existing repo!

Victor Adossi (Jun 18 2025 at 13:44):

No worries, I actually refactored it only a little while ago

Till Schneidereit (Jun 18 2025 at 13:45):

in this case then, the main question is whether it needs to, and what it'd mean to be a new project. If you look at the wasm-tools repo for example, that contains a ton of independently usable libraries and apps, none of which have their own identity as hosted projects

Victor Adossi (Jun 18 2025 at 13:45):

the "new project" I was trying to get was under the NPM org really, not GH

Till Schneidereit (Jun 18 2025 at 13:45):

I was about to reply to that part, and yes: obviously we do need to get that sorted, and I'll work on it

Till Schneidereit (Jun 18 2025 at 13:47):

ideally in a way where it's not bottle-necked anymore. I think the easiest way for that would be to have a CD setup we could use for multiple packages. @Victor Adossi, you and I talked about setting that up a while ago; did you do anything towards automated releases, by any chance?

Victor Adossi (Jun 18 2025 at 13:47):

As far as being a hosted project, I think I made that clear in the proposal: https://github.com/bytecodealliance/governance/pull/127/files

I was actually following the SM model, w/ the listing of associated projects (https://github.com/bytecodealliance/governance/tree/main/projects). If it's not listed as a hosted project that's fine as well, but what matters most to me is the usability aspect/unblocking the people who requested it... though I do wonder if they're still goin gto bother!

Victor Adossi (Jun 18 2025 at 13:48):

Oh yes! So Jco releases (including p2-shim, component-bindgen) are fully automated now -- I can't do an actual release for p2-shim because I'm not a maintainer there yet though

Till Schneidereit (Jun 18 2025 at 13:49):

ah, okay. The idea is that the projects list mentions individual projects, and where there's not a 1:1 mapping with repositories it makes that clear. In this case I don't think we need to change anything about the list: neither are there multiple repositories for one project, nor multiple projects in one repository

Victor Adossi (Jun 18 2025 at 13:50):

I think it might be good to get a user w/ credentials that can be contributed to the BCA 1Pass into the relevant NPM orgs/projects -- that way we don't get blocked in the future

I made a jco-release-bot GH account that's now in there and it's been very useful (it's used in the automation)

Till Schneidereit (Jun 18 2025 at 13:51):

okay, I just added the p2 shim and componentizeJS to the jco-publishers team on npm, so that should be sorted now, I think

Victor Adossi (Jun 18 2025 at 13:55):

Thanks, that's great -- so as far as making projects like @bytecodealliance/jco-transpile happen, how would that work?

Till Schneidereit (Jun 18 2025 at 14:00):

what should the description for the project be?

Victor Adossi (Jun 18 2025 at 14:01):

How about:

"Convert WebAssembly components to runnable ES modules" ?

Till Schneidereit (Jun 18 2025 at 14:02):

actually, from the documentation it looks like you should be able to create the package, being an org member

Victor Adossi (Jun 18 2025 at 14:02):

I'm also happy with the word "Transpile" over "Convert", what do you think?

Victor Adossi (Jun 18 2025 at 14:02):

Oh let me see

Victor Adossi (Jun 18 2025 at 14:04):

OK welp I guess I should be able to -- I'll try it out and report back iff something goes wrong! :bow:

Ralph (Jun 18 2025 at 16:23):

well, crap, this was a productive moment! Thanks a ton to you both!!!!

Last updated: Dec 06 2025 at 07:03 UTC