Stream: general

Topic: Endive


view this post on Zulip andreaTP (May 27 2026 at 09:05):

Announcing Endive, a WebAssembly runtime built natively for the JVM, now a Bytecode Alliance project.
Zero native dependencies. One JAR, every OS.

https://bytecodealliance.org/articles/endive-and-the-next-chapter-of-webassembly-on-the-jvm
join the conversation: #endive

view this post on Zulip Zachary Whitley (Tegmentum) (Jun 30 2026 at 16:29):

Congratulations. I just released webassembly4j with Endive support.

view this post on Zulip Zachary Whitley (Tegmentum) (Jun 30 2026 at 17:01):

I've also got a bunch of new projects coming out one of which might go well with Endive. I've ported over a complete JDK to Webassembly supporting both OpenJ9 and Hotspot. I'm using for a project called Svalinn that runs a JRE on WebAssembly....in a JRE and Endive would be cool because it would all be Java. Why would you do such a thing? Say you get hit with another Log4J CVE. Svalinn will auto generate a compatible facade and drop in jar that replaces Log4J with a Log4J that is now running on WebAssembly. It doesn't get rid of the vulnerability but now it's sandboxed (Java got rid of the SecurityManager) It's a little slower but at least it keeps you running and safe until you can get a fix.

view this post on Zulip andreaTP (Jun 30 2026 at 17:20):

Very interesting to see this happening! So far the most known project for running a full Jvm in Wasm was using CheerpJ.
Is your project open source?

view this post on Zulip Zachary Whitley (Tegmentum) (Jun 30 2026 at 17:21):

It is. Same license as OpenJDK

view this post on Zulip Zachary Whitley (Tegmentum) (Jun 30 2026 at 17:21):

All based on components so there's a JNI<->WIT bridge

view this post on Zulip Zachary Whitley (Tegmentum) (Jun 30 2026 at 17:25):

I think CheerpJ is targeted at browser deployment but this runs standalone just fine........or on WasmCloud? :thinking:

view this post on Zulip andreaTP (Jun 30 2026 at 17:32):

link?

JNI<->WIT bridge

automated or crafted?

CheerpJ is targeted at browser deployment

that's correct

view this post on Zulip Catalin Calistru (Jul 01 2026 at 08:41):

Is the wasm build for clickhouse usable with endive.run on the JVM? https://github.com/chdb-io/chdb-wasm https://wasm.chdb.io/

view this post on Zulip andreaTP (Jul 01 2026 at 09:32):

From the docs seems like it uses Memory64 which is not supported at the moment: https://www.npmjs.com/package/chdb-wasm

view this post on Zulip Catalin Calistru (Jul 01 2026 at 10:19):

andreaTP said:

From the docs seems like it uses Memory64 which is not supported at the moment: https://www.npmjs.com/package/chdb-wasm

Thank you very much!

view this post on Zulip andreaTP (Jul 01 2026 at 11:51):

@Zachary Whitley (Tegmentum) your projects are missing in the adopters page: https://github.com/bytecodealliance/endive/blob/main/ADOPTERS.md happy to have a PR fixing it :slight_smile:

view this post on Zulip Zachary Whitley (Tegmentum) (Jul 01 2026 at 19:01):

Thanks. Just put in a PR

view this post on Zulip Zachary Whitley (Tegmentum) (Jul 01 2026 at 19:11):

I'm just getting the repo ready to open up. It's a bit of a mess and I wanted to clean it up a bit but I'll let you know as soon as it does. I also have componentized runtimes for Python, MicroPython, R, Octive, Fortran and Cobol on the way too. I know there's Pyodide but it's built with Emscripten and doesn't use components. I'm just finishing up getting to package parity with Pyodide and I have an escape hatch that allows you to run any python native package (although with poor performance but it will run until a webassembly package can be built.

You can run python inside a webassembly sanbox in Java on Endive...or you can run Java in WebAssembly running Endive, runing Python.

view this post on Zulip Zachary Whitley (Tegmentum) (Jul 01 2026 at 19:18):

I'll have to check out chdb-wasm . I'm guessing they're using mem64 to get over the 4Gb limit. I've come across that several times and the unfortunate thing is there's a performance cliff when using it. To get around that I built something that gets you to 256Gb while staying within mem32. The performance penalty is much more gentle and you don't pay the price until you go over 4Gb. It's called Tiered Virtual Memory (TVM) https://github.com/tegmentum/tvm-wasm It made for a nice compromise. 256Gb is a pretty reasonable size and the performance profile is nice while staying within mem32

It's what I use to scale my webassembly JDK heap over 4Gb. The project is called Fiji

I'll take a look at chdb and see if I can get it running with TVM

view this post on Zulip Till Schneidereit (Jul 01 2026 at 19:53):

@Zachary Whitley (Tegmentum) that all sounds quite fantastic—very exciting to see so many more guest runtimes! For Python, note that componentize-py already provides full support for components and WASIp2 and p3. Might be useful to compare that to your solution

view this post on Zulip Zachary Whitley (Tegmentum) (Jul 02 2026 at 17:04):

That’s a fair comparison. My understanding is that componentize-py is primarily about turning a Python application into a WebAssembly component targeting a WIT world. Pylon (my cpython packaging in wasm) is aimed at a different layer. The Python runtime is not bundled into every produced component, and packages/extensions are modeled as reusable components themselves.

So the distinction I’d draw is:

componentize-py = Python app -> self-contained component.

Pylon = Python runtime as infrastructure + Python packages/native extensions as independently composable components.

That matters for package distribution, reuse, cold-start behavior, native-extension support, and avoiding the “every Python artifact carries its own runtime” model. So yes, it’s absolutely worth comparing against componentize-py, but I don’t think they’re solving the same packaging problem.

Conceptually it's closer to "Pyodide rebuilt around the Component Model" than "Python code compiled into a component."

view this post on Zulip Zachary Whitley (Tegmentum) (Jul 02 2026 at 17:06):

It was a great demonstration of the value of the componet model. A lot of work could be reused at each step because of it. numpy -> LAPACK -> Fortran -> R

view this post on Zulip Till Schneidereit (Jul 03 2026 at 14:26):

thanks, that makes sense. The plan for componentize-py is to eventually also support reusing the runtime. In fact, componentize-py itself won't need to do anything for that: components in principle support referencing contained core modules (and contained components) as external files. That's something wasmtime embedders can already make use of, but the CLI doesn't yet support. I talked to @Alex Crichton about this just yesterday though, and we have a rough plan for how to go about it.

But with that used, I think componentize-py should be able to achieve the same level of reuse as what you're describing

view this post on Zulip Zachary Whitley (Tegmentum) (Jul 03 2026 at 14:57):

That makes sense, and it's good to hear there's active work in that direction.

I think the distinction I'm trying to draw is less about whether runtime reuse is theoretically possible in the Component Model and more about the packaging model built around it.

With Pylon, the runtime, stdlib, and packages/extensions are all independently distributable components rather than application-local assets. In that model something like NumPy becomes a reusable component with Python bindings rather than a Python package embedded inside a component.

Pylon also isn't tied to a single runtime or packaging ecosystem. Today it supports both CPython and MicroPython runtimes as well as both pip and uv workflows on top of the same component substrate.

That said, I definitely see the value in componentize-py. The UX of "take this Python application and turn it into a component" is extremely appealing, and I think there are a lot of use cases where that's exactly the right abstraction. Pylon is aiming at a somewhat different problem space, but I wouldn't be surprised if the two approaches end up sharing a lot of ideas over time.

view this post on Zulip Zachary Whitley (Tegmentum) (Jul 03 2026 at 15:04):

Also, I'd be interested in learning more about where componentize-py is headed. The UX is very appealing, and I suspect we're going to encounter many of the same underlying problems from different directions.

I suspect Python package distribution and packaging semantics are going to end up being the majority of the work in this space, and I'd rather not see every project independently rebuild that machinery if we can avoid it.

One of the things I've been watching with Pyodide is that they're solving many of the same problems from an Emscripten centric world with the recent PEP 783 work. My bias is that the Component Model eventually provides more natural primitives for a lot of those concerns.

view this post on Zulip Till Schneidereit (Jul 03 2026 at 15:11):

entirely agreed in particular on that last part! And my hope would be that even with componentize-py's model, we should eventually see the ability to use prebuilt wheels targeting wasm/wasi that can be referenced (or bundled in) by a Python component. IIUC, what you're building is in a way more component model-native, whereas componentize-py is more geared towards something that fits in with how native extensions are built today: those are .dylibfiles that get loaded "as though" they were dynamically loaded. So they should be distributable the same as dylibs for native platforms.

@Joel Dice could say more about all this, though!

view this post on Zulip Zachary Whitley (Tegmentum) (Jul 03 2026 at 15:25):

That's interesting. One thing I'm curious about is how the security and capability story works in that model.

One of the things that pushed me toward component-native packages was the idea that packages and native extensions could have their own explicit interfaces and capability envelopes rather than inheriting whatever authority the Python runtime itself has.

If native extensions are loaded more like traditional dylibs/so files, are those still isolated Wasm artifacts with their own capabilities, or do they effectively execute with the authority of the Python runtime that loaded them?

I've already componentized roughly the equivalent surface that Pyodide exposes, and for cases that fall outside that envelope I've built a customized version of v86 as a compatibility fallback capable of running arbitrary native Python packages inside WebAssembly, albeit with a significant performance penalty.

I'd definitely be interested in understanding how componentize-py is thinking about that tradeoff.

view this post on Zulip Alex Crichton (Jul 03 2026 at 17:10):

The componentize-py model of shared-everything-linking is indeed where extensions have the same authority as the python runtime itself. Componentizing dependencies (like numpy) would indeed provide a much stronger security boundary yeah

view this post on Zulip Till Schneidereit (Jul 05 2026 at 10:57):

agreed yes: isolating extensions into their own components provides much better isolation. I'm under the impression that for a lot of extensions, including numpy, that would impose prohibitive overheads though, since it'd mean that you can't make buffers accessible without copying to both the extension and Python code.

I wonder if there'd be advantages to supporting both in the same approach: selectively running extensions as either their own components for better isolation, or in the same address space for better performance & compatibility?

I've been thinking about this in the context of our JS runtime, StarlingMonkey, as well, fwiw

view this post on Zulip Zachary Whitley (Tegmentum) (Jul 06 2026 at 00:11):

That is exactly the tradeoff I’ve been thinking about.

Not every extension necessarily has to run as a separate isolated component. For something like NumPy, forcing every buffer interaction through a hard component boundary would likely destroy the performance model users expect.

What I’m actually doing in Pylon is supporting both models. Extensions can either be:

We also track what has been bundled so that collapsing an extension into the runtime remains an explicit and auditable decision rather than an invisible optimization.

That allows isolation and performance to become deployment decisions rather than architectural constraints.

The thing I’m trying to avoid is making “extension loaded into the Python runtime” the only authority model available.

For NumPy specifically, I agree that the data and buffer model is the hard part. But for many extensions like parsers, crypto libraries, codecs, database clients, policy engines, protocol implementations, and similar service-oriented components, the stronger component boundary may be a good default.

I suspect the long-term answer is likely to be a mixture of execution models selected according to the characteristics and trust requirements of the extension rather than a single universal approach.

This pattern has come up in a number of projects that I'm working on at the moment.

view this post on Zulip Joel Dice (Jul 06 2026 at 14:44):

Cool stuff, @Zachary Whitley (Tegmentum); thanks for sharing!

I don't have a lot to add to what's been discussed already, but I'll note that we chose to use shared-everything linking rather than shared-nothing component linking for native extensions in componentize-py given that the Python C API assumes a shared-everything model (i.e. it relies heavily on pointers, direct buffer access, callback functions, etc.), and it wasn't clear how to preserve the semantics of that API correctly across a component boundary. The component model doesn't have a direct equivalent to pointers or callbacks, for example, so it's not obvious how e.g. PyBytes_AsStringAndSize or PyDict_AddWatcher would be bridged across such a boundary. I'd be curious to know how you handled cases like that.

I do love the idea of automatically sandboxing native extensions in their own components, even if it involves a performance tradeoff, but I always assumed it would be difficult or impossible in the general case given the shared-everything design of the Python C API.

view this post on Zulip Brett Cannon (Jul 06 2026 at 17:48):

Zachary Whitley (Tegmentum) said:

Pylon (my cpython packaging in wasm)

Just an FYI that "Pylon" is a known name in the Python community: https://github.com/Pylons .

Till Schneidereit said:

we should eventually see the ability to use prebuilt wheels targeting wasm/wasi that can be referenced (or bundled in) by a Python component.

Anyone interested in this might also be interested in the monthly Python subgroup call.

view this post on Zulip Zachary Whitley (Tegmentum) (Jul 06 2026 at 17:51):

Ugh, eternal search for a uniqe name....thanks for the heads up.

view this post on Zulip Zachary Whitley (Tegmentum) (Jul 06 2026 at 17:58):

...well I was going to release it but now you're just going to have to wait while I rename it :grinning:


Last updated: Jul 29 2026 at 05:03 UTC