Ralph (Jan 19 2024 at 14:22): Ralph (Jan 19 2024 at 14:22):OK, bear with me, but I want to use javascript (NOT my language, though easy learn) to create a wasi:http component for preview 2. I can do custom wit components easily enough using this sample: https://github.com/bytecodealliance/ComponentizeJS/blob/main/EXAMPLE.md. What is done up front is to take a .wit file and then implement the function listed there, nothing fancy. So far so good. So I have the wasi wit block in a repo.... But I need to understand the binding step for js. In the example above, you merely reference hello and then implement hello and call jco componentize and you have a component.
Ralph (Jan 19 2024 at 14:22):But that's custom wit.
Ralph (Jan 19 2024 at 14:24):so for wasi:http, do I need to implement this?
Ralph (Jan 19 2024 at 14:24):interface incoming-handler {
use types.{incoming-request, response-outparam};
/// This function is invoked with an incoming HTTP Request, and a resource
/// response-outparam which provides the capability to reply with an HTTP
/// Response. The response is sent by calling the response-outparam.set
/// method, which allows execution to continue after the response has been
/// sent. This enables both streaming to the response body, and performing other
/// work.
///
/// The implementor of this function must write a response to the
/// response-outparam before returning, or else the caller will respond
/// with an error on its behalf.
handle: func(
request: incoming-request,
response-out: response-outparam
);
}
Ralph (Jan 19 2024 at 14:26):what I'm missing as a newbie here is the binding mechanism between the WIT function " and the js needed to implement it. Do I simply "implement the function" name, as I can do with the above hello function?
Ralph (Jan 19 2024 at 14:26):or is there a js binding generator for wasi:http/proxy that I need to locate and them merely "import" the types to use?
Ralph (Jan 19 2024 at 14:27):i get that concept, and I'm golden here.
Ralph (Jan 19 2024 at 15:14):wait, are these they: do I just need to reference them in the package.json?
Ralph (Jan 19 2024 at 15:14):},
"devDependencies": {
"@bytecodealliance/jco": "^0.14.2",
"@bytecodealliance/preview2-shim": "^0.14.2",
Ramon Klass (Jan 19 2024 at 15:23):I'm not sure if the shim includes wasi-http but in general yes, you add jco to your devDependencies and preview2-shim to your dependencies and then jco transpile'd components "just work" but as I said I only use preview2, I didn't follow. the wasi-http changes yet
Ralph (Jan 19 2024 at 15:27):I'm on preview 12_05, so that's fine
Ralph (Jan 19 2024 at 15:33):sorry for not using
Ralph (Jan 19 2024 at 15:33):
in terms of the JS to define to componentize, you'd want to write a JS file like:
export const incomingHandler = {
handle(request, response) {
// ...
},
};
we do have an example of using outgoing handler in ComponentizeJS in https://github.com/bytecodealliance/jco/blob/main/test/fixtures/componentize/wasi-http-proxy/source.js although it doesn't contain the incoming handler currently, so you'd need to check the types to figure it out.
Generating template JS / TypeScript bindings for these ComponentizeJS workflows has been a TODO for a while... agreed it should be a thing.
Ralph (Jan 19 2024 at 19:08):thanks Guy; if I've imported preview2-shim and npm intalled, I seem to get the incomingHandler type and it resolves, so hopefully that's really all I need to do.....
Ralph (Jan 19 2024 at 19:10):when you say generating bindings, what do you mean that's not:
image.png
Guy Bedford (Jan 19 2024 at 19:15):that looks right, although in ComponentizeJS the bindings have a different import convention - import 'wasi:http@0.2.0-rc-2024-01-16/incoming-handler etc instead
Guy Bedford (Jan 19 2024 at 19:16):but if you are just having this import for IDE types to work out making it an import type { IncomingHandler } from '@bytecodealliance/preview2/http' might make sense with tsc itself stripping that out
Ramon Klass (Jan 19 2024 at 19:16):at least with jco transpile, it does put the .d.ts files in the right locations and IDEs can just autocomplete imports
Guy Bedford (Jan 19 2024 at 19:17):yeah so if we had something similar for ComponentizeJS it might be useful to generate types that work for authoring components to arbitrary WIT
Ralph (Jan 19 2024 at 19:17):well, from my position I've been building all the wasi:http languages as best I can
Ralph (Jan 19 2024 at 19:18):I have dan's rust, Joe's go, and Joel's Python implementations of incomingHandler(). So here I am, wanting to make a JavaScript incomingHandler() impl for comparison.
Ralph (Jan 19 2024 at 19:19):but I couldn't figure out where to get the .js/.ts types for wasi:http
Ralph (Jan 19 2024 at 19:20):npm install preview2-shim gave them to me, which sure seemed easy to me......
Ralph (Jan 19 2024 at 19:21):but note, once I have this running I'll be at the point where I need to create a module from the .js (with spidermonkey) and then I can make that module a component.
Ramon Klass (Jan 19 2024 at 19:22):I think that's where componentizeJS comes in, it should implement the "make a component out of a js file and spidermonkey" step
Ralph (Jan 19 2024 at 19:23):that's what I think, too, but isn't jco component that comment?
Ralph (Jan 19 2024 at 19:23):command?
Ralph (Jan 19 2024 at 19:23):because help says that YOU MUST have a component on the argument line
Ramon Klass (Jan 19 2024 at 19:24):yeah I also was confused, from the docs both implement the same, @Guy Bedford probably needs to answer that
Ralph (Jan 19 2024 at 19:24):whoops, no, I'm wrong
Ralph (Jan 19 2024 at 19:24):apologies
Ralph (Jan 19 2024 at 19:25):It appears I need to: jco componentize -w <path> -o hello-wasi-http-js.wasm index.js
Ramon Klass (Jan 19 2024 at 19:25):transpile takes a component and turns that into browser and node-compatible js, componentize seems to implement the same as componentizeJS does, if going by the docs?
Ralph (Jan 19 2024 at 19:25):and that should work
Ralph (Jan 19 2024 at 19:26):yeah, just working over Guy's stuff so I can complain because he and wassim and yosh and others worked so hard
Ralph (Jan 19 2024 at 19:26)::-)
Ralph (Jan 19 2024 at 19:26):goint to give this a try
Ralph (Jan 19 2024 at 19:26):if it works, I'm going to drink and tell Guy it's amazing
Ramon Klass (Jan 19 2024 at 19:28):it was a weird feeling the first time I ran my own pyo3+python based example in a browser, it's impressive how far the component model has come :)
Tylr (Nov 22 2024 at 20:22):@Ralph Do you have your example hosted somewhere? I'm also struggling to know the correct implementation for my incoming request handler response object.
Ralph (Nov 22 2024 at 20:23):Oh geez, I'll look for it. But I bet @Guy Bedford or @Tomasz Andrzejak has one newer
Ralph (Nov 22 2024 at 20:24):That was a while back
Tylr (Nov 22 2024 at 20:28):Thanks, I just found it at https://github.com/squillace/hello-wasi-http
Ralph (Nov 22 2024 at 20:29):Rust is easy
Ralph (Nov 22 2024 at 20:29):Javascript is FUN
Ralph (Nov 22 2024 at 20:29):Python is FUN
Ralph (Nov 22 2024 at 20:29):Rust... Boring
Tylr (Nov 22 2024 at 20:29):haha yeah, sorry. I copied wrong tab https://github.com/squillace/hello-wasi-http-js
Ralph (Nov 22 2024 at 20:31):That's old
Tylr (Nov 22 2024 at 20:31):Ill iterate on this and see what I can get
Ralph (Nov 22 2024 at 20:31):Hope it works
Tylr (Nov 22 2024 at 20:31):That's old
do you have a newer example? :D (hope builds)
Ralph (Nov 22 2024 at 20:31):Now a days of ask @Tomasz Andrzejak because he's supposed to teach me how it's done
Ralph (Nov 22 2024 at 20:32):But I think he's asleep now
Ralph (Nov 22 2024 at 20:32):Il send a new one soon or get him to do it
Ralph (Nov 22 2024 at 20:32):But it's Friday night in Europe so I'm buzzed and I'm on my way to drunk
Ralph (Nov 22 2024 at 20:32):Do forgive me
Tylr (Nov 22 2024 at 20:42):enjoy your night! thanks :wave:
Victor Adossi (Nov 25 2024 at 12:03):Hey a bit late, but jco has an examples folder now with some examples that you might found useful:
https://github.com/bytecodealliance/jco/tree/main/examples/components
There absolutely should be a HTTP hello world example in there as well but there isn't (yet), but you can use the one from wasmcloud as a stand-in!
https://github.com/wasmCloud/wasmCloud/tree/main/examples/typescript/components/http-hello-world
Written in TS, but you can obviously remove the transpilation steps :)
No need to use wasmCloud either -- if you reference the package.json you can see the jco build command!
Tylr (Nov 25 2024 at 15:57):Victor thank you so much!
Tylr (Nov 26 2024 at 03:01):Maybe im missing so js knowledge, but where do the object definitions come from? I see in the example you linked the type files referenced, but there is no implementation library included. Are these somehow fulfilled by the wasmCloud runtime? Else im not seeing how it would work as is
Tylr (Nov 26 2024 at 03:20):looks like the preview shim implements them, https://github.com/bytecodealliance/jco/blob/main/packages/preview2-shim/lib/nodejs/http.js#L605
but it isn't in the dependency list of the pkg.json file at https://github.com/wasmCloud/wasmCloud/blob/main/examples/typescript/components/http-hello-world/package.json
Tylr (Nov 26 2024 at 03:39):it looks like jco arg --preview2-adapter <adapter> provide a custom preview2 adapter path hints that it will fulfill the shim for me, but it isnt happening
Victor Adossi (Nov 26 2024 at 06:57):Hey @Tylr sorry a bit late here, but what happens is that jco types will generate a type signature given the WIT interface that is specified/passed in
Victor Adossi (Nov 26 2024 at 06:58):The types that are defined there are a result of building a preview2 component, which adds relevant functionality -- the imports and usage are baked into the binary itself, and when the component is run, they are patched in/used.
Tylr (Nov 26 2024 at 06:59):hmm, I'll keep trying some variations. I get class undefined when using jco serve, or wasmtime serve
Victor Adossi (Nov 26 2024 at 06:59):So for example, if you run wasmtime serve ... with a component, the wasmtime script is responsible for adding the implementations of preview2 APIs.
https://github.com/bytecodealliance/wasmtime/blob/main/src/commands/serve.rs#L316
Victor Adossi (Nov 26 2024 at 06:59):Hmnn that's really odd -- could you post the error/a repo I can poke around in, if it's something you can share?
Tylr (Nov 26 2024 at 07:00):yes, one sec, switching computers
Victor Adossi (Nov 26 2024 at 07:02):Also, doing it in pure JS might be the easiest way to start -- this is something we don't have an example for on the JCO side but it would certainly reduce complexity to start -- would be great if you could contribute that after we're done tinkering
Tylr (Nov 26 2024 at 07:07):i had started with just js, but kept getting errors and the same class undefined
Tylr (Nov 26 2024 at 07:08):the current state; https://github.com/tylerhjones/wasm-lambda
Tylr (Nov 26 2024 at 07:18):i reset back to the same setup you had in the example, but ive broken / missed something still and get an error from wizer.
Victor Adossi (Nov 26 2024 at 07:19):Hey cranking on it right now
Victor Adossi (Nov 26 2024 at 07:19):Will have a PR up to your repo soon that details how to get everything set up!
Tylr (Nov 26 2024 at 07:32):appreciate the help, but ive gotta sleep :smiling_face_with_tear: ill have to check in again in the morning
Victor Adossi (Nov 26 2024 at 07:32):No worries! Will make the PR and put it up to your repo :)
Victor Adossi (Nov 26 2024 at 07:52):Not sure if you're still up but done:
https://github.com/tylerhjones/wasm-lambda/pull/1
Works on my machine :thumbs_up:
Tylr (Nov 26 2024 at 23:42):When i try to componentize the js target I get an error about a duplicate export,
> js_wasm_lambda@1.0.0 build:js
> jco componentize -w wit/ --world-name component -o dist/component.wasm dist/index.js
(jco componentize) ComponentError: failed to encode a component from module
$failed to validate component output
Caused by:
duplicate export name `39` already defined (at offset 0xab351c)
at componentNew (file:///app/node_modules/@bytecodealliance/jco/obj/wasm-tools.js:3618:11)
If I add code to the jco src/api.js componentNew function and have it write the given binary to a file and use wasm-tools after to inspect the binary it received as an argument, I see the following exports.
(export "memory" (memory 0))
(export "wizer.initialize" (func 145))
(export "wizer.resume" (func 146))
(export "cabi_realloc" (func 5062))
(export "wasi:cli/run@0.2.2#run" (func 260))
(export "wasi:http/incoming-handler@0.2.2#handle" (func 261))
(export "cabi_realloc_adapter" (func 5061))
(export "check_init" (func 5068))
(export "wasi:http/incoming-handler@0.2.0#handle" (func $wasi:http/incoming-handler@0.2.0#handle))
(export "cabi_post_wasi:http/incoming-handler@0.2.0#handle" (func $post_wasi:http/incoming-handler@0.2.0#handle))
im guessing the conflicts is from this cabi_post_wasi? Im not sure what else would be in conflict.
Victor Adossi (Nov 27 2024 at 03:47):Are we componentizing with exactly the same code and getting different output? Did you add some code to what the PR had?
Tylr (Nov 27 2024 at 03:51):I'm using what is on master, which again can see is the exact same
Tylr (Nov 27 2024 at 14:37):again* -> afaict
Tylr (Nov 27 2024 at 14:38):I'll check again this morning, maybe I've changed a line accidentally in merge
Victor Adossi (Nov 27 2024 at 15:11):Oh you know what -- what version of node are you using? Could you post your Node and NPM versions?
Victor Adossi (Nov 27 2024 at 15:12):And also if you're running on windows or not -- it shouldn't matter too much but I'm just assuming macbook + some version of Node & NPM
Tylr (Nov 27 2024 at 15:12):23
https://github.com/tylerhjones/wasm-lambda/blob/main/Dockerfile#L2
Victor Adossi (Nov 27 2024 at 15:13):That is your problem! Please try using stable node!
Tylr (Nov 27 2024 at 15:13)::sweat_smile:oh
Victor Adossi (Nov 27 2024 at 15:13):I think I saw this during wasmcon/kubecon just last week -- please try Node 22 -- if you're using nvm then the .nvmrcin there
Tylr (Nov 27 2024 at 15:14):I'll try with 22.
I have nvm, but wanted to keep with docker so I can use the image as a build agent on Jenkins.
Victor Adossi (Nov 27 2024 at 15:14):Don't worry -- this is absolutely a new failure mode, I think this is what I saw just a week or two ago in person and me (and the person who ran into it) were VERY stumped. We were getting weird errors from npm itself about not detecting terminal colors right, and there were odd errors -- even versions of node are stable, so if you wouldn't mind trying that it might just work
Victor Adossi (Nov 27 2024 at 15:14):Ah you can use docker as well, that's totally fine -- as long as you use Node 22 however you'd like!
Victor Adossi (Nov 27 2024 at 15:15):if it solves the issue we might have to do something upstream to try and prevent/control this -- we've had this discussion before and were expecting node to be pretty good about maintaining compat/stuff not breaking underneath us... But if it has, then we've just run into an exception case very quickly :)
Tylr (Nov 27 2024 at 15:16):I'll swap it out and try in abt 20min, not by the computer right now.
Tylr (Nov 27 2024 at 17:02):had some other delays, but I swapped to node 22 and it didn't fix it.
i switched to pnpm instead of using npm, rm'd the node_modules and lock file and it worked.
There is some dependency resolution difference from pnpm and npm, but im not seeing in from the pnpm-lock vs package-lock
Tylr (Nov 27 2024 at 17:03):ill evaluate the two files in more detail later today and see which versions do not match
Tylr (Nov 27 2024 at 17:06):ah,
'@bytecodealliance/jco':
specifier: ^1.7.0
version: 1.8.1
while npm picked
"node_modules/@bytecodealliance/jco": {
"version": "1.7.2",
so for this example, jco must be ^1.8
Victor Adossi (Nov 27 2024 at 17:11):Ah OK, so an old JCO version was the issue? thanks for debugging this!
Tylr (Nov 27 2024 at 17:14):thanks for the example and help.
yes an older jco version is the issue. If I update the pkg json to specify ^1.8
"@bytecodealliance/jco": "^1.8.1",
the example works with npm and node 22 (didnt rebuild image with 23, ill stay with 22 as its the 'active' version https://nodejs.org/en/about/previous-releases)
Thomas Wang (Dec 21 2024 at 08:40):Hello! I'm having trouble using fetch in a wasm component that is invoked in the browser. Have any of y'all been able to do this successfully with the brower wasi-preview2 shim?
Roughly, here's what I have
// world.wit
package jcofetch:hello;
world hello {
export ping: func(s: string) -> string;
}
And my component written in js which I componentize and transpile using jco transpile --map 'wasi-*=@bytecodealliance/preview2-shim/*'.
export async function ping(s) {
const url = new URL("https://jsonplaceholder.typicode.com/posts/1");
const r = await (await fetch(url)).json();
return `hello:${s}, r: ${JSON.stringify(r)}`;
}
My entrypoint index.js:
import { ping } from "../gen/hello/hello.component.js";
console.log(ping("ping"));
Which gets bundled using esbuild and embeded in my index.html with
<script type="module" src="bundle.js"></script>
I get the following error:
bundle.js:992 Uncaught TypeError: Fields is not a constructor
at trampoline7 (bundle.js:992:15)
at hello.component.core.wasm:0x227f6
at hello.component.core.wasm:0x58cdd
at hello.component.core.wasm:0x4dca8
at hello.component.core.wasm:0x253c4c
at hello.component.core.wasm:0x24d4a7
at hello.component.core.wasm:0x24544f
at hello.component.core.wasm:0x253b12
at hello.component.core.wasm:0x25487d
at hello.component.core.wasm:0x2d4bef
I wonder if there's something missing in the wasi preview2-shim for the browser... https://github.com/bytecodealliance/jco/blob/fe90d2ff0af2fc43d42002373e3edb9f892d7f53/packages/preview2-shim/lib/browser/http.js#L35-L45
Any help (or pointers for how I can contribute upstream) would be really appreciated! :pray:
Thomas Wang (Dec 21 2024 at 08:45):FWIW I got a different pure "cowsay" component (no wasi imports) to run in the browser using this bundler setup, so I suspect it's an underlying issue in the shim
Last updated: Dec 23 2024 at 12:05 UTC