Build-up of a wasm component · wasi

This discussion has already been initiated a few days ago, but sadly without good faith and proper respect. I hope I can reopen the same discussion but with proper respect as I also have the interest in better understanding the details of what makes a component tick.

(component
  (core module $app-module
    (func (export "core-run") (result i32)
      i32.const 0
    )
  )
  (core instance $app-instance (instantiate $app-module))

  (func $lifted-run (result (result)) (canon lift (core func $app-instance "core-run")))

  (component $wrapper
    (import "imported-run" (func $wrapper-run (result (result))))
    (export "run" (func $wrapper-run))
  )
  (instance (export "wasi:cli/run@0.2.0") (instantiate $wrapper
    (with "imported-run" (func $lifted-run))
  ))
)

Then the next two I have a harder time understanding. The component and the instance.
For this component to correctly export the wasi:cli/run@0.2.0 world, it needs to export an instance.
And to create an instance I need a component definition, which for a wasi:cli/run@0.2.0 component only needs to export run: func() -> result.

But to get the lifted run function out as the expected wasi:cli/run@0.2.0 run function I have to do the mapping I do in the component definition and in the instantiation binding? Is this correct? Or is there a simpler way to get the lifted run out as the wasi:cli/run@0.2.0 expected run?

In the case of a wasi:cli/run@0.2.0 component do I always need to have a component definition and a instance definition as in my example?

IFcoltransG (Sep 16 2024 at 10:34):

Your explanation for why the component and instance are needed seems right to me — targeting wasi:cli/run@0.2.0 means exporting a component instance, hence why you declare an inner component exporting a run then an outer component which exports an instance of the inner one under the run@0.2.0 name.
If it strikes you as simpler, you may be able to move everything before $wrapper (the core module, the core instance and the component function) into $wrapper, to avoid needing the function import. That would also allow you to directly export the component function inside its declaration.

Soni L. (Sep 16 2024 at 11:53):

the drawback of moving everything into $wrapper is that if you wanna import anything... well, imported component instances are typed, and types aren't shared between components as far as we know, so you'd need to declare them twice: once to import into the outer component, and once again to import into $wrapper

Soni L. (Sep 16 2024 at 11:54):

Soni L. (Sep 16 2024 at 11:56):

arguably the real problem is that the WASI command world exports a WASI run interface instead of a plain run function... but anyway

Alex Crichton (Sep 16 2024 at 15:03):

The instance export here is required by wasi:cli/run@0.2.0 indeed, but instantiating a component is not the only way of creating a component instance in the component model. There's also the ability to create a component instance from a "bag of exports" where you basically bundled up items within a component as named exports, creating an instance which can itself then be exported. For example another way of creating the above component would look like this:

(component
  (core module $app-module
    (func (export "core-run") (result i32)
      i32.const 0
    )
  )
  (core instance $app-instance (instantiate $app-module))

  (func $lifted-run (result (result)) (canon lift (core func $app-instance "core-run")))

  (instance $wrapper
    (export "run" (func $lifted-run)))
  (export "wasi:cli/run@0.2.0" (instance $wrapper))
)

This is probably simpler and more along the lines of what you might expect for "hello world" style use cases. The component you pasted and this one here are basically equivalent and are different ways to have the same structure. Within Wasmtime they end up compiling to basically the same result too (and the same in jco I believe).

So that might raise the question: why bother having the component $wrapper in the first place? The reason for that is subtle unfortunately and has to do with resources in the component model. Resources are "more advanced" sort of in the sense of how all the typing rules work. The wasm-tool component new command used to use (instance (export "..." ...)) syntactical form and only switch to using nested components when resources were introduced. The precise reason for this has to do with validation of resources and how typing them in the component model works which is probably out of scope of this discussion.

Regardless it's definitely possible to omit component $wrapper, but for wasm-tools component new it's easier to have one path that always emits a component. There's no runtime overhead associated with it and so far there hasn't been motivation to otherwise change it. Implementing such an optimization would be reasonable, however.

If that leaves any lingering questions though or doesn't answer anything please let me know!

Dax Huiberts (Sep 17 2024 at 10:47):

Thank you IFcoltransG and Alex Crichton both for the clear explanations. This really helps solidify a better understanding of the building blocks of a component model module. Using the (instance (export ...)) approach gives a simpler/easier understanding of the bare essentials and helps me build up on that.

Soni L. (Sep 17 2024 at 16:08):

component model is a cursed typed scripting language for wasm modules, basically it's just RPC. it also doesn't align with how wasm engines actually work (most wasm engines aren't as strictly typed as CM is) so there's a huge impedance mismatch for implementers (thankfully components are self-contained, there are no type imports or generics or anything you need to explicitly declare, in excruciating detail, every aspect of the types you wanna use, so you can just do type checking as part of validation)

IFcoltransG (Sep 17 2024 at 20:25):

Dax Huiberts (Sep 18 2024 at 11:01):

With this new knowledge I've been able to strip down a version of rust "hello world" to the following which is as simple as I could get it.

Is this as simple as this could get? Or are there some more reductions possible?

(component
  (instance $error (import "wasi:io/error@0.2.0")
    (export "error" (type (sub resource)))
  )
  (alias export $error "error" (type $error))

  (instance $streams (import "wasi:io/streams@0.2.0")
    (alias outer 1 $error (type $error-alias))
    (export $output-stream "output-stream" (type (sub resource)))
    (export $error "error" (type (eq $error-alias)))
    (type $variant (variant (case "last-operation-failed" (own $error)) (case "closed")))
    (export $stream-error "stream-error" (type (eq $variant)))
    (export "[method]output-stream.blocking-write-and-flush" (func (param "self" (borrow $output-stream)) (param "contents" (list u8)) (result (result (error $stream-error)))))
  )
  (alias export $streams "output-stream" (type $output-stream))

  (instance $stdout (import "wasi:cli/stdout@0.2.0")
    (alias outer 1 $output-stream (type $output-stream-alias))
    (export $output-stream "output-stream" (type (eq $output-stream-alias)))
    (export "get-stdout" (func (result (own $output-stream))))
  )

  (core module $memory
    (memory (export "memory") 1)
  )
  (core instance $memory (instantiate $memory))
  (alias core export $memory "memory" (core memory $memory))

  (core func $stdout (canon lower (func $stdout "get-stdout")))
  (core func $write (canon lower (func $streams "[method]output-stream.blocking-write-and-flush") (memory $memory)))

  (core module $app
    (memory (import "memory" "memory") 1)
    (data (i32.const 4) "Hello world!\n")
    (func $stdout (import "wasi" "stdout") (result i32))
    (func $write (import "wasi" "write") (param i32 i32 i32 i32))
    (func (export "main") (result i32)
      (call $write
        call $stdout ;; puts stdout handle on stack
        i32.const 4 ;; memory address of string
        i32.const 13 ;; length of string
        i32.const 0 ;; memory address to where return value should be placed
      )
      (i32.load (i32.const 0)) ;; gets the result enum type at return address 0, 0 = success, 1 = error
    )
  )

  (core instance $app (instantiate $app
    (with "wasi" (instance
      (export "stdout" (func $stdout))
      (export "write" (func $write))
    ))
    (with "memory" (instance $memory))
  ))

  (func $main (result (result)) (canon lift (core func $app "main")))

  (instance (export "wasi:cli/run@0.2.0")
    (export "run" (func $main))
  )
)

Dax Huiberts (Sep 18 2024 at 11:13):

Especially the parts with the imports I'm not sure if they can be simplified further. Are the alias exports the only way to reference the types in the other imports.

Is the wasi:io/error@0.2.0 import mandatory because of the need of the error resource?

Is the need to split the core module so I can first instantiate the memory, which is needed by the lowering definitions, which can then be used in the actual core module?

Alex Crichton (Sep 18 2024 at 14:55):

Yeah that's a pretty minimal module and I'm not sure how to make it significantly smaller. You could use some text format shorthands perhaps such as omitting (alias outer ...) and just using the outer name (which is automatically turned into an alias), but beyond that you've got most bits.

There's more than one way to go about this, for example components produced from Rust/C/etc typically don't do this split since they're exporting memory from the main module. What you've done here though works just fine and is defnitely a way to resolve the problem of "canon lower needs access to a memory to lower into"

Victor Adossi (Oct 01 2024 at 03:02):

Does this warrant including in the docs somewhere? An annotated/commented version of this might be a fantastic introduction to understanding component WAT, and do the heavy lifting in explaining many concepts of the enhancements made possible by the CM

Stream: wasi

Topic: Build-up of a wasm component

Dax Huiberts (Sep 16 2024 at 10:01):