Stream: general

Topic: wit-bindgen - rules for deallocating lists?

Peter Vetere (Dec 20 2021 at 22:30):

Hi! I’m hoping to understand a little bit about the design choices around memory deallocation in the bindings that wit-bindgen creates.

For background, I’m working on a WASM UDF implementation for a database. To enable this, I’ve implemented host-side support for the Canonical ABI in C++. The idea is that the user supplies the WIT and WASM files in their CREATE statement, and the database then knows how to communicate with the exports contained therein.

In general, it all works as expected, except when passing lists through the ABI. When the user wants to call a WASM UDF that accepts a list, I use the canonical_abi_realloc export to allocate some of the module’s linear memory to hold the list contents, and store the pointer in the appropriate location expected by the ABI. The WASM code is able to receive the array correctly, but it looks like (sometimes) that the act of passing this memory through the ABI may cause it to be deallocated before it returns back.

Looking at the wit-bindgen code here, I see that there are rules that govern whether or not incoming lists are deallocated. It looks like the decision differs for each binding implementation. In my case (gen-rust-wasm), the decision lies with is_list_canonical, which uses all_bits_valid to make the determination. Ultimately, it looks like it depends on the list’s element type.

In my host code, I need to know whether I should free the memory after the call has been made. I can certainly code to these specific rules, but it seems awkward – and I don’t know whether the WASM module’s ABI was generated for a language that never deallocates (C, for example). Is there some way that I can guarantee that the memory I allocate prior to the function call remains allocated when the call exits (barring any uncontrollable action taken by the WASM code itself)?

Thanks!
Pete

Alex Crichton (Dec 20 2021 at 22:43):

Hm @Peter Vetere ownership shouldn't depend on the type of the list being sent, it should always be uniform. This may indicate a bug? Can you share the *.wit file that you're using?

Peter Vetere (Dec 20 2021 at 23:44):

Hi Alex. I've attached two wit files, and the .rs bindings generated from them. The command I used was like this:

wit-bindgen rust-wasm --export ./dealloc-list.wit

In particular, notice how, in dealloc-list.rs, a dealloc call is added to the test export that frees the memory pointed to by the first argument. I believe this is because the list contains a record with a string type, which fails the all_bits_valid check. Conversely, there is no dealloc generated in no-dealloc-list.rs, even though a list is also at play. In this case, however, the list contains a record with only i32 fields.

dealloc-list.rs
dealloc-list.wit
no-dealloc-list.rs
no-dealloc-list.wit

Scott Waye (Dec 20 2021 at 23:47):

What a good idea. Which database is it?

Alex Crichton (Dec 20 2021 at 23:59):

@Peter Vetere cool thanks! sorry ran out of time today but I'll poke at this tomorrow

Pete Vetere (Dec 21 2021 at 00:03):

@Scott Waye The company I work for is called SingleStore. We’re adding WASM support to the database engine. I agree — it’s a neat idea!

Alex Crichton (Dec 21 2021 at 15:24):

@Pete Vetere oh the behavior you're seeing here is the API in Rust itself, it's not actually affecting the canonical abi itself

Alex Crichton (Dec 21 2021 at 15:24):

the API in Rust is generated differently depending on the types, but the canonical abi has one notion of ownership

Peter Vetere (Dec 21 2021 at 15:53):

Thanks for taking a look. It sounds I was conflating the ABI with the API a bit -- sorry about that. However, the rules for generating the API code this way are still a bit confusing to me. I'd ideally like to tell the user "build your WASM module using the bindings that wit-bindgen creates and we'll be able to consume it as a UDF."

With the current approach, I'm not sure how to tell whether lists that I pass into a WASM function will be deallocated by the generated binding code. For example, the "rust-wasm" generator seems like it will deallocate incoming lists automatically if the elements are of certain types, whereas the generated C bindings look like they will never attempt to dealloc the incoming memory. Am I misunderstanding, or approaching this from the wrong angle?

Alex Crichton (Dec 21 2021 at 15:58):

From a wasm-boundary perspective the asnwer is always fixed

Alex Crichton (Dec 21 2021 at 15:58):

from an API-of-the-language-bindings it's variable based on the type, due to how the natural representation of the type in the language differs from the canonical ABI

Alex Crichton (Dec 21 2021 at 15:58):

but as an interface between the db and the wasm module there is a fixed answer of "lists always allocated" or not, depending on whether the wasm is exporting a function or importing a function

Alex Crichton (Dec 21 2021 at 15:58):

allocation is irrespective of the type of the list

Bailey Hayes (Dec 21 2021 at 18:54):

Scott Waye said:

What a good idea. Which database is it?

(shameless plug :sweat_smile: ) Checkout our Wasm day talk: YouTube - Distributed Computation with WASM and WASI - Bailey Hayes & Carl Sverre, SingleStore

Bailey Hayes (Dec 21 2021 at 19:14):

This seems like a bug in the bindings generated for c vs rust. From what @Peter Vetere is seeing, wasm modules created with the canonical ABI behave differently across the wasm module boundary depending on if it was generated with rust or c.

Wasm modules created from rust (with wit-bindgen) deallocs a string list. If the host attempts to free the list, it will receive an exception.
c wasm modules do not dealloc a string list. The host must free this list or there is a leak. The host shouldn't need to know how a wasm module was created only match the canonical ABI.

Alex Crichton (Dec 21 2021 at 19:23):

The Rust/C APIs generated are indeed different, but that's because the native type representations are different (in C everything matches the canonical ABI). For Rust when an allocation is performed in the generated code it should also be deallocated in the generated code (or in C the API given expects everything to be allocated)

Alex Crichton (Dec 21 2021 at 19:23):

In the wit gisted here so far I don't think there's a bug but I can try to answer more specific questions about specific code if that would help

Bailey Hayes (Dec 21 2021 at 19:32):

This might not be clear, the issue is with API's that are generated and compiled into the wasm module and not the ones generated for the host. So if we're sent a random wasm module and wit file, we don't know when to dealloc because we need to know whether the original program was written in rust or C.

Alex Crichton (Dec 21 2021 at 19:38):

right yeah if that were the case that's bad, but that should not be the case. The ownership semantics are driven by the canonical ABI, no the language-of-wasm

Alex Crichton (Dec 21 2021 at 19:38):

The source of truth of what to deallocate on the host and such and what's owned by wasm is the canonical abi itself

Alex Crichton (Dec 21 2021 at 19:39):

there's not a formal spec right now though so it's mostly "what the code does", but the code should do something that I think is in a PR to the interface-types repo at least

Alex Crichton (Dec 21 2021 at 19:40):

I think what I'm trying to say is that ownership semantics are driven by the *.wit file independent of the language the wasm was written in (or the language of the host). I believe that's true today for all wit-bindgen-generated things, but if you think that it's not I can dig in more

Peter Vetere (Dec 21 2021 at 19:40):

In order for me (as the host) to pass a list into a WASM function, I allocate memory in the guest space (using canonical_abi_realloc), write to it, and pass the guest-relative address in. I can do this consistently regardless of whether the guest is implemented in C or Rust because the canonical_abi_realloc is exported by the bindgen regardless. The problem occurs when the function returns back to the host -- I don't know whether or not to deallocate the memory I passed in (using canonical_abi_free). To help illustrate, I've attached a wit file and both the generated Rust and C bindings. Notice how the C code does not generate a free() call for the "test" routine right now, but the Rust code does.

foo.c
foo.rs
foo.wit

Alex Crichton (Dec 21 2021 at 19:49):

it needs deallocation in both cases

Alex Crichton (Dec 21 2021 at 19:49):

it's just implicit in C that you must do that

Alex Crichton (Dec 21 2021 at 19:49):

and it's automatically done for you in Rust

Alex Crichton (Dec 21 2021 at 19:49):

if C doesn't free it then it's a memory leak

Peter Vetere (Dec 21 2021 at 20:11):

Ok, so just to verify: Authors wanting to create WASM modules compatible with the canonical ABI must free any and all lists they receive from the host before returning from their function. Is that correct?

Peter Vetere (Dec 21 2021 at 20:11):

(or ensure that they are freed)

Alex Crichton (Dec 21 2021 at 20:12):

correct yeah

Peter Vetere (Dec 21 2021 at 20:18):

Ok. That's very helpful. The rule limits the amount of optimization the host can do around memory allocation (i.e. the host wouldn't be able to allocate one large block that is the sum total of all lists and then sub-divide it as separate pointers), but on the other hand it implies that if the guest wants faster memory allocation, it can provide its own optimizations to this effect.

Alex Crichton (Dec 21 2021 at 20:22):

indeed yeah, this is also part of the canonical abi itself where in the far future wasm modules can customize it all they want

Peter Vetere (Dec 21 2021 at 20:22):

Thank you so much for helping me understand, Alex. Appreciate it.

noonien (Dec 22 2021 at 22:22):

hello folks!
i was wondering if it's possible to return arbitrarily sized opaque types from within wasm, to store and be able to pass back into wasm functions.

noonien (Dec 22 2021 at 22:24):

i'd have a WASM function that returns a value of that type, with an initial state, i'd probably need to store that function as a Vec<u8> (can't be an array since i don't know it's value beforehand), and i'd pass it back as a &mut [u8]

noonien (Dec 22 2021 at 22:25):

i'd prefer to not do sizeof in wasm, but that's completely ok if there's no other option available

Dan Gohman (Dec 22 2021 at 22:26):

When you say "opaque", do mean opaque to the outside world?

noonien (Dec 22 2021 at 22:26):

yes, i don't care what's in it, i just need to be able to clone it

Dan Gohman (Dec 22 2021 at 22:27):

The outside world needs to be able to clone it?

noonien (Dec 22 2021 at 22:27):

yes

Dan Gohman (Dec 22 2021 at 22:28):

One option would be to keep the bytes inside the wasm, and return a handle representing those bytes. To clone them, you could export a clone function which takes a handle and returns a new handle.

Dan Gohman (Dec 22 2021 at 22:28):

Another option would be to simply return a list<u8>, which can have a dynamic size.

Dan Gohman (Dec 22 2021 at 22:29):

Though you can't get a list<u8> passed back in as a &mut [u8], so if you want updates like that, the handle approach tends to be better

noonien (Dec 22 2021 at 22:29):

pub trait Script {
    type State: Clone;

    fn init(&mut self) -> Self::State;
    fn tick(&mut self, state: &mut Self::State, candle: &Candle, history: &History<Self::State>);
}

i'm basically trying to implement this trait using WASM

noonien (Dec 22 2021 at 22:29):

i'm talking about the state parameter

Dan Gohman (Dec 22 2021 at 22:30):

My first idea would be to use a resource, and return and pass handles.

noonien (Dec 22 2021 at 22:32):

a resource? so basically a memory region that WASM controlls, and gives me access to?

noonien (Dec 22 2021 at 22:32):

thing is, i'd also need to be able to store it, and provide its previous values through the History parameters

noonien (Dec 22 2021 at 22:33):

that's why i'd need to clone it

Dan Gohman (Dec 22 2021 at 22:33):

Cloning can be done with a clone function

noonien (Dec 22 2021 at 22:34):

and that would mean all cloned values would need to be stored in WASM?

Dan Gohman (Dec 22 2021 at 22:34):

It depends on which way the API goes. The outside world is the one calling tick?

noonien (Dec 22 2021 at 22:35):

yes

Dan Gohman (Dec 22 2021 at 22:35):

Then I think what you'd really want here is for the outside world to define the resource.

Dan Gohman (Dec 22 2021 at 22:36):

But I expect that will hit a present limitation: https://github.com/bytecodealliance/wit-bindgen/issues/120

noonien (Dec 22 2021 at 22:40):

i would prefer not to require access to the WASM file during compile time

noonien (Dec 22 2021 at 22:41):

it appears that's necessary for what you linked

Dan Gohman (Dec 22 2021 at 22:42):

I'm not sure what you mean by access to the wasm

noonien (Dec 22 2021 at 22:43):

i load the WASM at my programs runtime, not during compile time

noonien (Dec 22 2021 at 22:44):

so, i'm not sure wit_bindgen_rust is usable in this case

Dan Gohman (Dec 22 2021 at 22:45):

I'm still not clear on what you're referring to. wit_bindgen_rust is fine with modules being loaded at runtime

noonien (Dec 22 2021 at 22:45):

sorry, i'm new to WASM and wasmtime, i'm not sure I understand everything just yet

Dan Gohman (Dec 22 2021 at 22:46):

The main thing going on in #120 is that there are two wit files, rather than one

noonien (Dec 22 2021 at 22:46):

Dan Gohman said:

I'm still not clear on what you're referring to. wit_bindgen_rust is fine with modules being loaded at runtime

yes, but does that not require access to the wasm file during my programs compilation?

Dan Gohman (Dec 22 2021 at 22:47):

Ah, no, that's what the wit IDL does. Your program gets everything it needs to know from wit file; it doesn't need to see the actual wasm

noonien (Dec 22 2021 at 22:47):

ah, ok, still, each WASM script has a different state type

noonien (Dec 22 2021 at 22:48):

there is no commonality there

noonien (Dec 22 2021 at 22:48):

and, from what i can understand, wit files are like an interface to define common things (?), in this case, probably a resource for my type, or the type itself

Dan Gohman (Dec 22 2021 at 22:53):

Interface types isn't really built to represent this kind of polymorphism directly.

Dan Gohman (Dec 22 2021 at 22:54):

So there are no variable-size types that would be needed to model this directly.

Dan Gohman (Dec 22 2021 at 22:55):

My suggestion to use a handle would involve adding a layer of indirection. Instead of literally returning the State value, there'd be a way to create state values, and you could use handles to reference them in opaque ways.

Dan Gohman (Dec 22 2021 at 22:58):

Handles would also allow the state to be mutated.

noonien (Dec 22 2021 at 23:01):

if i'd implement this in C, id would probably be something along these lines:

struct InitState {
  size_t size;
  uint8_t *data;
};

struct State {
  int foo;
}

struct InitState init() {
  struct State state;
  state.foo = 42;

  struct InitState res;
  res.size = sizeof state;
  res.data = malloc(sizeof state);
  memcpy(res.data, &state, sizeof state);

  return res;
};

struct Candle {};
struct History{
  const struct State *states;
};

void tick(State *state, const struct Candle *candle, const struct History *history) {
  state.foo += 1;
}

noonien (Dec 22 2021 at 23:01):

that C is probably broken, it's just there as an example

noonien (Dec 22 2021 at 23:05):

i also don't mind receiving State as an *uint8_t in tick

noonien (Dec 22 2021 at 23:09):

or, maybe this:

struct State {
  int foo;
};

size_t state_size() { return sizeof (struct State); }

void init(void *buf) {
  struct State *state = buf;
  state->foo = 42;
};

struct Candle {};
struct History {
  const void **state_bufs;
};

void tick(void *state_buf, const struct Candle *candle, const struct History *history) {
  struct State *state = state_buf;
  state->foo += 1;
}

noonien (Dec 22 2021 at 23:13):

compiling on https://webassembly.studio/: that code seems to generate the following .wat:

(module
  (type $t0 (func))
  (type $t1 (func (result i32)))
  (type $t2 (func (param i32)))
  (type $t3 (func (param i32 i32 i32)))
  (func $__wasm_call_ctors (type $t0))
  (func $state_size (export "state_size") (type $t1) (result i32)
    i32.const 4)
  (func $init (export "init") (type $t2) (param $p0 i32)
    get_local $p0
    i32.const 42
    i32.store)
  (func $tick (export "tick") (type $t3) (param $p0 i32) (param $p1 i32) (param $p2 i32)
    get_local $p0
    get_local $p0
    i32.load
    i32.const 1
    i32.add
    i32.store)
  (table $T0 1 1 anyfunc)
  (memory $memory (export "memory") 2)
  (global $g0 (mut i32) (i32.const 66560))
  (global $__heap_base (export "__heap_base") i32 (i32.const 66560))
  (global $__data_end (export "__data_end") i32 (i32.const 1024)))

Dan Gohman (Dec 22 2021 at 23:15):

If the outside world is calling tick here, then this looks like the State resource should be defined on the wasm side.

noonien (Dec 22 2021 at 23:15):

yes

Dan Gohman (Dec 22 2021 at 23:15):

noonien (Dec 22 2021 at 23:15):

the problem is that History also contains a history of previous States from each tick

Dan Gohman (Dec 22 2021 at 23:15):

oops

Dan Gohman (Dec 22 2021 at 23:16):

Dan Gohman (Dec 22 2021 at 23:19):

resource state {
    static init: function() -> handle<state>
    tick: function(candle: candle, history: handle<state>)
    clone: function() -> handle<state>
}

noonien (Dec 22 2021 at 23:21):

since history should be a list of previous states, should it not be list<handle<state>>?

Dan Gohman (Dec 22 2021 at 23:21):

yeah

noonien (Dec 22 2021 at 23:25):

i see. that sounds good enough. however, in case i'd need to store/restore the states, is there a way of doing that?

Dan Gohman (Dec 22 2021 at 23:26):

serialize: function() -> list<u8>
static  deserialize: function(bytes: list<u8>) -> handle<state>

would be one approach.

noonien (Dec 22 2021 at 23:26):

awesome!

noonien (Dec 22 2021 at 23:27):

what's a resource? i'm trying to find more documentation, but searching for "wasm resource" doesn't yield very good results

Dan Gohman (Dec 22 2021 at 23:28):

There's unfortunately not a lot of docs right now. This will eventually be answered by the interface-types docs. But for now:

Dan Gohman (Dec 22 2021 at 23:28):

A resource is just a type for opaque objects that can be pointed to by handle types.

noonien (Dec 22 2021 at 23:29):

are there code examples that use resources with wasmtime (preferably with rust)?

Dan Gohman (Dec 22 2021 at 23:30):

The wasi-filesystem API rewritten into wit has a resource: https://github.com/WebAssembly/wasi-filesystem/blob/main/wasi-filesystem.wit.md

Dan Gohman (Dec 22 2021 at 23:31):

descriptor is a resource which one can think of as "the thing referenced by a file descriptor", with handles being the actual file descriptors

noonien (Dec 22 2021 at 23:36):

hmm, is there actual code?

Dan Gohman (Dec 22 2021 at 23:39):

The rust-wasm side is in-progress here: https://github.com/bytecodealliance/rustix/tree/wasi/src/imp/wasi

noonien (Dec 22 2021 at 23:41):

hmm, from what i can tell, resources are basically just something that one can get a handle<T> to, and handle<T> is basically a pointer to T

Dan Gohman (Dec 22 2021 at 23:43):

That's the gist of it. handle is a smart pointer, so it won't dangle, and it promptly deallocates the resource when dropped.

noonien (Dec 22 2021 at 23:51):

i think i understand now why i'd need the WASM code to do its own allocation

noonien (Dec 22 2021 at 23:53):

from what i can tell, if i don't do this, i'd have to make sure that if i allocate memory outside of the wasm code, it doesn't interfere with what WASM does with the memory (since it doesn't know there is anything allocated)

Dan Gohman (Dec 23 2021 at 00:01):

Yeah. With wit-bindgen and interface-types in general, code on the outside of a wasm component doesn't have access to the linear-memory address space on the inside.

noonien (Dec 23 2021 at 00:13):

so, from what I can tell, i basically need

resource state {
    // create an initial state
    static init: function() -> handle<state>

    // clone a state, so we can provide tick with a previous state
    clone: function(state: handle<state>) -> handle<state>

    // serialize state for storage
    serialize: function() -> list<u8>
    static deserialize: function(bytes: list<u8>) -> handle<state>

    // tick has access to a mutable state (usually a clone of the last state in history)
    // after the tick is done, the modified state might get stored in the history
    tick: function(state: handle<state>, candle: candle, history: list<handle<state>>)
}

noonien (Dec 23 2021 at 00:13):

then, i use wit-bindgen in my rust->wasm program to provide those functions

noonien (Dec 23 2021 at 00:14):

however, in my host rust program, where wasmtime is run, i'm guessing handle<state> is just an i32

Dan Gohman (Dec 23 2021 at 00:15):

Yeah. The canonical ABI manages tables of handles, and provides Rust code with i32 indices into those tables.

noonien (Dec 23 2021 at 00:15):

awesome

noonien (Dec 23 2021 at 00:16):

also, i'm guessing since i don't have multiple wit modules, i won't hit the issue you linked

Dan Gohman (Dec 23 2021 at 00:17):

Yeah, with the wasm module defining the resource, you should be good

noonien (Dec 23 2021 at 00:19):

will this work?

resource state {
    // clone a state, so we can provide tick with a previous state
    clone: function(state: handle<state>) -> handle<state>

    // serialize state for storage
    serialize: function() -> list<u8>
    static deserialize: function(bytes: list<u8>) -> handle<state>

}

// create an initial state
static init: function() -> handle<state>

// tick has access to a mutable state (usually a clone of the last state in history)
// after the tick is done, the modified state might get stored in the history
tick: function(state: handle<state>, candle: candle, history: list<handle<state>>)

noonien (Dec 23 2021 at 00:20):

also, i can't seem to be able to use https://bytecodealliance.github.io/wit-bindgen/ with handle<state>

noonien (Dec 23 2021 at 00:21):

ah, i'm probably misusing that page

noonien (Dec 23 2021 at 00:23):

hmm, or not? i'm not exactly sure

Dan Gohman (Dec 23 2021 at 00:24):

I'm not sure offhand how up to date that page is. wit-bindgen has been changing pretty rapidly

Dan Gohman (Dec 23 2021 at 00:25):

Hmm, I was mistaken. It looks like it's current.

Dan Gohman (Dec 23 2021 at 00:29):

Ah, right. It's handle state rather than handle<state>

Dan Gohman (Dec 23 2021 at 00:30):

Also, the static init function needs to be declared inside the resource

noonien (Dec 23 2021 at 00:30):

can I ask why that is?

Dan Gohman (Dec 23 2021 at 00:31):

I think you could define it outside too, but then you'd omit the static keyword

Dan Gohman (Dec 23 2021 at 00:31):

static means "don't add a self parameter"

noonien (Dec 23 2021 at 00:31):

ah, i see, that makes sense!

noonien (Dec 23 2021 at 00:45):

can *.wit files have parameterized types (generics)?

Dan Gohman (Dec 23 2021 at 00:50):

no

Dan Gohman (Dec 23 2021 at 00:51):

Some builtin types are parameterized, like list<T>, but no user-defined types are.

noonien (Dec 23 2021 at 00:51):

i see. are there methods?

Dan Gohman (Dec 23 2021 at 00:51):

Functions defined in resources are effectively methods

noonien (Dec 23 2021 at 00:52):

other than that, "methods" are just prefixed functions i'm guessing

noonien (Dec 23 2021 at 00:53):

where the first parameter is the receiver

Dan Gohman (Dec 23 2021 at 00:54):

Yeah. In some langauge bindings, such as JS, having the concept of a receiver is useful to make the code more ergonomic.

noonien (Dec 23 2021 at 01:01):

thanks for the help @Dan Gohman! i would have been aimlessly lost without it! i really appreciate it

Dan Gohman (Dec 23 2021 at 01:02):

You're welcome!

Peter Vetere (Jan 07 2022 at 16:17):

Peter Vetere said:

Ok, so just to verify: Authors wanting to create WASM modules compatible with the canonical ABI must free any and all lists they receive from the host before returning from their function. Is that correct?

@Alex Crichton Just a quick follow-up on this: when a Wasm function returns a list result to the host, is the host then responsible for calling canonical_abi_free on the linear memory offset it receives? Or is that memory lifetime also assumed to be managed by the guest?

Alex Crichton (Jan 07 2022 at 16:26):

@Peter Vetere correct yeah, the host takes "ownership" of the memory and must free it

Peter Vetere (Jan 07 2022 at 16:26):

Excellent. Thank you!

Alex Crichton (Jan 07 2022 at 16:26):

(this is where we need to more thoroughly document the canonical ABI)

Last updated: Nov 22 2024 at 16:03 UTC