Hi! I’m hoping to understand a little bit about the design choices around memory deallocation in the bindings that wit-bindgen creates.
For background, I’m working on a WASM UDF implementation for a database. To enable this, I’ve implemented host-side support for the Canonical ABI in C++. The idea is that the user supplies the WIT and WASM files in their CREATE
statement, and the database then knows how to communicate with the exports contained therein.
In general, it all works as expected, except when passing lists through the ABI. When the user wants to call a WASM UDF that accepts a list, I use the canonical_abi_realloc
export to allocate some of the module’s linear memory to hold the list contents, and store the pointer in the appropriate location expected by the ABI. The WASM code is able to receive the array correctly, but it looks like (sometimes) that the act of passing this memory through the ABI may cause it to be deallocated before it returns back.
Looking at the wit-bindgen code here, I see that there are rules that govern whether or not incoming lists are deallocated. It looks like the decision differs for each binding implementation. In my case (gen-rust-wasm), the decision lies with is_list_canonical
, which uses all_bits_valid
to make the determination. Ultimately, it looks like it depends on the list’s element type.
In my host code, I need to know whether I should free the memory after the call has been made. I can certainly code to these specific rules, but it seems awkward – and I don’t know whether the WASM module’s ABI was generated for a language that never deallocates (C, for example). Is there some way that I can guarantee that the memory I allocate prior to the function call remains allocated when the call exits (barring any uncontrollable action taken by the WASM code itself)?
Thanks!
Pete
Hm @Peter Vetere ownership shouldn't depend on the type of the list being sent, it should always be uniform. This may indicate a bug? Can you share the *.wit
file that you're using?
Hi Alex. I've attached two wit files, and the .rs bindings generated from them. The command I used was like this:
wit-bindgen rust-wasm --export ./dealloc-list.wit
In particular, notice how, in dealloc-list.rs, a dealloc
call is added to the test
export that frees the memory pointed to by the first argument. I believe this is because the list contains a record with a string type, which fails the all_bits_valid
check. Conversely, there is no dealloc
generated in no-dealloc-list.rs, even though a list is also at play. In this case, however, the list contains a record with only i32 fields.
dealloc-list.rs
dealloc-list.wit
no-dealloc-list.rs
no-dealloc-list.wit
What a good idea. Which database is it?
@Peter Vetere cool thanks! sorry ran out of time today but I'll poke at this tomorrow
@Scott Waye The company I work for is called SingleStore. We’re adding WASM support to the database engine. I agree — it’s a neat idea!
@Pete Vetere oh the behavior you're seeing here is the API in Rust itself, it's not actually affecting the canonical abi itself
the API in Rust is generated differently depending on the types, but the canonical abi has one notion of ownership
Thanks for taking a look. It sounds I was conflating the ABI with the API a bit -- sorry about that. However, the rules for generating the API code this way are still a bit confusing to me. I'd ideally like to tell the user "build your WASM module using the bindings that wit-bindgen creates and we'll be able to consume it as a UDF."
With the current approach, I'm not sure how to tell whether lists that I pass into a WASM function will be deallocated by the generated binding code. For example, the "rust-wasm" generator seems like it will deallocate incoming lists automatically if the elements are of certain types, whereas the generated C bindings look like they will never attempt to dealloc the incoming memory. Am I misunderstanding, or approaching this from the wrong angle?
From a wasm-boundary perspective the asnwer is always fixed
from an API-of-the-language-bindings it's variable based on the type, due to how the natural representation of the type in the language differs from the canonical ABI
but as an interface between the db and the wasm module there is a fixed answer of "lists always allocated" or not, depending on whether the wasm is exporting a function or importing a function
allocation is irrespective of the type of the list
Scott Waye said:
What a good idea. Which database is it?
(shameless plug :sweat_smile: ) Checkout our Wasm day talk: YouTube - Distributed Computation with WASM and WASI - Bailey Hayes & Carl Sverre, SingleStore
This seems like a bug in the bindings generated for c vs rust. From what @Peter Vetere is seeing, wasm modules created with the canonical ABI behave differently across the wasm module boundary depending on if it was generated with rust or c.
Wasm modules created from rust (with wit-bindgen) deallocs a string list. If the host attempts to free the list, it will receive an exception.
c wasm modules do not dealloc a string list. The host must free this list or there is a leak. The host shouldn't need to know how a wasm module was created only match the canonical ABI.
The Rust/C APIs generated are indeed different, but that's because the native type representations are different (in C everything matches the canonical ABI). For Rust when an allocation is performed in the generated code it should also be deallocated in the generated code (or in C the API given expects everything to be allocated)
In the wit gisted here so far I don't think there's a bug but I can try to answer more specific questions about specific code if that would help
This might not be clear, the issue is with API's that are generated and compiled into the wasm module and not the ones generated for the host. So if we're sent a random wasm module and wit file, we don't know when to dealloc because we need to know whether the original program was written in rust or C.
right yeah if that were the case that's bad, but that should not be the case. The ownership semantics are driven by the canonical ABI, no the language-of-wasm
The source of truth of what to deallocate on the host and such and what's owned by wasm is the canonical abi itself
there's not a formal spec right now though so it's mostly "what the code does", but the code should do something that I think is in a PR to the interface-types repo at least
I think what I'm trying to say is that ownership semantics are driven by the *.wit
file independent of the language the wasm was written in (or the language of the host). I believe that's true today for all wit-bindgen
-generated things, but if you think that it's not I can dig in more
In order for me (as the host) to pass a list into a WASM function, I allocate memory in the guest space (using canonical_abi_realloc), write to it, and pass the guest-relative address in. I can do this consistently regardless of whether the guest is implemented in C or Rust because the canonical_abi_realloc is exported by the bindgen regardless. The problem occurs when the function returns back to the host -- I don't know whether or not to deallocate the memory I passed in (using canonical_abi_free). To help illustrate, I've attached a wit file and both the generated Rust and C bindings. Notice how the C code does not generate a free() call for the "test" routine right now, but the Rust code does.
it needs deallocation in both cases
it's just implicit in C that you must do that
and it's automatically done for you in Rust
if C doesn't free it then it's a memory leak
Ok, so just to verify: Authors wanting to create WASM modules compatible with the canonical ABI must free any and all lists they receive from the host before returning from their function. Is that correct?
(or ensure that they are freed)
correct yeah
Ok. That's very helpful. The rule limits the amount of optimization the host can do around memory allocation (i.e. the host wouldn't be able to allocate one large block that is the sum total of all lists and then sub-divide it as separate pointers), but on the other hand it implies that if the guest wants faster memory allocation, it can provide its own optimizations to this effect.
indeed yeah, this is also part of the canonical abi itself where in the far future wasm modules can customize it all they want
Thank you so much for helping me understand, Alex. Appreciate it.
hello folks!
i was wondering if it's possible to return arbitrarily sized opaque types from within wasm, to store and be able to pass back into wasm functions.
i'd have a WASM function that returns a value of that type, with an initial state, i'd probably need to store that function as a Vec<u8> (can't be an array since i don't know it's value beforehand), and i'd pass it back as a &mut [u8]
i'd prefer to not do sizeof in wasm, but that's completely ok if there's no other option available
When you say "opaque", do mean opaque to the outside world?
yes, i don't care what's in it, i just need to be able to clone it
The outside world needs to be able to clone it?
yes
One option would be to keep the bytes inside the wasm, and return a handle representing those bytes. To clone them, you could export a clone
function which takes a handle and returns a new handle.
Another option would be to simply return a list<u8>
, which can have a dynamic size.
Though you can't get a list<u8>
passed back in as a &mut [u8]
, so if you want updates like that, the handle approach tends to be better
pub trait Script {
type State: Clone;
fn init(&mut self) -> Self::State;
fn tick(&mut self, state: &mut Self::State, candle: &Candle, history: &History<Self::State>);
}
i'm basically trying to implement this trait using WASM
i'm talking about the state parameter
My first idea would be to use a resource, and return and pass handles.
a resource? so basically a memory region that WASM controlls, and gives me access to?
thing is, i'd also need to be able to store it, and provide its previous values through the History parameters
that's why i'd need to clone it
Cloning can be done with a clone
function
and that would mean all cloned values would need to be stored in WASM?
It depends on which way the API goes. The outside world is the one calling tick
?
yes
Then I think what you'd really want here is for the outside world to define the resource.
But I expect that will hit a present limitation: https://github.com/bytecodealliance/wit-bindgen/issues/120
i would prefer not to require access to the WASM file during compile time
it appears that's necessary for what you linked
I'm not sure what you mean by access to the wasm
i load the WASM at my programs runtime, not during compile time
so, i'm not sure wit_bindgen_rust is usable in this case
I'm still not clear on what you're referring to. wit_bindgen_rust is fine with modules being loaded at runtime
sorry, i'm new to WASM and wasmtime, i'm not sure I understand everything just yet
The main thing going on in #120 is that there are two wit files, rather than one
Dan Gohman said:
I'm still not clear on what you're referring to. wit_bindgen_rust is fine with modules being loaded at runtime
yes, but does that not require access to the wasm file during my programs compilation?
Ah, no, that's what the wit IDL does. Your program gets everything it needs to know from wit file; it doesn't need to see the actual wasm
ah, ok, still, each WASM script has a different state type
there is no commonality there
and, from what i can understand, wit files are like an interface to define common things (?), in this case, probably a resource for my type, or the type itself
Interface types isn't really built to represent this kind of polymorphism directly.
So there are no variable-size types that would be needed to model this directly.
My suggestion to use a handle would involve adding a layer of indirection. Instead of literally returning the State
value, there'd be a way to create state values, and you could use handles to reference them in opaque ways.
Handles would also allow the state to be mutated.
if i'd implement this in C, id would probably be something along these lines:
struct InitState {
size_t size;
uint8_t *data;
};
struct State {
int foo;
}
struct InitState init() {
struct State state;
state.foo = 42;
struct InitState res;
res.size = sizeof state;
res.data = malloc(sizeof state);
memcpy(res.data, &state, sizeof state);
return res;
};
struct Candle {};
struct History{
const struct State *states;
};
void tick(State *state, const struct Candle *candle, const struct History *history) {
state.foo += 1;
}
that C is probably broken, it's just there as an example
i also don't mind receiving State as an *uint8_t in tick
or, maybe this:
struct State {
int foo;
};
size_t state_size() { return sizeof (struct State); }
void init(void *buf) {
struct State *state = buf;
state->foo = 42;
};
struct Candle {};
struct History {
const void **state_bufs;
};
void tick(void *state_buf, const struct Candle *candle, const struct History *history) {
struct State *state = state_buf;
state->foo += 1;
}
compiling on https://webassembly.studio/: that code seems to generate the following .wat:
(module
(type $t0 (func))
(type $t1 (func (result i32)))
(type $t2 (func (param i32)))
(type $t3 (func (param i32 i32 i32)))
(func $__wasm_call_ctors (type $t0))
(func $state_size (export "state_size") (type $t1) (result i32)
i32.const 4)
(func $init (export "init") (type $t2) (param $p0 i32)
get_local $p0
i32.const 42
i32.store)
(func $tick (export "tick") (type $t3) (param $p0 i32) (param $p1 i32) (param $p2 i32)
get_local $p0
get_local $p0
i32.load
i32.const 1
i32.add
i32.store)
(table $T0 1 1 anyfunc)
(memory $memory (export "memory") 2)
(global $g0 (mut i32) (i32.const 66560))
(global $__heap_base (export "__heap_base") i32 (i32.const 66560))
(global $__data_end (export "__data_end") i32 (i32.const 1024)))
If the outside world is calling tick
here, then this looks like the State
resource should be defined on the wasm side.
yes
the problem is that History also contains a history of previous States from each tick
oops
resource state {
static init: function() -> handle<state>
tick: function(candle: candle, history: handle<state>)
clone: function() -> handle<state>
}
since history should be a list of previous states, should it not be list<handle<state>>?
yeah
i see. that sounds good enough. however, in case i'd need to store/restore the states, is there a way of doing that?
serialize: function() -> list<u8>
static deserialize: function(bytes: list<u8>) -> handle<state>
would be one approach.
awesome!
what's a resource? i'm trying to find more documentation, but searching for "wasm resource" doesn't yield very good results
There's unfortunately not a lot of docs right now. This will eventually be answered by the interface-types docs. But for now:
A resource is just a type for opaque objects that can be pointed to by handle types.
are there code examples that use resources with wasmtime (preferably with rust)?
The wasi-filesystem API rewritten into wit has a resource: https://github.com/WebAssembly/wasi-filesystem/blob/main/wasi-filesystem.wit.md
descriptor
is a resource which one can think of as "the thing referenced by a file descriptor", with handles being the actual file descriptors
hmm, is there actual code?
The rust-wasm side is in-progress here: https://github.com/bytecodealliance/rustix/tree/wasi/src/imp/wasi
hmm, from what i can tell, resources are basically just something that one can get a handle<T> to, and handle<T> is basically a pointer to T
That's the gist of it. handle
is a smart pointer, so it won't dangle, and it promptly deallocates the resource when dropped.
i think i understand now why i'd need the WASM code to do its own allocation
from what i can tell, if i don't do this, i'd have to make sure that if i allocate memory outside of the wasm code, it doesn't interfere with what WASM does with the memory (since it doesn't know there is anything allocated)
Yeah. With wit-bindgen and interface-types in general, code on the outside of a wasm component doesn't have access to the linear-memory address space on the inside.
so, from what I can tell, i basically need
resource state {
// create an initial state
static init: function() -> handle<state>
// clone a state, so we can provide tick with a previous state
clone: function(state: handle<state>) -> handle<state>
// serialize state for storage
serialize: function() -> list<u8>
static deserialize: function(bytes: list<u8>) -> handle<state>
// tick has access to a mutable state (usually a clone of the last state in history)
// after the tick is done, the modified state might get stored in the history
tick: function(state: handle<state>, candle: candle, history: list<handle<state>>)
}
then, i use wit-bindgen in my rust->wasm program to provide those functions
however, in my host rust program, where wasmtime is run, i'm guessing handle<state> is just an i32
Yeah. The canonical ABI manages tables of handles, and provides Rust code with i32
indices into those tables.
awesome
also, i'm guessing since i don't have multiple wit modules, i won't hit the issue you linked
Yeah, with the wasm module defining the resource, you should be good
will this work?
resource state {
// clone a state, so we can provide tick with a previous state
clone: function(state: handle<state>) -> handle<state>
// serialize state for storage
serialize: function() -> list<u8>
static deserialize: function(bytes: list<u8>) -> handle<state>
}
// create an initial state
static init: function() -> handle<state>
// tick has access to a mutable state (usually a clone of the last state in history)
// after the tick is done, the modified state might get stored in the history
tick: function(state: handle<state>, candle: candle, history: list<handle<state>>)
also, i can't seem to be able to use https://bytecodealliance.github.io/wit-bindgen/ with handle<state>
ah, i'm probably misusing that page
hmm, or not? i'm not exactly sure
I'm not sure offhand how up to date that page is. wit-bindgen has been changing pretty rapidly
Hmm, I was mistaken. It looks like it's current.
Ah, right. It's handle state
rather than handle<state>
Also, the static init
function needs to be declared inside the resource
can I ask why that is?
I think you could define it outside too, but then you'd omit the static
keyword
static
means "don't add a self parameter"
ah, i see, that makes sense!
can *.wit files have parameterized types (generics)?
no
Some builtin types are parameterized, like list<T>
, but no user-defined types are.
i see. are there methods?
Functions defined in resources are effectively methods
other than that, "methods" are just prefixed functions i'm guessing
where the first parameter is the receiver
Yeah. In some langauge bindings, such as JS, having the concept of a receiver is useful to make the code more ergonomic.
thanks for the help @Dan Gohman! i would have been aimlessly lost without it! i really appreciate it
You're welcome!
Peter Vetere said:
Ok, so just to verify: Authors wanting to create WASM modules compatible with the canonical ABI must free any and all lists they receive from the host before returning from their function. Is that correct?
@Alex Crichton Just a quick follow-up on this: when a Wasm function returns a list result to the host, is the host then responsible for calling canonical_abi_free
on the linear memory offset it receives? Or is that memory lifetime also assumed to be managed by the guest?
@Peter Vetere correct yeah, the host takes "ownership" of the memory and must free it
Excellent. Thank you!
(this is where we need to more thoroughly document the canonical ABI)
Last updated: Dec 23 2024 at 12:05 UTC