Stream: wit-bindgen

Topic: Memory ownership with the C binding generator

Sy Brand (Jun 17 2025 at 13:00):

I'm confused by the memory ownership semantics when using the C binding generator. The project README shows:

#include "host.h"

void host_run() {
    host_string_t my_string;
    host_string_set(&my_string, "Hello, world!");

    host_print(&my_string);
}

But the headers generated by wit-bindgen have the following comment on xxx_string_set:

// Transfers ownership of `s` into the string `ret`

Which implies to me that ret is going to be freed somewhere. But that doesn't make sense for the code in the README, considering that xxx_string_set simply stores the pointer and length, so if my_string were to be freed, it would be freeing a static string.

Is there documentation somewhere that specifies who is responsible for freeing memory that is passed to an import?

Lann Martin (Jun 17 2025 at 13:09):

I don't know about documentation but the caller is responsible. There should be a host_string_free if I'm understanding the example.

Lann Martin (Jun 17 2025 at 13:10):

The "transfer ownership" terminology is a little confusing in context; I think it really just means that the returned struct will own the allocation.

Sy Brand (Jun 17 2025 at 13:11):

There would be a host_string_free defined, but calling it on my_string would call free on something that wasn't allocated via malloc

Lann Martin (Jun 17 2025 at 13:13):

OK fair so my terminology was also confusing :slight_smile:

Lann Martin (Jun 17 2025 at 13:15):

Imports will not free arguments that are passed to them; the generated canonical ABI code may do its own memory management internally but the caller is responsible for anything it constructs and passes in. If you know how to express this in a way that would be more familiar to C devs I'm sure a PR would be welcome.

Sy Brand (Jun 17 2025 at 13:15):

:D
So basically nothing will be freed without you explicitly freeing it, xxx_string_set "takes ownership" as far as if you eventually call xxx_string_free on the result, you better not have references to the internal string lying around

Sy Brand (Jun 17 2025 at 13:16):

That's clear now, thank you!

Lann Martin (Jun 17 2025 at 13:16):

Note there may be some caveats around resource handles and upcoming async stuff, but otherwise I think that is correct.

Sy Brand (Jun 17 2025 at 13:19):

I think as far as documenting goes, a short description of the ownership model in the C section of the README and a bit more detail in the header comments to indicate the intended use of the functions would be helpful and make it so that users wouldn't have to read between the lines

Sy Brand (Jun 17 2025 at 13:19):

I'd be happy to put in a PR

Lann Martin (Jun 17 2025 at 13:20):

On the flip side, imports may return allocated data and the caller does take ownership of that. iirc you can just blindly call *_free on any non-primitive return values; there should be at least an empty impl generated.

Sy Brand (Jun 17 2025 at 13:21):

Makes sense!

Lann Martin (Jun 17 2025 at 13:24):

The (especially non-Rust) bindings are always in need of more love

Ralph (Jun 17 2025 at 14:10):

C means <3

Alex Crichton (Jun 17 2025 at 14:36):

I can probably apologize for the lack of documentation here, sorry!

Overall the memory ownership story for C is pretty tricky. The bindings are generated in such a way (in theory) that it's possible to use them without memory leaks, but it's not always easy to do so. I have no doubt that existing examples probably leak memory as nothing is checking for no leaks and that internal documentation can likely be improved (e.g. *_string_set)

Sy Brand (Jun 17 2025 at 14:55):

I can go through the code and try to come up with a short document that explains the different functions that can be generated, what their intended uses are, and how to use them for arguments and return values to avoid leaks/bad frees. That can live in crates/c/README.md or some other name and be linked to from the C section of the top-level README. Would that be useful?

Alex Crichton (Jun 17 2025 at 14:57):

that would be extremely useful!

Sy Brand (Jun 17 2025 at 14:58):

Cool cool! I'll drop a message here if I need clarity on anything :blush:

Sy Brand (Jun 17 2025 at 14:58):

I appreciate all the quick responses

Lann Martin (Jun 17 2025 at 15:14):

When someone offers to help suddenly everyone comes out of the woodwork :slight_smile:

Ralph (Jun 17 2025 at 16:54):

EXTREMELY USEFUL LANN

Sy Brand (Jun 18 2025 at 10:39):

I've made decent progress, but am a bit stuck on the dropping of borrowed handles of imported resources in exported functions

Sy Brand (Jun 18 2025 at 10:41):

My understanding is that, if the guest has a borrowed handle to an imported resource, its responsibility for dropping that handle depends on how that borrow was obtained:

• If the borrow was obtained by calling *_borrow_* on an owned handle, no action is required on the borrow; only the owned handle should be dropped when the guest wants to end the lifetime of the resource
• If the borrow was received as an argument to one of the guest's exported functions, the borrowed handle must be dropped by the guest

Sy Brand (Jun 18 2025 at 10:43):

On that second bullet point, there are two sub-cases that I can see for how the guest is supposed to drop the borrowed handle:

• If --autodrop-borrows=yes is supplied on the command line, the bindings automatically drop the borrowed handle after the implementation of the exported function returns
• If --autodrop-borrows=no is supplied, or that argument is not supplied at all, the guest code must explicitly drop the borrowed handle

Sy Brand (Jun 18 2025 at 10:45):

The problem is, I don't see what it is intended for the guest code to call to explicitly drop the handle for the second case here, as the *_drop_borrow function is only generated if autodrop is enabled

Sy Brand (Jun 18 2025 at 10:48):

My conclusion is that the condition here is inverted and should instead be !self.autodrop_enabled(), so that the guest can manually call *_drop_borrow before exiting the exported function to drop the received borrow handle

Sy Brand (Jun 18 2025 at 10:49):

But perhaps some the understanding I've written leading up to this conclusion is wrong

Alex Crichton (Jun 18 2025 at 15:10):

Agreed with you, I think that's a typo and not-very-broadly-tested code, the *_drop_borrow functions should definitely be provided if autodrop is disabled

Alex Crichton (Jun 18 2025 at 15:10):

Also can confirm your understanding of it all is indeed correct :+1:

Sy Brand (Jun 18 2025 at 15:11):

Cool, I can make a PR for that

Alex Crichton (Jun 18 2025 at 15:14):

If you're feeling particularly ambitious we've also nowadays got a system that's much more robust for adding tests for various bindings generators, so you an also try adding a test to tests/runtime/* for this behavior too

Sy Brand (Jun 18 2025 at 15:14):

Maybe this indicates that autodrop could be made the default? Since the *_drop_borrow functions haven't been provided for non-autodrop builds since January last year, it seems that any users of borrowed imported resources in exported functions must either:

• Have since turned autodrop on
• Be leaking borrowed handles
• Have resorted to calling the dropping intrinsic directly (icky icky)
  Making it the default would cause that last group to double-free their borrowed handles, but you'd know better if anybody falls into that group :smile:

Alex Crichton (Jun 18 2025 at 15:18):

I fear you might be overestimating the number of users of the C bindings with borrows of imported resources in exports heh, I'd expect that set of folks to be almost empty

Alex Crichton (Jun 18 2025 at 15:18):

but historically the reason I wanted autodrop turned off is that it doesn't support all signatures, but I also wanted bindings to be possible to generate by default

Sy Brand (Jun 18 2025 at 15:19):

Yep, that is what I imagined :smile:

Alex Crichton (Jun 18 2025 at 15:19):

so it's a bit of a tradeoff with autodrop, it's (a) unambiguously nicer IMO but (b) more restrictive

Sy Brand (Jun 18 2025 at 15:19):

Ah, makes sense!

Alex Crichton (Jun 18 2025 at 15:19):

IIRC I think list<borrow<thing>> in an exported function's arguments won't be supported and the bindings generation process should panic/fail

Alex Crichton (Jun 18 2025 at 15:19):

it's always lists ;_;

Sy Brand (Jun 18 2025 at 15:20):

With the imported resources thing cleared up, I'm aaaalmost done with the doc draft

Sy Brand (Jun 18 2025 at 15:38):

The last bit I'm a unsure about is some parts of exported resources:

1. It seems that the *_drop_borrow function gets defined for exported resources if the exported resource is used in an imported function. Am I right that there's no reason for this to happen and the component that exports the resource should never drop a borrowed handle to that resource, even if it receives one as an argument in one of its exported functions?

2. I'm not sure about the intended allocation and destruction pattern for the resource. Say I have a resource like:

package cat:registry;
 resource cat {
    get-name: func() -> string;
    get-nicknames: func() -> list<string>;
}

And I define the internal representation like this:

struct exports_cat_registry_cat_registry_api_cat_t {
  cat_registry_string_t name;
  cat_registry_list_string_t nicknames;
};

Is the intended lifecycle of a single cat resource as follows:

• malloc an exports_cat_registry_cat_registry_api_cat_t
• Fill in the name and nicknames fields
• Call exports_cat_registry_cat_registry_api_cat_new with the malloced pointer
• Pass around the returned handle, do whatever
• Eventually call exports_cat_registry_cat_registry_api_cat_drop_own
• Dropping the owning reference automatically calls exports_cat_registry_cat_registry_api_cat_destructor
• exports_cat_registry_cat_registry_api_cat_destructor should clean up the name and nicknames fields, then call free on the argument to the destructor

Alex Crichton (Jun 18 2025 at 15:52):

Am I right that there's no reason for this to happen and the component that exports the resource should never drop a borrowed handle to that resource, even if it receives one as an argument in one of its exported functions?

Correct, it's not actually even possible for a borrow handle to make its way to the component defining the handle. When an export receives a borrow of a resource the component defined it gets a rep, not a handle, so there's fundamentally nothing to drop

Alex Crichton (Jun 18 2025 at 15:53):

Is the intended lifecycle of a single cat resource as follows:

That all looks correct to me yeah

Sy Brand (Jun 18 2025 at 15:59):

Thanks!

Sy Brand (Jun 18 2025 at 18:19):

Another exported resource question. Given this definition:

package cat:registry;

interface cat-registry-api {
    resource cat {
        get-name: func() -> string;
        get-nicknames: func() -> list<string>;
    }
}

interface cat-registry-user-api {
    use cat-registry-api.{cat};
    init: func();
    notify-cat-registered: func(cat: borrow<cat>);
}

world cat-registry {
    import cat-registry-user-api;
    export cat-registry-api;
    export wasi:cli/run@0.2.6;
}

I expect that, when generating bindings for cat-registry, the bindings for notify-cat-registered should take a pointer to the resource repr. But instead, the generator seems to treat it as if I'm importing cat because it is appearing in the context of an import (the cat-registry-user-api interface)

Sy Brand (Jun 18 2025 at 18:19):

Which is why the *_drop_borrow from my earlier question gets generated

Sy Brand (Jun 18 2025 at 18:21):

More concretely, the generator declares this:

extern void cat_registry_cat_registry_user_api_notify_cat_registered(cat_registry_cat_registry_user_api_borrow_cat_t cat);

When I expect it to generate this:

extern void cat_registry_cat_registry_user_api_notify_cat_registered(exports_cat_registry_cat_registry_api_borrow_cat_t cat);

Sy Brand (Jun 18 2025 at 18:21):

I think the generator is missing tracking of whether a resource that appears in an imported function is one that the component we're generating for is exporting

Sy Brand (Jun 18 2025 at 18:23):

Am I on the right lines, or is there some way I'm missing to do the right thing with the generated bindings, or some issue with my WIT?

Alex Crichton (Jun 18 2025 at 18:35):

Ah ok so you're hitting on a particular subtle point of the component model

Alex Crichton (Jun 18 2025 at 18:35):

this is working as expected, but it's known to be surprising, lemme see if I can dig up any docs on this

Alex Crichton (Jun 18 2025 at 18:39):

hm ok no shame on me those don't exist...

Anyway the basic limitation here is that imports into a component cannot refer to types defined in the component itself, e.g. exports, as that would create a cycle which can't otherwise be broken. Effectively the exported resource is a type that doesn't actually exist until the component is instantiated, meaning that imports, which must exist before the component is instantiated, otherwise couldn't exist.

This then leads to the question "why is this valid WIT?" and that gets into the concept of world elaboration. A small digression is exploring what happens if you write down import wasi:filesystem/preopens. That's also a valid WIT file but the preopens interface depends on wasi:filesystem/types, so somehow that has to get resolved. What ends up happening here is something I've been calling world elaboration where import statements transitively, and automatically, import dependency interfaces too.

Coming back to your example, if you do a wasm-tools component wit to print the elaborated form of the WIT you'll see that world cat-registry actually does import cat-registry-api despite you not actually writing this down. That's beause cat-registry-user-api depends on cat-registry-api, and it's otherwise not imported, so it's then forcibly imported.

So at the end of the day what you actually wrote down is both importing and exporting cat-registry-api. The notify-cat-registered function is referring to the imported resource, and the exported function is referring to a different resource, one that's exported

Alex Crichton (Jun 18 2025 at 18:40):

There's unfortunately a lot of subtelty about this which can come up relatively quickly when folks are initially learning/playing around with WIT. If you're working with preexisting WIT you tend to be fine as this situation doesn't come up, but this is something where we could definitely have better documentation

Sy Brand (Jun 18 2025 at 19:05):

Ah, I understand, thanks for the detailed explanation!

Sy Brand (Jun 19 2025 at 13:18):

Apologies, I'm still having trouble wrapping my head round some of these subtleties. I can successfully create a component that imports some resource and exports a function that takes a borrow of that resource. But I'm struggling to create a configuration by which I can actually call that exported function, likely due to misunderstanding some of the finer points of the component model.

Say I have the following WIT:

package cat:example;

interface registry-api {
    resource cat {
        get-name: func() -> string;
        get-nicknames: func() -> list<string>;
    }
    adopt-cat: func(name: string) -> option<cat>;
}

interface adoption-authority-api {
    use registry-api.{cat};
    // The function I am trying to call
    notify-adoption: func(cat: borrow<cat>);
}

world adoption-authority {
    import registry-api;
    export adoption-authority-api;
}

world adopter {
    import adoption-authority-api;
    export init: func();
}

world registry {
    export registry-api;
    export wasi:cli/run@0.2.6;
}

The idea is that the adopter calls registry/adopt-cat in order to retrieve an owing handle to a cat. adopter then calls adoption-authority/notify-adoption, passing it a borrow of its owned handle.

Sy Brand (Jun 19 2025 at 13:19):

Composing these with wac plug fails, which I'm guessing is because it sees the import of cat inside adoption-authority and adopter as separate types

Sy Brand (Jun 19 2025 at 13:21):

So I try to do it with wac compose, hoping that this will be more successful by instantiating registry once and using that to fulfil the imports of both adopter and adoption-authority. Using this script:

package cat:composition;

let registry = new cat:registry{};

let authority = new cat:adoption-authority {
    registry-api: registry
};

let adopter = new cat:adopter {
    adoption-authority-api: authority,
    registry-api: registry
};

export registry.run;

I get the error:

  × mismatched instantiation argument `cat:example/registry-api`
  ╰─▶ instance is missing expected resource export `cat`
   ╭─[registry.wac:6:5]
 5 │ let authority = new cat:adoption-authority {
 6 │     registry-api: registry
   ·     ──────┬─────
   ·           ╰── mismatched argument `cat:example/registry-api`
 7 │ };
   ╰────

Sy Brand (Jun 19 2025 at 13:25):

There's clearly some fundamental issue with my understanding of the import and export system. I'd be totally happy for a "what you're doing is totally unsupported" answer, I'm just hoping for an example that calls an exported function that has an imported resource as an argument.

Ralph (Jun 19 2025 at 13:42):

wac may also not be perfect yet, as the tools have to keep pace.... and maintainers are spilt all over. :-)

Sy Brand (Jun 19 2025 at 14:08):

Aha! The problem was my WAC syntax was wrong, I need to manually select the interface from the component like

let authority = new cat:adoption-authority {
    registry-api: registry.registry-api,
};

Sy Brand (Jun 19 2025 at 15:13):

It works!!!!

[registry] Initialised
[adopter] Attempting to adopt Poptart...
[registry] Adopted cat:
[registry] Poptart
[adopter] Adopted cat name:
[adopter] Poptart
[adoption-authority] Notified of adoption
[adoption-authority] Cat name:
[adoption-authority] Poptart
[adopter] Adoption complete!
[registry] Destroyed

Sy Brand (Jun 19 2025 at 16:22):

Alright, the documentation is ready for review: https://github.com/bytecodealliance/wit-bindgen/pull/1319

Ralph (Jun 19 2025 at 16:59):

The TartanLlama Abides.

Lann Martin (Jun 19 2025 at 18:39):

Apologies, I'm still having trouble wrapping my head round some of these subtleties.

You and almost everyone else :slight_smile:

Alex Crichton (Jun 20 2025 at 14:09):

(back today)

Ah yeah I would comment saying that the WIT you describe above should indeed work, and I'm glad the wac syntax got sorted out! I'm still learning wac myself :)

I should be able to review the C docs today as well, thanks again!

Sy Brand (Jun 20 2025 at 15:03):

Thanks! I also submitted a PR to fix the non-autodropped borrows case, including a runtime test: https://github.com/bytecodealliance/wit-bindgen/pull/1320

Sy Brand (Jun 20 2025 at 15:04):

I couldn't seem to run the tests with cargo test as explained in the documentation though; I had to use wit-bindgen test. Is the documentation out of date or did I do something wrong?

Sy Brand (Jun 20 2025 at 15:11):

By which I mean running cargo test -p wit-bindgen-cli --no-default-features -F c prints

running 1 test
test verify_cli ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Which doesn't seem like it's running all the tests, and if I intentionally break one, it still passes

Sy Brand (Jun 20 2025 at 15:21):

Seems like maybe a similar change to this is required to the entire readme: https://github.com/bytecodealliance/wit-bindgen/pull/1310

Alex Crichton (Jun 20 2025 at 15:25):

oh sorry yeah that's out of date docs

Alex Crichton (Jun 20 2025 at 15:26):

nowadays it's, from the root of the repo:

cargo run test -l rust,c --artifacts target/artifacts --rust-wit-bindgen-path ./crates/guest-rust ./tests/runtime

Alex Crichton (Jun 20 2025 at 15:26):

you can also pass just -l c to skip rust tests

Sy Brand (Jun 20 2025 at 15:29):

Cool cool, I'll make a PR for that too!

Sy Brand (Jun 20 2025 at 23:36):

Thanks for all the feedback! Will likely address it on Monday

Sy Brand (Jun 25 2025 at 16:17):

Alex Crichton said:

Anyway the basic limitation here is that imports into a component cannot refer to types defined in the component itself snip

I took your explanation and drafted a blog post that explains all of this in more detail. Mostly for my own benefit, but hopefully it'll be useful to point people at if they make the same mistakes I did. If you happen to have time to give it a look for correctness, I'd much appreciate it! Otherwise I'll just yeet it to the public in the next week or so and hope there are no mistakes :smile:.

Alex Crichton (Jun 25 2025 at 17:46):

Oh that's a fantastic post, I'm definitely going to save this and link this to others when this situation comes up!

All looks accurate to me as well, thanks for taking the time to do this!

Last updated: Dec 06 2025 at 06:05 UTC