Stream: wit-bindgen

Topic: dependency namespacing


view this post on Zulip Lann Martin (Apr 19 2023 at 22:18):

Is there any plan for managing dependency name conflicts? It seems that if you have two transitive dependencies that both define an interface with the same name they cannot be used in the same world.

view this post on Zulip Lann Martin (Apr 19 2023 at 22:19):

(I'm looking at you wasi-http.types :smile:)

view this post on Zulip Alex Crichton (Apr 19 2023 at 22:28):

It's definitely intended that this is not a longstanding issue, but it's known that the deps folder organization currently suffers from this. Fully fixing that will require more in-depth integration with a registry or a better scheme for storing packages which can be referred to by other packages, and improving that has largely been "on hold" until the registry story is more advanced.

Conflicting interfaces, though, in theory shouldn't cause an issue, so you may be running into a bug? Are you able to share a reproduction?

view this post on Zulip Lann Martin (Apr 19 2023 at 22:39):

This fails under cargo component build complaining about duplicate unmergable types interfaces: https://github.com/lann/wasi-http-experiment

Contribute to lann/wasi-http-experiment development by creating an account on GitHub.

view this post on Zulip Lann Martin (Apr 19 2023 at 22:41):

In this particular case it's because the preview2 adapter includes an older version of wasi-http with an incompatible types interface, but that version is in http/ (in the adapter repo) while my copy is in wasi-http/

view this post on Zulip Alex Crichton (Apr 19 2023 at 23:19):

ok I'll try to dig into this tomorrow to see if I can confirm whether or not this is a bug or whether it's intended to "work later"

view this post on Zulip Lann Martin (Apr 19 2023 at 23:32):

I can work around the problem by renaming the interface in my local copy to e.g. types2, but it's awkward to need to patch a dependency like that

view this post on Zulip Alex Crichton (Apr 20 2023 at 00:09):

yeah you definitely shouldn't have to do that, and if the need is here today then we'll need to fix this in the tooling sooner rather than later

view this post on Zulip Alex Crichton (Apr 20 2023 at 19:28):

Ok I've dug a bit more into this now. Everything is working as expected currently, but that doesn't mean everything is in a great shape. Some thoughts I have poring over this are:

  1. You can get everything working today by in your local wit/world.wit adding import my-types: wasi-http.types.types to your world. This is basically explicitly giving your interface named types a different name than the adapter's http.types.types.
  2. The adapter probably shouldn't have information about http since it doesn't even use it. This is an artifact of the preview2-prototyping repository being a central place for WASI right now, which I think will get "fixed" in the future.
  3. My guess is that the wasi-http package you're using should be the same as the adapter's http package, but the name differences are preventing unioning these two (e.g. different names are, today, creating different URLs). This should work, however, if the package names agree and the contents are the same.

Overall what's happening here, assuming that this is a use case where you're not actually shooting for the same http package, is that worlds are encapsulating a contract between the guest and the host about "here's what names we'll be using to import/export things" and something has to be done about name clashes. I'm not sure this is something that we can automatically handle since the component needs to do something and the host needs to be expecting the same thing. That's where the explicit disambiguation I mentioned above is a way to fix this by switching from the default "name after the interface" logic. That being said the types interface should probably be called something like wasi-types or wasi-http-types to be a little less "generic".

I'm not confident this won't cause a lot of issues down the road though. I'm not sure of a better solution off the top of my head, however, other than renaming interfaces. At least changing the names of interfaces won't cause issues ergonomically because most of them are default interface ... so the name isn't even mentioned in WIT, only in the component.

view this post on Zulip Lann Martin (Apr 20 2023 at 19:36):

My guess is that the wasi-http package you're using should be the same as the adapter's http package, but the name differences are preventing unioning these two (e.g. different names are, today, creating different URLs). This should work, however, if the package names agree and the contents are the same.

They aren't the same. The adapter repo has an old version of wasi-http which is a couple of weeks old and incompatible with the wasi-http repo's current contents.

view this post on Zulip Alex Crichton (Apr 20 2023 at 19:38):

oh sure yeah, but they're intended to be the same right?

view this post on Zulip Lann Martin (Apr 20 2023 at 19:39):

Ah yeah, they're different versions of the same things

view this post on Zulip Alex Crichton (Apr 20 2023 at 19:39):

if they're sync'd up I think that things should work

view this post on Zulip Alex Crichton (Apr 20 2023 at 19:39):

or at least it's intended that everything works at that point

view this post on Zulip Alex Crichton (Apr 20 2023 at 19:39):

where they'd naturally get unified into one

view this post on Zulip Lann Martin (Apr 20 2023 at 19:44):

am I right in thinking that this means that every transitively imported interface name is part of one namespace?

view this post on Zulip Lann Martin (Apr 20 2023 at 19:44):

at least if any of its types are exposed to the world?

view this post on Zulip Alex Crichton (Apr 20 2023 at 19:48):

currently, yes, because components have a flat list of imports

view this post on Zulip Alex Crichton (Apr 20 2023 at 19:48):

one thing we can do though is control those names as we see fit to have less likelihood of collisions

view this post on Zulip Alex Crichton (Apr 20 2023 at 19:48):

and most of the naming doesn't need to get surfaced in the tooling itself or to developers in theory

view this post on Zulip Lann Martin (Apr 20 2023 at 19:52):

It seems that practically all common interfaces would need to be prefixed with a namespace to avoid conflicts

view this post on Zulip Lann Martin (Apr 20 2023 at 19:53):

if that is going to be the case long-term we should start updating interfaces sooner than later

view this post on Zulip Alex Crichton (Apr 20 2023 at 19:55):

yeah the long-term picture isn't super clear to me, these sorts of details are really only sort of crystallizing in the last few weeks so it's only now really possible to reason about what the concrete problems are and what solutions we'll want to shape up

view this post on Zulip Alex Crichton (Apr 20 2023 at 19:55):

we could alternatively employ an entirely different model of translating WIT to wasm which sidesteps this problem

view this post on Zulip Alex Crichton (Apr 20 2023 at 19:56):

not that I know what such a model would be, though

view this post on Zulip Alex Crichton (Apr 20 2023 at 19:59):

It is a bit of a shame to have a foundation of the component model which doesn't really have these problems per se but a system is layered on top which has these problems

view this post on Zulip Alex Crichton (Apr 20 2023 at 20:00):

If you're up for it I think it'd be good to start an issue on the component-model repository about this

view this post on Zulip Alex Crichton (Apr 20 2023 at 20:01):

one upside is that this may be a relatively isolated problem, basically the issue of "how is a world and it's transitive deps elaborated" which might be akin to "add one or two new features to worlds" as opposed to 'redesign lots of different things from scratch'

view this post on Zulip Lann Martin (Apr 20 2023 at 20:02):

This seems like essentially the same discussion: https://github.com/WebAssembly/component-model/issues/177

My understanding is that right now there is a single namespace in the context with WIT. That can cause some confusion as you can see in this discussion here. For the case of wasi:http/outgoing-hand...

view this post on Zulip Alex Crichton (Apr 20 2023 at 20:05):

perhaps yeah! I'd have to reread later


Last updated: Dec 23 2024 at 12:05 UTC