Lann Martin (Apr 19 2023 at 22:18): Lann Martin (Apr 19 2023 at 22:18):Is there any plan for managing dependency name conflicts? It seems that if you have two transitive dependencies that both define an interface with the same name they cannot be used in the same world.
Lann Martin (Apr 19 2023 at 22:19):(I'm looking at you wasi-http.types :smile:)
Alex Crichton (Apr 19 2023 at 22:28):It's definitely intended that this is not a longstanding issue, but it's known that the deps folder organization currently suffers from this. Fully fixing that will require more in-depth integration with a registry or a better scheme for storing packages which can be referred to by other packages, and improving that has largely been "on hold" until the registry story is more advanced.
Conflicting interfaces, though, in theory shouldn't cause an issue, so you may be running into a bug? Are you able to share a reproduction?
Lann Martin (Apr 19 2023 at 22:39):This fails under cargo component build complaining about duplicate unmergable types interfaces: https://github.com/lann/wasi-http-experiment
Lann Martin (Apr 19 2023 at 22:41):In this particular case it's because the preview2 adapter includes an older version of wasi-http with an incompatible types interface, but that version is in http/ (in the adapter repo) while my copy is in wasi-http/
Alex Crichton (Apr 19 2023 at 23:19):ok I'll try to dig into this tomorrow to see if I can confirm whether or not this is a bug or whether it's intended to "work later"
Lann Martin (Apr 19 2023 at 23:32):I can work around the problem by renaming the interface in my local copy to e.g. types2, but it's awkward to need to patch a dependency like that
Alex Crichton (Apr 20 2023 at 00:09):yeah you definitely shouldn't have to do that, and if the need is here today then we'll need to fix this in the tooling sooner rather than later
Alex Crichton (Apr 20 2023 at 19:28):Ok I've dug a bit more into this now. Everything is working as expected currently, but that doesn't mean everything is in a great shape. Some thoughts I have poring over this are:
wit/world.wit adding import my-types: wasi-http.types.types to your world. This is basically explicitly giving your interface named types a different name than the adapter's http.types.types.wasi-http package you're using should be the same as the adapter's http package, but the name differences are preventing unioning these two (e.g. different names are, today, creating different URLs). This should work, however, if the package names agree and the contents are the same.Overall what's happening here, assuming that this is a use case where you're not actually shooting for the same http package, is that worlds are encapsulating a contract between the guest and the host about "here's what names we'll be using to import/export things" and something has to be done about name clashes. I'm not sure this is something that we can automatically handle since the component needs to do something and the host needs to be expecting the same thing. That's where the explicit disambiguation I mentioned above is a way to fix this by switching from the default "name after the interface" logic. That being said the types interface should probably be called something like wasi-types or wasi-http-types to be a little less "generic".
I'm not confident this won't cause a lot of issues down the road though. I'm not sure of a better solution off the top of my head, however, other than renaming interfaces. At least changing the names of interfaces won't cause issues ergonomically because most of them are default interface ... so the name isn't even mentioned in WIT, only in the component.
Lann Martin (Apr 20 2023 at 19:36):My guess is that the wasi-http package you're using should be the same as the adapter's http package, but the name differences are preventing unioning these two (e.g. different names are, today, creating different URLs). This should work, however, if the package names agree and the contents are the same.
They aren't the same. The adapter repo has an old version of wasi-http which is a couple of weeks old and incompatible with the wasi-http repo's current contents.
Alex Crichton (Apr 20 2023 at 19:38):oh sure yeah, but they're intended to be the same right?
Lann Martin (Apr 20 2023 at 19:39):Ah yeah, they're different versions of the same things
Alex Crichton (Apr 20 2023 at 19:39):if they're sync'd up I think that things should work
Alex Crichton (Apr 20 2023 at 19:39):or at least it's intended that everything works at that point
Alex Crichton (Apr 20 2023 at 19:39):where they'd naturally get unified into one
Lann Martin (Apr 20 2023 at 19:44):am I right in thinking that this means that every transitively imported interface name is part of one namespace?
Lann Martin (Apr 20 2023 at 19:44):at least if any of its types are exposed to the world?
Alex Crichton (Apr 20 2023 at 19:48):currently, yes, because components have a flat list of imports
Alex Crichton (Apr 20 2023 at 19:48):one thing we can do though is control those names as we see fit to have less likelihood of collisions
Alex Crichton (Apr 20 2023 at 19:48):and most of the naming doesn't need to get surfaced in the tooling itself or to developers in theory
Lann Martin (Apr 20 2023 at 19:52):It seems that practically all common interfaces would need to be prefixed with a namespace to avoid conflicts
Lann Martin (Apr 20 2023 at 19:53):if that is going to be the case long-term we should start updating interfaces sooner than later
Alex Crichton (Apr 20 2023 at 19:55):yeah the long-term picture isn't super clear to me, these sorts of details are really only sort of crystallizing in the last few weeks so it's only now really possible to reason about what the concrete problems are and what solutions we'll want to shape up
Alex Crichton (Apr 20 2023 at 19:55):we could alternatively employ an entirely different model of translating WIT to wasm which sidesteps this problem
Alex Crichton (Apr 20 2023 at 19:56):not that I know what such a model would be, though
Alex Crichton (Apr 20 2023 at 19:59):It is a bit of a shame to have a foundation of the component model which doesn't really have these problems per se but a system is layered on top which has these problems
Alex Crichton (Apr 20 2023 at 20:00):If you're up for it I think it'd be good to start an issue on the component-model repository about this
Alex Crichton (Apr 20 2023 at 20:01):one upside is that this may be a relatively isolated problem, basically the issue of "how is a world and it's transitive deps elaborated" which might be akin to "add one or two new features to worlds" as opposed to 'redesign lots of different things from scratch'
Lann Martin (Apr 20 2023 at 20:02):This seems like essentially the same discussion: https://github.com/WebAssembly/component-model/issues/177
Alex Crichton (Apr 20 2023 at 20:05):perhaps yeah! I'd have to reread later
Last updated: Nov 22 2024 at 16:03 UTC