Conflicting dependencies and use of the ~> operator

You can override dependencies if necessary, but I think there is something major missing from overriding dependencies, specifically I think you ought to be able to say why you are overriding a dependency, and then mix can tell you when you don’t need to anymore.

For example:

{:jq, "~> x.x", override: [:merquery]}

says “I know that merquery wants a different version, but I’m overriding it”. Then if later you update merquery, and no longer need the override, mix will instruct you to remove it. Additionally, if you add some new dependency that wants an older version of jq, you have to acknowledge that you are also overriding that dependency.

This makes it a bit safer to use override as a consumer of libraries, which I think helps alleviate this issue a bit as well.

Yeah, good point… I didn’t know about the override option. Feels like !important is CSS

But for making things smooth for Elixir devs, they would need to know about override (I’ve been using Elixir for 6-7 years and I just learned about it!)

I think in the case I outline that you’d need to add poison to your application’s mix.exs and then force that into place maybe? Or could I force jq and then it ignores its dependencies?

I’m definitely not against the option of making override clear in the documentation (with warnings, probably, and maybe in addition to my suggestions?)

Either one of those would work, yes

Usually when I see this option being used in a codebase, it’s a red flag for me, as you can get into a situation where the dependency override might break that other library.

There are tools that allow library developers to define more flexible dependency versions, here is an example from ecto: ecto/mix.exs at master · elixir-ecto/ecto · GitHub

FWIW, >= in libraries can cause its own headaches, especially if the package being referenced doesn’t follow semver.

For instance, here’s a pair of issues from standardrb that were ultimately driven by interactions between >= and already-locked versions:

https://github.com/standardrb/standard/issues/340

https://github.com/standardrb/standard/issues/613

I agree, but I think in some cases it is effectively unavoidable. I think we need to enhance it, not discourage its use. If I can say “I’m overriding to solve this specific dependency conflict that I’m aware of”, then most/all of the danger of using that option goes away.

Well, this is the problem, you can’t really guarantee that unless you inspect the codebase of dependencies. The dynamic nature of elixir makes this even more dangerous, as it might blow in your face at runtime.

I think a potential solution would be to run unit tests of that library with the dependency you have at hand? If I’m not wrong this is what PERL does when you install packages, to make sure you don’t have dependencies mismatch somewhere. This OFC heavily depends on the library, as some libraries require a complex setup to run tests.

I’m suggesting that this is built into the mix dependency resolver. Right now the following happens all the time:

• I want to add foo to my app. I already have bar and baz.
• foo depends on a newer version of bar.
• I can’t update bar because baz depends on it.
• I check how they baz and foo use bar, and confirm that its fine to just override.
• So I add {:bar, "~> ...", override: true}
• Someone else adds buzz to the app, which also depends on an old version of bar.
• Problem 1: We never find out that we just overrode bar for the sake of buzz as well.
• Next, foo releases an update that depends on more stuff from the old version of bar.
• Problem 2: mix tells us we can update foo, so we update it and have bugs.

If instead of override: true, I could say:

{:bar, "~> x.x", override: [foo: "x.x.x"]}

which would say “this override only overrides the dependency that foo at exactly version x.x.x has on bar”, then we are protected from any of those accidental changes.

Adding buzz would produce an appropriate dependency conflict warning, solving Problem 1. We can then go look at the code/docs and decide if we want to override the bar dependency for that version of buzz as well.

foo won’t appear to be automatically upgradeable, solving Problem 2.

Great idea! It would certainly help a lot to get mismatch version errors, this is especially true for some of the bigger applications, where you might have 100+ dependencies.

I wonder if there is a language out there that solved this problem of “global” dependencies?

Yeah, for sure… I guess that’s part of why I was trying to ephasize that one should prefer the use of >= in a library, but I definitely wouldn’t say that you should always use it. I guess it feels like many people don’t think of it as an option, and so I worry about people not really thinking about it (and I get it, you’re trying to get your cool idea for a library out there in front of people)

I think if we wanted to provide guidance to people, we should give people both options on dealing with such things depending on their usage of the dependency:

• poison >= 4.0
• poison ~> 4.0 or ~> 5.0 or ~> 6.0 / poison >= 4.0 and < 7.0

But also perhaps some sort of tool that lets them know (as a library maintainer) when there is a new version of sub dependencies which aren’t being allowed (either as a part of mix somehow, or a bot that points in out in a pull request).

For override, I can see how either the documentation or the functionality could be improved as @zackdaniel outlined, though I’m not personally interested in tackling that problem.