Why doesn't the Elixir core team accept `mix deps.add` proposals?

Chat / Discussions
1

I’m thinking of submitting yet another mix deps.add proposal. It has gotten to the point where I feel that such a simple and repetitive task could easily be shortened by adding a mix task for it. However, I have noticed that the Core team ignores/rejects such proposals. I tried look through the mails of the past 2 years and I haven’t found any explanation.

Here’s what I am going to propose:

mix deps.add will only be used for editing the Mixfile and adding a task. Added dependencies are only fetched with deps.get
By default, we would supply the package name, and we will add the latest version. For example:

mix deps.add ecto

Will add

{:ecto, "~> 3.6"}

at this point of time.

It will have the options of --version (to specify version) --git(to specify git repository) and --github (to specify GitHub repository)

So 2 things:

  1. Why are deps.add proposals rejected

  2. What do you think of this proposal ?

PS: This is my first post. Apologies if I violated any code of conduct.

6 Likes
2

One of the main reasons is that such a task wouldn’t be able to work reliably, as the mix.exs is code, and not structured data.

Someone experimented with a tool that has written and read an external deps file.

Not sure if this is still findable on GitHub or perhaps even was released on hex.

Though AFAIK it never left the “experimental” status and is also considered problematic as it needs to be installed “globally”

6 Likes
3

The tricky question is: where should it be added exactly? Given mix.exs is code there could be any mounts of logic in the list of deps. Even the default of having dependencies listed in a helper function is just convention, but not in any way enforced.

3 Likes
4

Let’s not go down the route of node.js. I absolutely hate it when npm install change my package.json file and mess up the indentation.

Having a file that regularly needs to be modified by human and machine is asking for trouble.

12 Likes
5

On top of what’s already been said, the amount of work you need to do to correctly insert the new dependency without messing up the rest of the code is a bit non trivial. Even if we make it work only in the case of the defp deps do ... convention, regex is unreliable and doing it by playing with the ast is quite a bit of work, even for this single use case. It is doable though.

Regarding the “deps could be anywhere besides deps”, this is one option:

You can make it bail out and give a good error message explaining why. If you’ve moved your deps away from the default, you’re already probably experienced enough to deal with it. This is letting niche use cases prevent usability improvements from the majority.

6 Likes
6

To me the best proposal would be to add a trailing comma in the one-click copy version on hex.pm. That’s it.

6 Likes
7

IMO getting something like mix deps.add to work reliably, we need a first-class engine for modifying Elixir code. @dorgan here is working on it. :heart:

2 Likes
8

I think we can only follow the convention of adding a dependency to the deps function.

When it comes to actually editing the file, the solution that came to mind is using some sort of regex. This though is unreliable albeit easy. I think using the AST is much more reliable, but I am not sure how you would do so. Any resources on editing the code programmatically is much appreciated.

9

I think the only way to reliably make this work is to introduce a plain data file - so something like deps.json. Everything else has the issue of having the need to evaluate the code. Not sure what are the exact requirements of .formatter.exs are, but it seems you can throw an if into that file and it just works. Same goes for mix.exs and a potential deps.exs.

And if you’re going to make it work in limited number of use cases - which also probably covers something like 90% of occurences - I think it makes sense to add it as an external package. You could mix install it just once and then just have a global task like phx.new.

2 Likes
10

Few examples of mix.exs for you to deduce how you would like to add the new dependency:

defmodule Foo.Mixfile do
  use Mix.Project

  def project, do: [app: :foo, version: "0.0.1"]
end

defmodule Foo.Mixfile do
  use Mix.Project

  def project do
    [app: :foo, version: "0.0.1"]
  end
end

defmodule Foo.Mixfile do
  use Mix.Project

  def project do
    [app: :foo, version: "0.0.1", deps: [{:mydep, ">= 0.0.0"}]]
  end
end

defmodule Foo.Mixfile do
  use Mix.Project

  def project do
    [
      app: :foo,
      version: "0.0.1",
      deps: foos()
    ]
  end

  defp foos, do: []
end

defmodule Foo.Mixfile do
  use Mix.Project

  def project do
    [
      app: :foo,
      version: "0.0.1",
      deps: deps(Mix.env())
    ]
  end

  defp deps(:dev) do
    []
  end

  defp deps(:test) do
    [{:benchee, ">= 0.0.0"}]
  end
end

defmodule Foo.Mixfile do
  use Mix.Project

  def project do
    [
      app: :foo,
      version: "0.0.1",
      deps: deps(System.get_env("BENCH"))
    ]
  end

  defp deps(nil) do
    []
  end

  defp deps(_) do
    [{:benchee, ">= 0.0.0"}]
  end
end

defmodule Foo.Mixfile do
  use Mix.Project

  def project do
    [
      app: :foo,
      version: "0.0.1",
      deps: deps()
    ]
  end

  defp deps do
    [{:foo, ">= 0.0.0"} | more_deps()]
  end

  defp more_deps() do
    [{:bar, ">= 0.0.0"}]
  end
end

defmodule Foo.Mixfile do
  use Mix.Project

  def project do
    [
      app: :foo,
      version: "0.0.1",
      deps: deps(System.get_env("BENCH"))
    ]
  end

  defp deps(nil) do
    []
  end

  defp deps(_) do
    [{:benchee, ">= 0.0.0"}]
  end
end

defmodule Foo.Mixfile do
  use Mix.Project

  @deps [{:foo, ">= 0.0.0"}]

  def project do
    [
      app: :foo,
      version: "0.0.1",
      deps: @deps
    ]
  end
end

defmodule WhyNot do
  def deps, do: [{:foo, ">= 0.0.0"}]
end

defmodule Foo.Mixfile do
  use Mix.Project

  def project do
    [
      app: :foo,
      version: "0.0.1",
      deps: WhyNot.deps()
    ]
  end
end
13 Likes
11

Thank you!

12

If one wants to see an example of those in the wild look at ecto_sql’s mix.exs.

3 Likes
13

Yeah. I am in favour of having a deps.json but it’s quite a major change (in my opinion), and I am not sure if it will be accepted till the next major release; i.e 2.0.

14

How much time do folks actually spend adding dependencies? I don’t do it much personally, but maybe I’m an outlier.

IMO “remember the right options to pass mix deps.add” is almost as much cognitive load as “remember the right tuple shape to put in deps”, just with extra steps.

12 Likes
15

I agree that such a mix task could be very useful, but should this maybe be a library instead of a core feature? My logic is: a library could use a file such as deps.json and instruct the user on how to set it up in mix.exs (a very simple operation on most projects), without forcing every project to adopt that structure.

4 Likes
16

Is this really a pressing issue? I really have no problem adding dependencies to mix.exs.

The VS Code extension, for example, fetches all dependencies on file change so it’s quite seamless.

2 Likes
17

With the exception of the parameterized deps or deps being loaded from an external file, all of your examples are easy to solve just by looking at the AST. Even the Ecto example is easy to solve once you have some other stuff in place. It’s not that much of a problem with “how do you do that”, but rather with “should it work in every single place you can think of, or should it give up on the not so common cases”. Or put it another way, “is this for every single elixir developer or is this to help onboard people to mix projects like phoenix generators do”? It may as well say

Mix can't recognize your mix.exs file structure. To add <dependency> to
your project, add

    {<dependency>: "~> 1.2.3"}

to your dependency list and then run mix deps.get.

So if it can’t deal with your setup it at least saves you from having to open hex, search for the package and copy paste the code from the browser to your editor.

My other concern, and the reasons I think it should be a third party library(an escript or an editor plugin) are that the Elixir team philosophy may be to work under every circumstance, and that changing the source code is doable but you need to introduce quite a lot of complexity to mix for that single use case, because mix should not only be able to add the dependency tuple, it needs to correctly update your code without messing up the formatting of the rest of the code.

3 Likes
18

It is a bit tedious to go look for the latest version of a library each time one adds a dependency. That said, the issues with adding this feature in the core in my opinion overweight the benefits. If implemented as a library though, I would definitely see myself using it.

4 Likes
19

I actually totally understand your point now, and I’ve had this problem when accidentally stumbling upon old docs on Hex.

I disagree with those saying that a data file is the solution though: mix.exs is so powerful and it’s standard in the ecosystem, I feel like it’d be a huge step backwards to fragment the tooling in the community at this stage. Coming from the disaster that is Python tooling, the standardized tooling is a huge win already.

On another note, the solution being proposed for a mix.exs parser sounds pretty dope.

2 Likes
20

Yep. If implemented as a library, I think it would be perfectly ok to simply read the mix.exs file, assuming that the dependencies are a plain static list of tuples in the deps method (and showing an informative error message otherwise), and add the new one to the list.

It would be up to the user to ensure that they follow this standard, if they choose to use the library. Elixir core cannot make such assumptions, but a library certainly can, as it is opt-in for the user.