What’s the proper way to have local only packages that don’t interfere with git / rebasing / changing branches / recompilation?

Questions / Help
1

Hello!

I’m wondering what’s the proper way to have local only packages that don’t interfere with git / rebasing / changing branches / recompilation.

I’ve seen Proper way to have a local, non-commitable addition to runtime.exs, but this is only about configuration.

For example, I’d like to install the following packages for my “personal” use (until at least the use case is validated to train other colleagues on them):

  • dialixir
  • mix-test.watch (which is the latest package I wanted to add for my own quality of life)

Until now, I’ve added them in my local env in mix.exs but this is tedious when changing branches, they get in the way of my stashes, etc. Every time I rebase/change branch, I have to recompile the whole project (because mix.exs changed), and 1000+ files is long.

I’ve invested some time this morning to figure out if there would be a more dynamic approach, e.g. by having a .gitigored file, let’s call it mix_local.exs.

What I have for now:

defmodule Project.Mixfile do
  use Mix.Project

  def project do
    local_mix =
      if File.exists?("mix_local.exs") do
        Code.require_file("mix_local.exs")
        Project.MixfileLocal
      else
       %{
          project: fn -> [] end,
          deps: fn -> [] end
        }
      end

    [
       # ...
       deps: deps() ++ local_mix.deps()
    ] ++ local_mix.project()
  end

defmodule Project.MixfileLocal do
  def project do
    [
      lockfile: "mix_local.lock"
    ]
  end

  def deps do
    [
      {:mix_test_watch, "~> 1.0", only: [:dev, :test], runtime: false}
    ]
  end
end

It’s a good start, I can add my own dependencies, but now the mix_local.lock file contains all the dependencies from the normal config + the newly added one. This means that when I pull an updated mix.lock it won’t be used and I risk differences in the lockfiles.

Would there be a way for my mix_local.lock to only contain the dependencies that I added in mix_local.exs ?

1 Like
2

Why not install them without training colleages first? They won’t do anything without using them actively.

3

Honestly, fair answer. But for the sake of completeness, I’d be very much interested if there’s a better solution.

4

The intractable problem here is that if each developer has their own “hidden” dependencies then these dependencies still influence what shows up in the regular mix.lock file even if you manage to segregate the actual deps themselves into a different file.

For example if you pull in a dependency for doing useful Ecto queries this may require certain versions of Ecto, but other developers may pull in that same dependency at a newer version that requires a later ecto version.

2 Likes
5

Yeah, dependency version resolution only works when knowing the full set of dependencies and their constraints. That’s also why umbrellas only have a single lock file for many applications.

2 Likes
6

All that makes perfect sense, thank you both for your answers :slight_smile:

7

@benwilson512 and @LostKobrakai are completely right about why a full mix_local.lock is needed and I highly recommend all devs using the same common deps.

However I still got curious about making this slightly more ergonomic since this is a pain point I have as well. Here’s the best I can come up with at the moment. The benefit of this approach is that you don’t need to define the entire package configuration and there are no compilation warnings from module calls that may not have been defined (e.g. if mix_loca.exs was not defined).

Here’s the individual snippets:

@use_local_deps System.get_env("USE_LOCAL_DEPS") == "1" && File.exists?("local_deps.exs")

# in def project
lockfile: if(@use_local_deps, do: "mix_local.lock", else: "mix.lock"),

defp deps do
  [
    ...common deps
  ]
  |> Enum.concat(local_deps())
end

if @use_local_deps do
  defp local_deps do
    Code.require_file("local_deps.exs")
    DataTracer.LocalDeps.local_deps()
  end
else
  defp local_deps, do: []
end

Here’s a pull request with full example code:

Also as a side-note if your local deps don’t have any deps of their own then you could add your local deps as path deps and continue to use the main mix.lock since path deps aren’t added to mix.lock.

2 Likes