Mix deps.get eats all memory

Questions / Help
1

i have the same issue like here Hex v0.20.2-5 eats all memory and dies, but maybe it is us - #4 by outlog

When i try to execute mix.deps.get after removing mix.lock, the terminal hangs at Resolving Hex dependencies... . The process then starts to eat all available memory and swap …

my-app|qa⚡ ⇒ mix hex.info
Hex:  0.20.5
Elixir: 1.9.4
OTP:  22.1.4

Built with: Elixir 1.9.4 and OTP 22.1.4

my-app|qa⚡ ⇒ mix deps.get                                    
* Updating google_api_storage (https://github.com/DinaWork/google-api-storage.git)
* Updating google_auth (https://github.com/DinaWork/google-auth.git)
* Updating paranoid (https://github.com/echobind/paranoid.git)
Resolving Hex dependencies...

it’s umbrella with ~30 apps

i think it started after i update hex to 0.20.2
but now, even if i trying to change it back to lower version - i still have the same issue.

2 Likes
2

Those issues are caused by the dependency resolution algorithm. It means the search space is too big and finding suitable solutions takes too long. The simplest suggestion to address this is to constraint your dependencies. For example, if you have Ecto ~> 3.0, update to latest Ecto ~> 3.6 and so on.

If you have a lock file, then update your deps in groups, doing mix deps.update phoenix ecto instead of passing the --all flag.

We have some possible improvements in mind for dependency resolution but, if the search space is too big, ultimately there won’t be much we can do.

7 Likes
3

I have a similar issue like yours on an 8GB RAM Ubuntu 21.0.4 system.

mix deps.get hangs at Resolving dependencies until the terminal gets tired and kills it.

I’m trying to set up a new Pheonix Liveview Project using Pheonix 1.5.12 and Elixir 1.10.3 (compiled with Erlang/OTP 22).

ANY help would be appreciated.

1 Like
4

you have an answer from @josevalim… Mix deps.get eats all memory - #2 by josevalim this is the solution.

1 Like
5

A key point I learned from this thread is that if you have a mix.lock file already, then this dependency resolution isn’t necessary, so you won’t run out of memory.

So if you’re having this problem on your server, try running on your development machine and committing the mix.lock.

(I came across this thread while trying out deploying a phoenix app on a tiny free google cloud server, and my first thought was to just run phx.new and fire it up, which didn’t work because it didn’t have the memory to run mix deps.get)

1 Like
6

Got this issue yesterday after deleting the mix.lock because of version conflict.

I commented all my deps, decommenting each deps, one at a time, and calling mix.deps get each time :smiley:

7

This is the right approach, you should always commit your mix.lock file.

6 Likes
8

We are working on a fix to this issue but it requires a full rewrite of the resolver algorithm so unfortunately it will take some time until it is released and in production quality.

In the mean time make sure you always commit your mix.lock file and follow the instructions here if you run into issues.

7 Likes
9

sure, we always commit it.
but sometimes, if you have dependencies version conflict - you need to remove the .lock file …

in my case - when i had to remove the lock file - i got this issue in my local mac (not in dock/any deployment env) - so @josevalim answer solved it for me, (updating the Ecto version was enough)

10

You can also unlock individual dependencies using mix deps.unlock package.

1 Like
11

You don’t have to remove the mix.lock when you have conflicts. Mix won’t write a lockfile if the resolution has conflicts. Instead unlock or update the individual deps that are conflicting.

3 Likes
12

Hey,
I am experiencing the same problem, on the deployment docker file even when I run it locally.
I identified that id I comment out these 3 deps, then we are ok, but the 3 together causes the problem.
I have a mix.lock

      {:ex_aws, "~> 2.1.6", override: true},
      {:quantum, "~> 3.3.0"},
      {:chromic_pdf, "~> 0.7.2"}

I am wondering that the docker file that I inherited might causes this partially because locally I can get the deps without any issues, only I am trying to build the container I am getting the problem.

I think this part is the relevant:

RUN
mix local.hex --force &&
mix local.rebar --force &&
mix clean &&
mix deps.clean --all &&
mix deps.get &&
mix deps.compile

Do any of these steps mess with the lock file?
I read the docs for the cleaning steps but I am still unsure.

Thanks

13

try to change the dependencies to fixed version - “== 2.1.6” instead “~> 2.1.6”

14

IIRC none of them should mess up the lock. Can you double check you are copying the mix.lock file to the Docker image before running those commands?

1 Like
15

Wow it is not there, but why?
The image is doing:

ADD . /application
WORKDIR /application/project_umbrella

And when I check before getting the deps with ls the following files/folders are there:

apps
config
mix.exs

16

My Docker knowledge is very limited but I think you need to explicitly copy all files or something?

17

Ok interesting, I checked the .dockerignore and the lock file was added there for some reason.

# prevent overwriting mix.lock file in base image
project_umbrella/mix.lock

If I remove it the deps are coming along nicely.
This should be safe to remove from the ignore right?

18

This:

I hadn’t come across your answer, but from the earlier posts I realized a similar thing. If the mix.lock file is gone, it will throw that error it seems. Simply comment out all but one dependency. Run a deps.get, then you can likely uncomment all the rest (I uncommented in groups of about 5 or so). I’d guess that having at least one in a lock file is the key to circumventing this search space issue.

19

I’ve just had this issue while upgrading from Phoenix 1.5 to 1.6 and, believe it or not, deleting the mix.lock did the trick.

20

This worked for me

1 Like