Disk usage growing on dokku

Questions / Help
1

Just deployed a couple phoenix apps to a dokku droplet and loving it so far. The only thing is that disk usage seems to be steadily growing for each, until I do a dokku ps:rebuild, at which point it sharply drops. Has anybody had this problem before? Dokku github issues seemed to suggest app logs, but I thought Phoenix just wrote those to stdoud…

Any suggestions appreciated…thanks!

2

Not familiar with dokku, is it based off of something like https://github.com/HashNuke/heroku-buildpack-elixir ?

3

Yep, it’s a user-administered heroku clone (ish) and I’m using that buildpack.

4

Then it definitely sounds like it could be: https://github.com/HashNuke/heroku-buildpack-elixir/issues/115

My workaround was adding /app/_build/build_backup/build_backup to .slugignore, (https://github.com/HashNuke/heroku-buildpack-elixir/issues/115#issuecomment-399313162) not sure if that is more difficult with dokku

5

Hmm, interesting. Thanks for looking into it. With this issue, your disk space was growing on each deploy, right? My problem seems to be that it’s growing even without being redeployed…

6

Ah, I see. I guess I misread your initial post then. I’m out of ideas in that case.

7

Without knowing your application its hard to tell.

Do you have any IO in your project? Do you possibly create a lot of large binaries which need to be swapped, while the system is configured to use a swap-file? Other temporary data on the host like file uploads before the get pushed to AWS or similar that you do not delete?

If you are able to, run du -sh /* right after starting and again and again after a couple of hours each (or any other timescale suitable to your problem). As you identify the growing folder dig deeper into it, until you are deep enough to identify the location where the data is gathered.

Perhaps you are able to identify the owner of the data by the name of the location or because of the filesystem ownership, perhaps you can identify it from the kind of the data. Perhaps it gives you another thing you can google for.

8

Turns out it was Ecto failing to connect to Postgres and spamming the logs. Thanks for your suggestions!

2 Likes
9

I’m glad you got it figured out! And thanks for reporting back to the thread (which will help Elixir developers in the future).

1 Like
10

Glad you found the solution I’ve found that cleaning up old containers and unused images also helps:

docker rm $(docker ps --all -q -f status=exited)
docker rmi $(docker images --filter "dangling=true" -q --no-trunc)
1 Like
11

There is also docker system prune and docker system prune --all.

3 Likes