So I’m working on docker-izing a LiveView side-project of mine, and I’m having some struggles with code reloading.
Specifically, when I make changes to a LiveView project file, the app simply won’t recompile, throwing the following error:
** (RuntimeError) could not compile application: docker_example.
You must restart your server after changing the following config or lib files:
* config/dev.exs
* config/config.exs
* mix.exs
(phoenix 1.5.4) lib/phoenix/code_reloader/server.ex:211: Phoenix.CodeReloader.Server.mix_compile_unless_stale_config/1
(phoenix 1.5.4) lib/phoenix/code_reloader/server.ex:196: Phoenix.CodeReloader.Server.mix_compile_project/3
(phoenix 1.5.4) lib/phoenix/code_reloader/server.ex:75: anonymous fn/2 in Phoenix.CodeReloader.Server.handle_call/3
(phoenix 1.5.4) lib/phoenix/code_reloader/server.ex:262: Phoenix.CodeReloader.Server.proxy_io/1
(phoenix 1.5.4) lib/phoenix/code_reloader/server.ex:73: Phoenix.CodeReloader.Server.handle_call/3
(stdlib 3.12.1) gen_server.erl:661: :gen_server.try_handle_call/4
(stdlib 3.12.1) gen_server.erl:690: :gen_server.handle_msg/6
(stdlib 3.12.1) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
I’ve been able to reproduce this in a fresh mix phx.new app_name --live --no-ecto project using the following files:
Dockerfile
FROM elixir:1.10
RUN apt-get update
RUN apt-get install --yes build-essential inotify-tools
# Set working directory
WORKDIR /app
# Install hex and rebar
RUN mix local.hex --force &&\
mix local.rebar --force
# Install nodejs
RUN curl -sL https://deb.nodesource.com/setup_12.x | bash - &&\
apt-get install --yes nodejs
# Set the run command
CMD ["./run.sh"]
With those files added to the project root, and after starting the project with docker-compose up --build, making a change to the generated lib/docker_example_web/live/page_live.ex file usually results in a compilation error. It sometimes reloads as it should, but I’m generally able to get it to fail compilation after saving a couple of times.
Additionally, when serving the app with mix phx.server, code reloading works flawlessly, so I’m fairly confident this issue is Docker-related.
Anyone have any idea what’s going on here? I’ve thrown a ton of different Dockerfile & docker-compose.yml setups at this problem to no avail.
Your given code does not even run for me. I get permission denied error when container tries to run run.sh.
Your general approach seems a bit wrong. There should be a COPY statement somewhere in your Dockerfile. Also you should mix deps.get and mix deps.compile as part of the build.
Try this setup:
Dockerfile:
FROM elixir:1.10
RUN curl -sL https://deb.nodesource.com/setup_12.x | bash -
RUN apt update
RUN apt install -y git nodejs inotify-tools
RUN apt install -y chromium-driver
RUN mkdir -p /app
WORKDIR /app
RUN mix local.hex --force && \
mix local.rebar --force && \
mix archive.install --force hex phx_new
COPY mix.exs .
COPY mix.lock .
# copy the deps in dev environment for faster builds
COPY deps ./deps
RUN ["mix", "deps.get"]
RUN ["mix", "deps.compile"]
COPY assets ./assets
WORKDIR /app/assets
RUN ["npm", "install"]
WORKDIR /app
COPY config ./config
COPY lib ./lib
COPY seeds ./seeds
COPY priv ./priv
COPY test ./test
COPY dev/support ./dev/support
RUN ["mix", "compile"]
# compile deps in test environment for faster test runs when built
RUN export MIX_ENV=test && mix deps.compile
COPY ./run.sh ./run.sh
CMD ["/bin/bash", "entrypoint.sh"]
I don’t know why your setup had the problem with config or mix.exs being “altered” but I suspect it has something to do with how your never actually COPY the files during your build.
Disclaimer: there is no one correct way to use docker. It depends on your own needs. What I have provided is good for local :dev environment, local debugging.
For development never use the COPY or ADD command in your Dockerfile, just map the the required folders in docker-compose.yml file. Also using using version 3.* in docker compose is only necessary if your are running the application in a docker swarm, otherwise just stick with the normal 2.* version that are more suitable for normal docker flow in development.
Because you then need to rebuild the container each time you change a file, while mapping them in docker-compose.yml always reflects immediately in the docker container the changes done in host.
So I don’t have to build every time I run the container. With my approach, your built image has all the deps compiled and the code from the time you built the image. The dev can add new deps and edit the code and still use the same image because the volume mounting will change what is necessary. Note how I use ro with the volume mounting to get read-only access, so container cannot edit source code.
I don’t have to build every time I run the docker container, because I map the entire app I am working on to the container, thus I have the exact same dev flow I would have if I was not using docker.
Sure, if the deps are new, then you would have to rebuild. But what I look for is the smallest possible "boot"time. I don’t want anything being compiled or fetched just because I docker-compose down to go to the shop for 10 minutes. How often do you actually add/remove new deps when developing? Not even once per day, on average? If you don’t COPY anything during build phase then every time you spin up a new container, you have to fetch and compile all deps and compile the app? That seems like inefficient dev workflow to me. I would rather be able to spin up a new container in 5 seconds. I don’t mind rebuilding for new deps. That is a fair price for me.
Like I said, I think there are different approaches for using docker in dev environment.I don’t think there is one true way. I also don’t think it’s good to have absolute rules such as “never use the COPY…command…[for development]”.
You should use COPY in development, because ideally you’d have 1 Dockerfile that is identical for both development and production and you can use multi-stage builds to make your production image smaller in the end.
Then for having a dev environment that’s developer friendly, you can set up a volume mount in docker-compose.yml because a volume mount will override what you COPY in, if it’s the same destination directory.
If you really wanted to focus on having the same Dockerfile / docker-compose.yml file in both dev and prod (which I highly recommend), you can define that volume with an environment variable and have the prod one lead to an unused directory.
Order of operations in your Dockerfile should be this as separate layers:
Copy in mix / lock files
Install dependencies
Copy in all of your code
This way you don’t need to rebuild all of your dependencies when you build a new image that only contains changing a few lines of application code (such as editing an EEx template).
If you map the entire app folder in the docker compose file you don’t loose your compiled dependencies when you do docker compose down or even if you remove/rebuild the docker image. Therefore it also doesn’t affect the boot time. You just need to to do the usual mix deps.get && mix compile when you use the app after the first time you build the docker image, but even this can be automated as I do in my docker projects.
Well I don’t like to use that word in my life, and should not have used it here, but I still stand for my principle that I don’t see it as a good fit for any development scenario.
I my opinion the COPY or ADD commands are very useful when building a release of the app inside a docker container, other them that is just getting in my way.
Your given code does not even run for me. I get permission denied error when container tries to run run.sh .
Ah right, forgot to mention: I had to give the run.sh file execute permissions with chmod +x run.sh.
Try this setup
This appears to have fixed my problem!
There were a couple of things I had to tweak to fit my setup, since it looks like the Dockerfile & docker-compose.yml you provided had some extra goodies I didn’t have/need (e.g. the chromium-driver & a seeds directory). I also had some problems with docker-compose up failing due to the assets directory volume being set to read-only, but removing the :ro fixed that.
Another quick note is that, because I’m on MacOS, COPYing existing dependencies and compiled artifacts can cause problems. This was solved by rm -rfing the appropriate directories before building the container.
Anyway, here’s the final result:
Dockerfile
FROM elixir:1.10
RUN apt-get update
RUN apt-get install --yes build-essential inotify-tools
# Set working directory
RUN mkdir -p /app
WORKDIR /app
# Install hex, rebar, and phx_new
RUN mix local.hex --force && \
mix local.rebar --force && \
mix archive.install --force hex phx_new
# Copy and install the elixir deps
COPY mix.exs .
COPY mix.lock .
RUN mix deps.get && mix deps.compile
# Install nodejs
RUN curl -sL https://deb.nodesource.com/setup_12.x | bash - &&\
apt-get install --yes nodejs
# Copy and install the nodejs deps
COPY assets ./assets
WORKDIR /app/assets
RUN npm install
WORKDIR /app
# Copy over compiled files
COPY config ./config
COPY lib ./lib
COPY priv ./priv
COPY test ./test
# Compile the app
RUN mix compile
# Compile deps in test environment for faster test runs when built
RUN MIX_ENV=test mix deps.compile
# Set the run command
CMD ["mix", "phx.server"]
My suspicion is that the root cause of my issue was not having a volume for my config directory. Maybe something about those files living in the container was causing the CodeReloader to freak out? Who knows.
Thanks for the help! This had been driving me crazy for a while.
My suspicion is that the root cause of my issue was not having a volume for my config directory. Maybe something about those files living in the container was causing the CodeReloader to freak out? Who knows.
This is exactly what solved the same problem on my setup. I didn’t copy anything in Dockerfile, just added bunch of volumes based on @slouchpie snippets.
What a coincidence, I’m just working also with LiveView and just noticed that the live reloading doesn’t work.
While everything was smooth with regular Phoenix controllers.
In my case, I always set a parent folder where I have all docker related files alongside a directory of the Phoenix project where the whole parent folder is set as a volume.
And in development I have that whole my_project folder as a volume.
It might be a little odd to have a different dockerfile for dev and prod, but there’s two reason why I’m doing like this:
In dev I’m using VSCode remote container capabilities, so somehow very different from the prod
For prod I’m using multi-stage builds and producing a release for the last step
Anyway, I have no idea why the LiveReload is not working.
At first I thought that I didn’t opened the Live Reload port (usually 35729) but then I noticed that here with phoenix the LIve Reload is done within the same port, and indeed the livereload WS correctly connects.
Then I suspect it’s about inotify (I’m on linux), but then I noticed that when I save a CSS file for example, it correctly triggers the asset pipeline build.
I then suspected that the LiveView files were not defined in the pattern of the live_reload config in the Endpoint. But it is.
So I have no clue and don’t fix yet the live reloading (which is a little annoying)
I’ve seen this class of issues so many times - attempt to create a development environment that will work across all developer machines in order that they don’t have to deal with the complexity of installing Elixir/Erlang and setting up a developer environment.
I don’t recommend this approach - make it so the developer only needs to run Docker in order to debug CI issues - for everything else, developer should be just running iex -S mix phx.server locally, add a Makefile with commands to run postgres in docker and expose the port, same for any other service dependencies.
