Yeah, that should do it.
I’m not sure how docker implements volumes, but I can tell you what happens when you use one.
Everything what you had in the container at the point where you mount the volume to, will dissapear and replaced by what you have on the host. Changes you make from the container in the volume, appear on the host as well and changes made on the host do appear in the container as well.
The most common use case for volumes is persisting data from the container, eg databases are writing their files on volume, such that the data persists even between container updates.
Also I’ve seen volumes beeing used as a kind of IPC, via a file that both (host and container) read and write. I tend to replace those ugly mechanics with proper signal handling or a management protocol over IP/REST on a dedicated port.
They are not meant to share sourcecode during development. If changes of your sourceode on the host are visible in the container instantly without needing it to rebuild, you are doing it wrong. Especially as build artifacts are created by host simply by starting the editor (at least in the days of ElixirLS) which might confuse the container as it sees those artifacts and thinks it doesn’t need to recompile, but in fact the artifacts might be for a different ARCH or OS or just with a different environment.
2 Likes
Here’s what I’m thinking. So ever since all this worked, I was thinking of doing a bind mount for my dependencies from outside my container and referencing my dependencies inside the container without building from within the container. So my question is how do I do so. I attempted the bind mount, but the problem is that my phoenix commands inside the docker file does mix ecto.create && mix ecto.migrate && mix run priv/repo/seeds.exs && mix phx.server and these 4 commands automatically compile my dependencies. To me, this is a problem because my dependencies don’t exist inside the container. Do you have any suggestion on this?
Here is my docker-compose.yml
version: '3.6'
services:
phoenix:
environment:
GOOGLE_CLIENT_ID: ${GOOGLE_CLIENT_ID}
GOOGLE_CLIENT_SECRET: ${GOOGLE_CLIENT_SECRET}
FACEBOOK_CLIENT_ID: ${FACEBOOK_CLIENT_ID}
FACEBOOK_CLIENT_SECRET: ${FACEBOOK_CLIENT_SECRET}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_USER: ${POSTGRES_USER}
PUBSUB_KEY: ${PUBSUB_KEY}
GUARDIAN_KEY: ${GUARDIAN_KEY}
# tell docker-compose which Dockerfile it needs to build
build:
context: .
dockerfile: Dockerfile.phoenix.development
# map the port of phoenix to the local dev port
ports:
- 4000:4000
# mount the code folder inside the running container for easy development
volumes:
- ./my_app/_build:/app/_build
- ./my_app/deps:/app/deps
# make sure we start mongodb when we start this service
depends_on:
- db
- redis
go:
build:
context: .
dockerfile: Dockerfile.go.development
ports:
- 8080:8080
volumes:
- ./genesys-api:/go/src/github.com/sc4224/genesys-api
depends_on:
- db
- redis
- phoenix
db:
container_name: db
hostname: db
image: postgres:latest
ports:
- "5432:5432"
environment:
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_USER: ${POSTGRES_USER}
volumes:
- postgres-data:/usr/local/var/postgres
restart: always
redis:
container_name: redis
image: redis:latest
ports:
- "6379:6379"
volumes:
- redis-data:/data/redis
entrypoint: redis-server
restart: always
volumes:
postgres-data:
redis-data:
Here is my dockerfile for phoenix
# base image elixer to start with
FROM elixir:1.6
# install hex package manager
RUN mix local.hex --force
RUN mix local.rebar --force
# install the latest phoenix
RUN mix archive.install https://github.com/phoenixframework/archives/raw/master/phx_new.ez --force
# create app folder
RUN mkdir /app
COPY ./my_app /app
WORKDIR /app
# install dependencies
# RUN mix deps.get
# RUN mix deps.compile
# (cd deps/bcrypt_elixir && make clean && make) &&
# WORKDIR /app
# run phoenix in *dev* mode on port 4000
# mix ecto.create && mix ecto.migrate
CMD mix run priv/repo/seeds.exs && mix phx.serverI don’t get this…
Hoiw do you want to use them, when you don’t build them?
Then you need to mix deps.get them first.
Again, you do not want to have anything in the container that is owned or built by the host.
Containers are meant to be as isolated and immutable as possible.
If you want to share your code between host and guest to develop on host and test on the guest, then there are better alternatives that allow this while beeing able to unshare subfolders (vagrant with rsync share is an example).
What I meant was that your dependencies are built outside by running mix deps.compile outside the container, but I reference my built dependencies by using the bind mount in my docker-compose.yml file
And this is exactly what you not want in docker. In docker you want to build everything in the container to avoid exactly the issues you had.
ok thanks!
I have a follow up question to this. Because I build everything within the container, I have to wait very very long before it finally builds with the dependencies. Is there a way to shorten this build time? My dockerfile looks like this now:
# base image elixer to start with
FROM elixir:1.6
# install hex package manager
RUN mix local.hex --force
RUN mix local.rebar --force
# install the latest phoenix
RUN mix archive.install https://github.com/phoenixframework/archives/raw/master/phx_new.ez --force
# create app folder
RUN mkdir /app
COPY ./my_app /app
WORKDIR /app
# install dependencies
RUN mix deps.get
# RUN mix deps.compile
# (cd deps/bcrypt_elixir && make clean && make) &&
# WORKDIR /app
# run phoenix in *dev* mode on port 4000
CMD mix ecto.create && mix ecto.migrate && mix run priv/repo/seeds.exs && mix phx.serverAs you usually should only rebuild the container on publishing a new release, compiletimes are not important.
If though you use the container for development you could in theory use an approach roughly like this:
# create app folder
RUN mkdir /app
WORKDIR /app
COPY mix.exs mix.lock /app
RUN mix do deps.get, deps.compile
COPY . /app
CMD …
This way, dependencies should only get rebuilt when the mix-files got changed.
Also, you do not need to install phx_newin the container, it is really only necessary when creating an application for the first time, after that everything necessary is self contained in the application and its dependencies.
Do you mean that I do not need to install phoenix in the container? That line in my dockerfile specifies to install phoenix, so if i remove that line, I can’t run mix phx.server in the container
Have you tried it? phx.server is not even part of phx_new, it’s part of the phoenix packages downloaded as dependencies.
Tried it, it works! Merry Christmas. Big follow up to that, why do we need to copy over a mix.lock file? Shouldn’t the mix.lock file be generated after doing a mix deps.get at build time? It seems when I don’t copy it over, the build process returns with an error saying the mix.lock file does not exist and that I should run mix deps.get to generate one. But the problem is, mix deps.get is already set as a command in the dockerfile. So I don’t understand why the mix.lock doesn’t exist at runtime
There shouldn’t be any need.
But copying it from the host (and also committing it into the repository) ensures using consistent versions of dependencies across machines.
I’m not sure if I can follow you with the remainder of you question. Can you perhaps open a new thread which includes a link to an example repository and an explanation about how to recreate your problem?
No, you are doing it right. I am using this approach for years for all programming languages I use or anything else, because I run everything from Docker, included for example Android Studio, that’s an heavy editor with lot’s of cache and magic going on 
Now here is where things are “wrong”. If you have two different systems building stuff and try to use them, then off-course things will go definitely side ways.
Also inotify-tools must be installed in both the host and in the container, and their limits increased so that both host and container can react to file changes in the mounted volumes.
If, and only if, you know what you are doing, and if, and only if, you know that built artifacts from host and container won’t mix, then the volume approach can work. Though, very few people do really know.
Well what it seems to me here is that people are mixing compiling from the host and compiling from the container interchangeable, and that will not work unless the operating systems are exactly the same.
So if people want to build artifacts in both host and container they must exclude that folders from being mapped in the volumes.
bcrypt_elixir
Microsoft ® Program Maintenance Utility Version 14.27.29112.0
Copyright © Microsoft Corporation. All rights reserved.
del /Q /F priv
erl -eval "io:format(\"~s~n\", [lists:concat([\"ERTS_INCLUDE_PATH=\", code:root_dir(), \"/erts-\", erlang:system_info(version), \"/include\"])])" -s init stop -noshell > Makefile.auto.win
nmake / /F Makefile.win priv\bcrypt_nif.dll
Microsoft ® Program Maintenance Utility Version 14.27.29112.0
Copyright © Microsoft Corporation. All rights reserved.
if NOT EXIST "priv" mkdir "priv"
cl /O2 /EHsc /I"c_src" /I"c:/Program Files/erl-23.0/erts-11.0/include" /LD /MD /Fepriv\bcrypt_nif.dll c_src\bcrypt_nif.c c_src\blowfish.c
Microsoft ® C/C++ Optimizing Compiler Version 19.27.29112 for x64
Copyright © Microsoft Corporation. All rights reserved.
bcrypt_nif.c
c_src\bcrypt_nif.c(52): fatal error C1083: Cannot open include file: ‘stdio.h’: No such file or directory
blowfish.c
e:\projects\phoenix-live-stream-master\deps\bcrypt_elixir\c_src\blf.h(37): fatal error C1083: Cannot open include file: ‘stdint.h’: No such file or directory
Generating Code…
NMAKE : fatal error U1077: ‘“C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\Hostx64\x64\cl.EXE”’ : return code ‘0x2’
Stop.
NMAKE : fatal error U1077: ‘“C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\Hostx64\x64\nmake.EXE”’ : return code ‘0x2’
Stop.
could not compile dependency :bcrypt_elixir, “mix compile” failed. You can recompile this dependency with “mix deps.compile bcrypt_elixir”, update it with “mix deps.update bcrypt_elixir” or clean it with “mix deps.clean bcrypt_elixir”
==> chat_app
** (Mix) Could not compile with “nmake” (exit status: 2).
One option is to install a recent version of
Visual C++ Build Tools
either manually or using Chocolatey -
choco install VisualCppBuildTools.
After installing Visual C++ Build Tools, look in the “Program Files (x86)”
directory and search for “Microsoft Visual Studio”. Note down the full path
of the folder with the highest version number. Open the “run” command and
type in the following command (make sure that the path and version number
are correct):
cmd /K "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" amd64
This should open up a command prompt with the necessary environment variables
set, and from which you will be able to run the “mix compile”, “mix deps.compile”,
and “mix test” commands.
Ques: What may be the problem running this on Windows 10 cmd
This worked for me:
mix deps.compile --force bcrypt_elixir
1 Like
Every time I come across this, if I blow away my _build and recompile, the problem is resolved.
