tiagodavi

tiagodavi

Compiling is taking too much time to finish while running tests in DOCKER

Hi Guys. I have elixir/phoenix running inside a docker container with docker compose and I realised my project takes much more time to run tests inside docker and it’s needed to compile every single time I run my tests, even by running just one test.

Does anyone know how to solve this problem?

Thanks.

docker-compose.yml

version: "3.4"
services:
  db:
    image: "mdillon/postgis"
    volumes:
      - db_store:/var/lib/postgresql/data
    ports:
      - "5432:5432"

  redis:
    image: "redis:5.0"

  web:
    build:
      context: .
      target: build
      args:
        MIX_ENV: "dev"
    command: '/bin/bash -c "while true; do sleep 10; done;"'
    volumes:
      - .:/opt/app
      - ./priv/ua_inspector:/opt/data/ua_inspector
      - build_cache:/opt/cache
      - elixir_mix:/root/.mix
    depends_on:
      - db
      - redis
    ports:
      - "4000:4000"
    ulimits:
      nofile: 1024
      # memlock: 64
      nproc: 63090
    environment:
      - LOCAL_USER_ID=${LOCAL_USER_ID}
    env_file: .env.development
    tty: true
    stdin_open: true

volumes:
  db_store: {}
  build_cache: {}
  elixir_mix: {}

Dockerfile

#########################################################
## BUILD STAGE - Development image with necessary deps ##
#########################################################

FROM verybigthings/elixir:1.8 AS build

ARG WORKDIR=/opt/app
ARG MIX_ENV=prod
ARG APP_USER=user

ENV MIX_ENV=${MIX_ENV}
ENV WORKDIR=$WORKDIR
ENV APP_USER=$APP_USER
ENV MIX_HOME=/opt/cache/mix
ENV HEX_HOME=/opt/cache/hex
ENV BUILD_PATH=/opt/cache/_build
ENV DEPS_PATH=/opt/cache/deps

RUN apt-get update \
  && apt-get install -y \
  #  wkhtmltopdf \
  xvfb \
  libssl1.0-dev \
  && apt-get clean \
  && rm -rf /var/lib/apt/lists/*

RUN wget https://github.com/wkhtmltopdf/wkhtmltopdf/releases/download/0.12.4/wkhtmltox-0.12.4_linux-generic-amd64.tar.xz
RUN tar vxf wkhtmltox-0.12.4_linux-generic-amd64.tar.xz
RUN cp wkhtmltox/bin/wk* /usr/local/bin/

RUN apt-get update && apt-get install -y \
  make vim less bash git locales inotify-tools postgresql-client

# Tool to propagate singals from the container to the app
ENV PID1_VERSION=0.1.2.0
RUN curl -sSL "https://github.com/fpco/pid1/releases/download/v${PID1_VERSION}/pid1-${PID1_VERSION}-linux-x86_64.tar.gz" | tar xzf - -C /usr/local \
  && chown root:root /usr/local/sbin \
  && chown root:root /usr/local/sbin/pid1

WORKDIR $WORKDIR

RUN mix local.hex --force && \
  mix local.rebar --force

# Set entrypoint
COPY entrypoint.sh /usr/local/bin/entrypoint.sh
RUN chmod +x /usr/local/bin/entrypoint.sh
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]

CMD ["/bin/bash", "-c", "while true; do sleep 10; done;"]

######################################################################################
## PRE-RELEASE STAGE - compiles code and bundles the application to Erlang release  ##
######################################################################################

FROM build AS pre-release

ARG APP_NAME
ARG APP_VSN

ENV APP_NAME=${APP_NAME} \
  APP_VSN=${APP_VSN}

COPY . .

ENV PHOENIX_VERSION 1.4.3
RUN mix archive.install hex phx_new $PHOENIX_VERSION --force

RUN mkdir -p /opt/built \
  && mix do deps.get, compile --warnings-as-errors \
  && cd assets\
  && yarn install && yarn build \
  && cd .. \
  && mix do phx.digest, release --verbose \
  && cp /opt/cache/_build/${MIX_ENV}/rel/${APP_NAME}/releases/${APP_VSN}/${APP_NAME}.tar.gz /opt/built \
  && cd /opt/built \
  && tar -xzf ${APP_NAME}.tar.gz \
  && rm ${APP_NAME}.tar.gz

############################################
## RELEASE STAGE - production application ##
############################################

FROM debian:9-slim AS release

RUN apt-get update \
  && apt-get install -y \
  libssl1.0-dev \
  wget \
  xz-utils \
  libxrender1 \
  libfontconfig \
  libxext6 \
  && apt-get clean \
  && rm -rf /var/lib/apt/lists/*

RUN wget https://github.com/wkhtmltopdf/wkhtmltopdf/releases/download/0.12.4/wkhtmltox-0.12.4_linux-generic-amd64.tar.xz
RUN tar vxf wkhtmltox-0.12.4_linux-generic-amd64.tar.xz
RUN cp wkhtmltox/bin/wk* /usr/local/bin/

# Install Aptible CLI tool.
RUN wget https://omnibus-aptible-toolbelt.s3.amazonaws.com/aptible/omnibus-aptible-toolbelt/master/187/pkg/aptible-toolbelt_0.16.2%2B20190829193224~ubuntu.16.04-1_amd64.deb
RUN dpkg -i aptible-toolbelt_0.16.2+20190829193224~ubuntu.16.04-1_amd64.deb

ARG APP_NAME

ENV LANG C.UTF-8
ENV LC_ALL C.UTF-8

RUN apt-get update && apt-get install -y \
  bash git libpq-dev libjson-c-dev

ENV REPLACE_OS_VARS=true \
  APP_NAME=${APP_NAME} 

WORKDIR /opt/app

COPY ./priv/ua_inspector /opt/data/ua_inspector
COPY --from=pre-release /opt/built .

EXPOSE 4000

CMD trap 'exit' INT; /opt/app/bin/${APP_NAME} foreground

Most Liked

Exadra37

Exadra37

You are using a Dockerfile for build a production release. This is not suitable for development use.

You need to drop a Dockerfile with stages and build everything in one stage.

Also you need to map your entire workdir in the host, and always work inside the docker container.

If you use Elixir in the host and then use inside the docker container you may get weird results because of mismatch in versions of Elixir/Phoenix/Erlang.

See an example in a old learning project of mine:

Dockerfile:

FROM elixir:1.3.4

# Inspired in: http://davidanguita.name/articles/dockerizing-a-phoenix-project/

ARG CONTAINER_USER='elixir'

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && \
    apt-get -y upgrade && \
    apt-get -y install \
        apt-utils \
        locales \
        curl \
        less \
        git \
        zsh && \
    apt install -f && \
    locale-gen en_GB.UTF-8 && \
    dpkg-reconfigure locales && \

    # Intall Oh My Zsh for root user
    bash -c "$(curl -fsSL https://raw.githubusercontent.com/robbyrussell/oh-my-zsh/master/tools/install.sh)" && \

    useradd -m -u 1000 -s /usr/bin/zsh "${CONTAINER_USER}" && \
    su "${CONTAINER_USER}" -c 'bash -c "$(curl -fsSL https://raw.githubusercontent.com/robbyrussell/oh-my-zsh/master/tools/install.sh)"' && \

    # Install NodeJS 6.x and the NPM
    curl -sL https://deb.nodesource.com/setup_6.x | bash - && \
    apt-get install -y -q nodejs

USER "${CONTAINER_USER}"

COPY .gitconfig /home/"${CONTAINER_USER}"/.gitconfig

# This commands will run under user defined above
RUN mix local.hex --force && \
    mix local.rebar --force

# Set the workdir for the container user defined above
WORKDIR /app

docker-compose.yml:

version: "2.1"

# Inspired in: http://davidanguita.name/articles/dockerizing-a-phoenix-project/

services:
  build:
    build: ./docker
    image: exadra37/elixir
    env_file: .env
    volumes_from:
      - app

  shell:
    image: exadra37/elixir
    env_file: .env
    environment:
      - MIX_ENV=dev
    depends_on:
      postgres:
        condition: service_healthy
    links:
      - build
    ports:
      - "4000:4000"
    command:
      - zsh
    volumes_from:
      - app
    volumes:
      - ${SHELL_SSH_VOLUME_MAP:-~/.ssh:/home/elixir/.ssh}
      - ${SHELL_GIT_CONFIG_VOLUME_MAP:-~/.gitconfig:/home/elixir/.gitconfig}

  iex:
    build: ./docker
    image: ${IEX_IMAGE:-exadra37/elixir:1.3.4}
    env_file: .env
    environment:
      - MIX_ENV=dev
    command:
      - iex
    volumes_from:
      - app

  web:
    build: ./docker/build/web
    image: ${WEB_IMAGE:-exadra37/phoenix:1.3.0}
    env_file: .env
    environment:
      - MIX_ENV=dev # That's the environment mode, you know
    depends_on:
      postgres:
        condition: service_healthy
    volumes_from:
      - app
    ports:
      - "4000:4000"
    links:
      - build

  app:
    image: tianon/true
    volumes:
      - ./:/app

  postgres:
    image: postgres
    env_file: .env
    healthcheck:
      test: ["CMD-SHELL", "psql -h 'localhost' -U 'postgres' -c '\\l'"]
      interval: 30s
      timeout: 30s
      retries: 3
    ports:
      - "5432"

I use a much more refined Docker stack nowadays, but is not ready to share… sorry.

Anyway you can use the above as a starting point to adapt to your needs.

cnck1387

cnck1387

Have you looked into using the edge version of Docker Desktop? It has a new way of syncing files using “Mutagen” that brings volume mount performance up to native MacOS speeds.

There’s a good speed comparison of it here: File system performance improvements · Issue #1592 · docker/for-mac · GitHub

Personally I run Windows and use Docker Desktop which is really fast so I have no first hand experience using this but other folks who use MacOS have said they’ve seen good results (I run a Docker course so I tend to keep a pulse on Docker things for every OS).

wolf4earth

wolf4earth

Alternatively you might want to look into the “consistency” options for mounts on Docker for MacOS. They exist to tune performance of mounts on MacOS.

The default here is consistent which is also the slowest option. When mounting local files into a container then I’ve made good experiences with cached:

The macOS host’s view of the mount is authoritative. There may be delays before updates made on the host are visible within a container.

Where Next?

Popular in Discussions Top

blackode
Elixir Upgrading is so Simple in Ubuntu and It worked for me Ubuntu 16.04 git clone https://github.com/elixir-lang/elixir.git cd elixir...
New
Nvim
Anybody knows a comprehensive comparison of Django and Phoenix, thanks for the help. Where are they similar? Where do they differ the m...
New
Nvim
Elixir appears to be a superior language to Python. I don’t see any advantage of Python over Elixir. Are there any?
New
CharlesO
Erlang :list.nth simple, but 1 - based nth(1, [H|_]) -> H; nth(N, [_|T]) when N > 1 -> nth(N - 1, T). Elixir Enum.at … coo...
New
ejpcmac
I have discovered Nix last month and I am currently on my way to migrating to it—both on macOS at home and the full NixOS distrubution at...
New
AstonJ
Please see the new poll here: Which code editor or IDE do you use? (Poll) (2022 Edition) It’s been a while since we first asked this, I...
208 31142 143
New
marciol
Please, let me know if this kind of discussion already took place in another topic . Hi all, how do you consider if is better to build ...
New
saif
Hello everyone, Long time lurker first time poster here. I’ve recently begun working on Elixir full-time again! :raised_hands: It’s been...
New
und0ck3d
Hello everyone! A few days ago I’ve created a topic here about how people were creating CMSs with Elixir and Phoenix. I’ve been studying...
New
RudManusachi
What configs will make sense to put to runtime.exs? – A bit of how I configure apps: I have generic configs in config/config.exs, dev...
New

Other popular topics Top

danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 29377 241
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 43622 214
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 53690 245
New
AngeloChecked
What learn first? Rust or Elixir Hi Elixir community! I’m here because i want learn a new language. I’m a junior developer and mainly i ...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
AstonJ
Seen any cool LiveView demos, sample apps or examples? Please post them here! :003:
New

We're in Beta

About us Mission Statement