Digested assests not loading properly in Phoenix 1.7 - cross building in mac using docker

Questions / Help
, , , , ,
1

Hi,

I recently started a new Phoenix 1.7 and am struggling with generating deployment release using mix release over a mac using docker for deploying on Ubuntu 22.04 server.

I have a project running Phoenix 1.6 and use distillery for generating deployment release. It survived phoenix and esbuild upgrades and has been working fine for about 4 years :slight_smile:

I am using same versions of erlang and elixir as in the old project since I ran into other issues like segmentation fault using newer versions of erlang/elixir combination. Hence, moved to stable versions that I know work fine for now till I am able to get the setup working and then will try to bump up the versions.

I copied most of the working code to current implementation. Finally got pretty much everything working after overcoming multiple challenges. The bit I am struggling with now is that the digetsted assests are not replaced in urls (app.js, app.css) in production.

My setup:

Docker images:

Dockerfile.1ubuntu

ARG UBUNTU_VERSION=noble-20240429

FROM --platform=linux/amd64 ubuntu:${UBUNTU_VERSION}

# Install necessary packages and clean up
RUN apt-get update && apt-get install -y --no-install-recommends \
    ca-certificates \
    curl \
    netbase \
    wget \
    gnupg \
    dirmngr \
    git \
    openssh-client \
    subversion \
    procps \
    autoconf \
    automake \
    bzip2 \
    dpkg-dev \
    file \
    g++ \
    gcc \
    imagemagick \
    libbz2-dev \
    libc6-dev \
    libcurl4-openssl-dev \
    libdb-dev \
    libevent-dev \
    libffi-dev \
    libgdbm-dev \
    libgeoip-dev \
    libglib2.0-dev \
    libjpeg-dev \
    libkrb5-dev \
    liblzma-dev \
    libmagickcore-dev \
    libmagickwand-dev \
    libncurses5-dev \
    libncursesw5-dev \
    libpng-dev \
    libpq-dev \
    libreadline-dev \
    libsqlite3-dev \
    libssl-dev \
    libtool \
    libwebp-dev \
    libxml2-dev \
    libxslt-dev \
    libyaml-dev \
    make \
    patch \
    unzip \
    xz-utils \
    zlib1g-dev \
    gcc \
    g++ \
    make \
    libodbc2 \
    libsctp1 \
    libwxgtk3.0 \
    unixodbc-dev \
    libsctp-dev \
    xsltproc \
    fop \
    libxml2-utils \
    libwxgtk3.2-dev \
&& rm -rf /var/lib/apt/lists/*

Dockerfile.2erlang

FROM --platform=linux/amd64 my_app_ubuntu

ENV ELIXIR_VERSION="v1.14.5"
ENV OTP_VERSION="24.3.4.17"

# Install Erlang
RUN set -xe \
  && OTP_DOWNLOAD_URL="https://github.com/erlang/otp/archive/OTP-${OTP_VERSION}.tar.gz" \
  && curl -fSL -o otp-src.tar.gz "$OTP_DOWNLOAD_URL"

RUN set -xe \
  && export ERL_TOP="/usr/src/otp_src_${OTP_VERSION%%@*}" \
  && mkdir -vp $ERL_TOP \
  && tar -xzf otp-src.tar.gz -C $ERL_TOP --strip-components=1 \
  && rm otp-src.tar.gz \
  && ( cd $ERL_TOP \
    && ./otp_build autoconf \
    && gnuArch="$(dpkg-architecture --query DEB_BUILD_GNU_TYPE)" \
    && ./configure --host="$gnuArch" --target="$gnuArch" --disable-jit \
    && make -j$(nproc) \
    && make install ) \
  && find /usr/local -name examples | xargs rm -rf \
  && apt-get purge -y --auto-remove $buildDeps \
  && rm -rf $ERL_TOP /var/lib/apt/lists/*

Dockerfile.3elixir

FROM --platform=linux/amd64 my_app_erlang

ENV ELIXIR_VERSION="v1.14.5"

# Install Elixir
RUN set -xe \
  && ELIXIR_DOWNLOAD_URL="https://github.com/elixir-lang/elixir/archive/${ELIXIR_VERSION}.tar.gz" \
  && curl -fSL -o elixir-src.tar.gz $ELIXIR_DOWNLOAD_URL \
  && mkdir -p /usr/local/src/elixir \
  && tar -xzC /usr/local/src/elixir --strip-components=1 -f elixir-src.tar.gz \
  && rm elixir-src.tar.gz \
  && cd /usr/local/src/elixir \
&& make install clean

Dockerfile.4mix

# Use the base image with runtime libraries
FROM my_app_elixir as build

ENV REFRESHED_AT=2024-05-15

# set build ENV
ENV MIX_ENV=prod

# Set environment variables for the locale
RUN apt-get update

RUN apt-get install -y locales

RUN sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen && \
    dpkg-reconfigure --frontend=noninteractive locales && \
    update-locale LANG=en_US.UTF-8

ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8

# Install Node.js and Yarn for assets building
RUN curl -fsSL https://deb.nodesource.com/setup_16.x | bash - && \
    apt-get install -y nodejs && \
    npm install -g yarn && \
    apt-get clean && rm -rf /var/lib/apt/lists/*

# Prepare the build directory for Yarn dependencies
# WORKDIR /app/assets

# Copy package.json and yarn.lock to install dependencies
# COPY assets/package.json assets/yarn.lock ./

# Install Yarn dependencies
# RUN yarn install

# Prepare the build directory for the application code
WORKDIR /app

# install hex + rebar
RUN mix local.hex --force && \
    mix local.rebar --force

CMD ["/bin/bash"]

bin/dockerfile_release_config

#!/usr/bin/env bash

set -e

cd /app

# APP_NAME="$(grep 'app:' mix.exs | sed -e 's/\[//g' -e 's/ //g' -e 's/app://' -e 's/[:,]//g')"
# APP_VSN="$(grep 'version:' mix.exs | cut -d '"' -f2)"

export MIX_ENV=prod

# Fetch deps and compile

# Following change is done due to frequent timeouts
# mix deps.get --only prod
HEX_HTTP_CONCURRENCY=1 HEX_HTTP_TIMEOUT=120 mix deps.get --only prod

# Run an explicit clean to remove any build artifacts from the host
mix do clean, compile --force

cd assets
yarn install --ignore-optional && node build.js --deploy
cd ..

mix phx.digest

# Generate the release
mix release

exit 0

Commands run:

env /usr/bin/arch -x86_64 /bin/zsh --login

export DOCKER_DEFAULT_PLATFORM=linux/amd64 && \
docker build --platform=linux/amd64 -t my_app_ubuntu -f Dockerfile.1ubuntu .

export DOCKER_DEFAULT_PLATFORM=linux/amd64 && \
docker build --platform=linux/amd64 -t my_app_erlang -f Dockerfile.2erlang .

export DOCKER_DEFAULT_PLATFORM=linux/amd64 && \
docker build --platform=linux/amd64 -t my_app_elixir -f Dockerfile.3elixir .

export DOCKER_DEFAULT_PLATFORM=linux/amd64 && \
docker build --platform=linux/amd64 -t my_app_mix -f Dockerfile.4mix .


export DOCKER_DEFAULT_PLATFORM=linux/amd64 && \
docker run -v $(pwd):/app -v /app/priv/static/ -v /app/assets/node_modules/ --rm --name my_app_app -it my_app_mix:latest /app/bin/dockerfile_release_config

Then I rsync the genereated release to production server and start the server.

other changes made:

config/config.exs

...

config :dart_sass,
  version: "1.77.0",
  default: [
    args: ~w(css/app.scss ../priv/static/assets/app.css),
    cd: Path.expand("../assets", __DIR__)
  ]

# Configure esbuild (the version is required)
config :esbuild,
  version: "0.16.17",
  default: [
    args:
      ~w(js/app.js --bundle --target=es2019 --outdir=../priv/static/assets --external:/fonts/* --external:/images/*),
    cd: Path.expand("../assets", __DIR__),
    env: %{"NODE_PATH" => Path.expand("../deps", __DIR__)}
  ]

...

build.js

const esbuild = require("esbuild");
const sassPlugin = require("esbuild-sass-plugin").default;
const postcss = require('postcss')
const autoprefixer = require('autoprefixer')
const postcssPresetEnv = require('postcss-preset-env')
const copyStaticFiles = require('esbuild-copy-static-files')

const args = process.argv.slice(2);
const watch = args.includes('--watch');
const deploy = args.includes('--deploy');

const loader = {
  // Add loaders for images/fonts/etc, e.g. { '.svg': 'file' }
  ".woff": "copy",
  ".woff2": "copy",
  ".eot": "copy",
  ".ttf": "copy",
  ".svg": "copy",
  ".png": "copy",
  ".jpg": "copy",
  ".jpeg": "copy",
  ".gif": "copy",
  ".ico": "copy"
}

const plugins = [
  // Add and configure plugins here
  sassPlugin({
    async transform(source, resolveDir) {
      // const {css} = await postcss([autoprefixer, postcssPresetEnv({stage: 1})]).process(source, {from: filePath, to: 'assets/css/app.css'})
      const {css} = await postcss([autoprefixer, postcssPresetEnv({stage: 1})]).process(source, { from: undefined })
      return css
    }
  }),
  copyStaticFiles({
    src: './fonts',
    dest: '../priv/static/fonts',
    dereference: true,
    errorOnExist: false,
    filter: function () { return true },
    preserveTimestamps: true,
    recursive: true
  }),
  copyStaticFiles({
    src: './images',
    dest: '../priv/static/images',
    dereference: true,
    errorOnExist: false,
    filter: function () { return true },
    preserveTimestamps: true,
    recursive: true
  }),
  copyStaticFiles({
    src: './favicon.ico',
    dest: '../priv/static/favicon.ico',
    dereference: true,
    errorOnExist: false,
    filter: function () { return true },
    preserveTimestamps: true,
    recursive: true
  }),
  copyStaticFiles({
    src: './robots.txt',
    dest: '../priv/static/robots.txt',
    dereference: true,
    errorOnExist: false,
    filter: function () { return true },
    preserveTimestamps: true,
    recursive: true
  })
]

// Define esbuild options
let opts = {
  entryPoints: [
    'js/app.js',
    'js/bootstrap.js',
    'js/popper.js',
    'css/app.scss'
  ],
  // entryNames: '[dir]/[name]-[hash]',
  entryNames: '[dir]/[name]',
  bundle: true,
  logLevel: "info",
  target: "es2019",
  outdir: "../priv/static/assets",
  // external: ["*.css", "fonts/*", "images/*"],
  // nodePaths: ["../deps"],
  loader,
  plugins
};

if (watch) {
  opts = {
    ...opts,
    watch,
    sourcemap: 'inline'
  }
}

if (deploy) {
  opts = {
    ...opts,
    minify: true
  }
}

const promise = esbuild.build(opts)

if (watch) {
  promise.then(_result => {
    process.stdin.on('close', () => {
      process.exit(0)
    })

    process.stdin.resume()
  })
}

my_app_web.ex

...

def static_paths, do: ~w(assets fonts images favicon.ico robots.txt)

...

One of the guess work could be some changes need to be done for verified routes… may be

Any help is much appreciated.

Best regards,
Gorav

2

The docker run command is attempting to mount volumes, but the second and third volume flags (-v /app/priv/static/ and -v /app/assets/node_modules/) are creating anonymous Docker volumes, which are ephemeral and will be discarded once the container exits.

  1. -v $(pwd):/app: This mounts the current working directory on your host system to /app in the container. This is correct and ensures that changes made within /app inside the container are reflected in your host system’s current directory.

  2. -v /app/priv/static/: This creates an anonymous volume for /app/priv/static/ inside the container. Any files generated here won’t be accessible from your host system.

  3. -v /app/assets/node_modules/: Similar to the above, this creates an anonymous volume for /app/assets/node_modules/, which will also be discarded when the container stops.

To ensure that the static assets are available in your host system’s current directory, you need to explicitly mount the specific directories where the assets will be generated:

docker run -v $(pwd):/app -v $(pwd)/priv/static:/app/priv/static -v /app/assets/node_modules --rm --name my_app_app -it my_app_mix:latest /app/bin/dockerfile_release_config

or omit it altogether since the first volume mount is already linking everything to the host pwd

docker run -v $(pwd):/app -v /app/assets/node_modules --rm --name my_app_app -it my_app_mix:latest /app/bin/dockerfile_release_config
1 Like
3

@03juan Thank you so much. It works finally :heart_eyes:

Feel like hitting myself for overlooking it!!!

1 Like
4

Glad that helped! It’s not always easy to remember each detail of all our tools and how they affect things. Good luck with the rest