Mix release - CI / CD Pipeline suggestions

I’m studying elixir for a while and never had a chance to use it in production on my day to day job.

Recently I’ve decided to put one of my phoenix side projects in “production”, by hosting a server a and config all the CI/CD on Github Actions to see how it works with elixir.

I’ve ended up with a pipeline such as this:

name: Server Deploy SDX
on:
  push:
    branches: ['sdx']
  pull_request:
    branches: ['sdx']

jobs:
  CI-CD:
    runs-on: ubuntu-latest
    env:
      MIX_ENV: test
      DATABASE_URL_SDX: ${{ secrets.DATABASE_URL_SDX }}
      SECRET_KEY_BASE_SDX: ${{ secrets.SECRET_KEY_BASE_SDX }}
      PORT_SDX: 7001

    services:
      db:
        image: postgres:12.8
        ports: ['5432:5432']
        env:
          POSTGRES_PASSWORD: postgres
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    steps:
      - uses: actions/checkout@v2
      - name: Set up Elixir
        uses: erlef/setup-elixir@885971a72ed1f9240973bd92ab57af8c1aa68f24
        with:
          elixir-version: '1.12.2'
          otp-version: '24.0'

      - name: Restore Dependencies Cache
        uses: actions/cache@v2
        with:
          path: deps
          key: ${{ runner.os }}-mix-${{ hashFiles('**/mix.lock') }}
          restore-keys: ${{ runner.os }}-mix-

      - name: Install dependencies
        run: mix deps.get

      - name: Run Tests
        run: mix test

      - name: Run Migrations SDX
        run: MIX_ENV=sdx mix ecto.migrate

      - name: Build SDX Release
        run: MIX_ENV=sdx mix release

      - name: Install SSH key
        uses: shimataro/ssh-key-action@v2
        with:
          key: ${{ secrets.APPLICATION_SERVER_SSH_KEY_SDX }}
          known_hosts: 'to be defined on next step'

      - name: Add Known Hosts
        run: ssh-keyscan -H ${{ secrets.APPLICATION_SERVER_HOST_SDX }} >> ~/.ssh/known_hosts

      - name: Rename Old Build Folder
        uses: appleboy/ssh-action@master
        with:
          host: ${{ secrets.APPLICATION_SERVER_HOST_SDX }}
          username: ${{ secrets.APPLICATION_SERVER_SSH_USER_SDX }}
          key: ${{ secrets.APPLICATION_SERVER_SSH_KEY_SDX }}
          script_stop: true
          script: mv _build _build_old
      
      - name: Stop Old Application
        uses: appleboy/ssh-action@master
        with:
          host: ${{ secrets.APPLICATION_SERVER_HOST_SDX }}
          username: ${{ secrets.APPLICATION_SERVER_SSH_USER_SDX }}
          key: ${{ secrets.APPLICATION_SERVER_SSH_KEY_SDX }}
          script_stop: true
          script: _build_old/sdx/rel/alfred_server/bin/alfred_server stop

      - name: Deploy Release with RSYNC 
        run: rsync -avz ./_build ${{ secrets.APPLICATION_SERVER_SSH_USER_SDX }}@${{ secrets.APPLICATION_SERVER_HOST_SDX }}:~

      - name: Start Application
        uses: appleboy/ssh-action@master
        with:
          host: ${{ secrets.APPLICATION_SERVER_HOST_SDX }}
          username: ${{ secrets.APPLICATION_SERVER_SSH_USER_SDX }}
          key: ${{ secrets.APPLICATION_SERVER_SSH_KEY_SDX }}
          script_stop: true
          script: echo ${{ secrets.APPLICATION_SERVER_USER_PWD_SDX }} | sudo -S _build/sdx/rel/alfred_server/bin/alfred_server daemon

      - name: Remove Old Build Folder
        uses: appleboy/ssh-action@master
        with:
          host: ${{ secrets.APPLICATION_SERVER_HOST_SDX }}
          username: ${{ secrets.APPLICATION_SERVER_SSH_USER_SDX }}
          key: ${{ secrets.APPLICATION_SERVER_SSH_KEY_SDX }}
          script_stop: true
          script: echo ${{ secrets.APPLICATION_SERVER_USER_PWD_SDX }} | sudo -S rm -rf ./_build_old

I’m not experienced with Github Actions stuff also, so I’m happy to hear some suggestions about it too (for instance how to parallelize mix test, build and migrations without having to setup elixir for every job! Did not figure it out yet). :smiley:

My questions are:

1 - Between the steps Rename Old Build Folder and Stop Old Application I have my application running but the original _build folder no longer exists. Can something goes wrong between these 2 steps? Should I stop before renaning? Nor renaning at all?

Something I’ve noticed is that if I run the server and rename the build folder, I can no long access it via the mix release remote command. I think it happens because the iex process cannot find the log files anymore but I’m not sure.

2 - Thinking about minimizing downtime, my first approach was to only run Stop Old Application just before Start Application, and this approach worked when there is no traffic in the application. But when the application has traffic for some reason the new release isn’t started and I don’t know why.

3 - How is your standard CI / CD pipeline for elixir projects? Are they similar to my approach? What we do similar? What we do different?

Thanks very much guys! :smiley:

1 Like

Maybe I’m wrong on this. But you don’t want to parallelize the Elixir test. The thing is that when you are in the testing step, a new independent VM is gonna be created for that on the Microsoft servers (I guess) although you can have your own servers if you want → your project is gonna be setup → the tests are gonna run. If the tests fail then you won’t want to continue with the next step, which is the deployment part. So it’s a secuencial job, (only if the former is successful then you will run the latter), you don’t want it parallelized. I hope it makes sense. Maybe I’m wrong with this, though

I used to run a pipeline on circleci that ran the build and tests in parallel but only ran the deployment step if both the test and build steps passed. Its a good way to shave off a few minutes/seconds of deployment time.

1 Like

BTW, if you want to save deployment time on the testing part you can use this API Reference — husky v1.0.3. This won’t let you push to GitHub unless all test have passed successfully. Maybe by using that you can skip doing the same test again on GitHub Actions. I don’t know if this is a good practice or not but it’s an idea :bulb: