Hi all, not an Elixir specific question, but hoping for a favourable signal to noise ratio if I ask it here.
In theory it is possible to run a Github Action locally, for example before performing a git push to pushing to their server. There are many potential reasons to do so, reduced context switching, reducing failed build noise, etc.
Anyhow, the project GitHub - nektos/act: Run your GitHub Actions locally 🚀 appears to offer this functionality, the idea being, it’s executable runs your github workflow on a local docker image. However, in practice, it seems to be downloading 40GB of container layers every time I run a workflow.
I’m just wondering, before going on the deep dive, is anyone in the Elixir community actually running it successfully, or if not, is there a working fork, or, has anyone used anything similar to run github actions locally, or should I just move on and accept this is a waste of time?
Oh, indeed, but I’m looking for fast cycle debugging of actual github actions (mostly erlef/setup-beam@v1 stuff, but also some occasional weirdness around coveralls (paths) and stuff like that. I’m not just looking to run mix test or the basic stuff.
Interesting, so the suggested optimisation around GitHub clunky CI / lack of local reproducibility seems to be to add a portable CI independent wrapper which can be run locally, in GitHub CI, or presumably on the providers cloud build system. I can see the attraction, particularly as many teams seem to be jumping back and forth between GitHub, Gitlab, Google, etc.
You could use Earthly or Dagger to encapsulate the Pipeline inside the container and then in CI (GitHub, GitLab, etc.) just invoking your code and injecting necessary configuration and secret.
I recommended Dagger personally if you have a plan to scale the usage through your organization since it support runner on kubernetes. And it can be run your pipeline code with the Elixir SDK.
I’m using act to run my Github actions locally. I was having an issue with missing OpenSSL libraries, which caused rebar to fail. Forcing the runner to use ubuntu-22.04 instead of ubuntu-latest resolved my issue.
My workflow is based off the “golden standard” and now looks like this:
name: Elixir CI
# Define workflow that runs when changes are pushed to the
# `main` branch or pushed to a PR branch that targets the `main`
# branch. Change the branch name if your project uses a
# different name for the main branch like "master" or "production".
on:
push:
branches: ["main"] # adapt branch for project
pull_request:
branches: ["main"] # adapt branch for project
# Sets the ENV `MIX_ENV` to `test` for running tests
env:
MIX_ENV: test
permissions:
contents: read
jobs:
test:
# Set up a Postgres DB service. By default, Phoenix applications
# use Postgres. This creates a database for running tests.
# Additional services can be defined here if required.
services:
db:
image: timescale/timescaledb-ha:pg16
ports: ["5432:5432"]
env:
POSTGRES_PASSWORD: postgres
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
runs-on: ubuntu-22.04
name: Test on OTP ${{matrix.otp}} / Elixir ${{matrix.elixir}}
strategy:
# Specify the OTP and Elixir versions to use when building
# and running the workflow steps.
matrix:
otp: ["26.2.3"] # Define the OTP version [required]
elixir: ["1.16.1"] # Define the elixir version [required]
steps:
# Step: Setup Elixir + Erlang image as the base.
- name: Set up Elixir
uses: erlef/setup-beam@v1
with:
otp-version: ${{matrix.otp}}
elixir-version: ${{matrix.elixir}}
# Step: Check out the code.
- name: Checkout code
uses: actions/checkout@v4
# Step: Define how to cache deps. Restores existing cache if present.
- name: Cache deps
id: cache-deps
uses: actions/cache@v4
env:
cache-name: cache-elixir-deps
with:
path: deps
key: ${{ runner.os }}-mix-${{ env.cache-name }}-${{ hashFiles('**/mix.lock') }}
restore-keys: |
${{ runner.os }}-mix-${{ env.cache-name }}-
# Step: Define how to cache the `_build` directory. After the first run,
# this speeds up tests runs a lot. This includes not re-compiling our
# project's downloaded deps every run.
- name: Cache compiled build
id: cache-build
uses: actions/cache@v4
env:
cache-name: cache-compiled-build
with:
path: _build
key: ${{ runner.os }}-mix-${{ env.cache-name }}-${{ hashFiles('**/mix.lock') }}
restore-keys: |
${{ runner.os }}-mix-${{ env.cache-name }}-
${{ runner.os }}-mix-
# Step: Conditionally bust the cache when job is re-run.
# Sometimes, we may have issues with incremental builds that are fixed by
# doing a full recompile. In order to not waste dev time on such trivial
# issues (while also reaping the time savings of incremental builds for
# *most* day-to-day development), force a full recompile only on builds
# that are retried.
- name: Clean to rule out incremental build as a source of flakiness
if: github.run_attempt != '1'
run: |
mix deps.clean --all
mix clean
shell: sh
# Step: Download project dependencies. If unchanged, uses
# the cached version.
- name: Install dependencies
run: mix deps.get
# Step: Compile the project treating any warnings as errors.
- name: Compiles without warnings
run: mix compile --warnings-as-errors
# Step: Check for unused dependencies.
- name: Unused dependencies
run: mix deps.unlock --check-unused
# Step: Check Hex dependencies that have been marked as retired
# by the package maintainers.
- name: Hex retiremefnts
run: mix hex.audit
# Step: Check for security vulnerabilities within dependencies.
- name: Mix Audit security vulnerabilities
run: mix deps.audit
# Step: Check that the checked in code has already been formatted.
# This step fails if something was found unformatted.
- name: Check formatting
run: mix format --check-formatted
# Step: Execute the tests.
- name: Run tests
run: mix test
