Testcontainers - A Testcontainers implementation for Elixir

Your Libraries & OS Mentoring Libraries
, , ,
1

Testcontainers

Testcontainers is an Elixir library that supports ExUnit tests, providing lightweight, throwaway instances of common databases, or anything else that can run in a Docker container.

It supports Testcontainers Desktop, but doesn’t need it to run. It will find and pick up a docker engine that works every time it runs.

Let me show you its features

In its basic form Testcontainers can used like this:

{:ok, _} = Testcontainers.start_link()
config = %Testcontainers.Container{image: "redis:latest"}
{:ok, container} = Testcontainers.start_container(config)

But the most interesting usage for Elixir is the Ecto module, which lets you run tests without having a postgres (or mysql) container running (see the README section for this advanced usage).

You can also spin up a throwaway container in a single ExUnit test:

container(:redis, Testcontainers.RedisContainer.new())

or shared container for multiple tests

container(:redis, Testcontainers.RedisContainer.new(), shared: true)

How it works

It has a dependency on an autogenerated elixir library for the docker engine api. The complex logic for calling out to docker is hidden inside that library.

16 Likes
2

That looks very very interesting, thanks for sharing! :blush: :+1:

1 Like
3

new version v1.3.0 released which dynamically sets host port in repo config when using Testcontainers.Ecto :partying_face: No more duplication of database options :partying_face: see here for more info

2 Likes
4

Version 1.4.0 released

  • debug failed containers, containers are no longer autoremoved
  • added Minio and Cassandra container modules
  • experimental support for persistent containers with Testcontainers.Ecto

:partying_face:

1 Like
5

This library looks awesome. I have some questions though:

  1. Are database instances isolated from each other?
  2. How much time does it take to spin up a single container? (I mean, less than a second, or several seconds, or even minutes)
  3. Does database container run migrations every time it’s created? Or does it preserve the schema and seeds between runs?
1 Like
6

Hi.

In its core it allows for spinning up containers in tests. I have additionally added support for running a database container in dev and test in Phoenix. I use this approach myself in all projects i work on lately. And it saves a lot of frustration not having to give the slightest thought about how the database is created. We use mix test.watch to automatically rerun test suites, and i cant say it takes that long to start the container. I think right now the bottleneck is the database startup, but we only have 170 simple tests which completes in sub second. For larger Phoenix projects and those with sync tests i would expect the tests being the bottle neck.

I am going to add some timing logs on container startup but i wouldnt use too much time on that. We use alpine version of postgres if remember correctly.

The readme has been improved a lot recently. So the example in the original post is outdated and just wrong. Not possible to compile for release when using Mix module in application.ex. So use the GitHub repo readme as example on how to do it properly.

The Ecto approach does run migrations each time tests are run or each time app is started. But it supports an option persistent_volume_name that makes it possible to run Phoenix in dev mode with persistent changes.

I dont understand the isolation question 100% but all containers use dynamic and random ports. And there is no reuse.

1 Like
7

Oh btw yes seeds is not run automatically. I should propably fix that. PRs are welcomed.

8

however, there is an alternative way to solve this though. You can easily start the phoenix application in interactive elixir shell with iex -S mix phx.server and run Code.eval_file("priv/repo/seeds.exs") in the interactive terminal.

1 Like
9

Any plans to support podman as an alternative to docker?

10

I might be missing something but podman can alias itself so the docker command actually calls podman.

11

This is supoorted by Testcontainers. There are docs on using podman on the Golang language docs on testcontainers.com. Testcontainers for elixir will work with any docker runtime given it can find a docker host. And it uses several strategies

1 Like
12

That’s cool. I wonder if it is possible to have concurrent tests, where each of those tests is running in an isolated database container. As far as I know, Ecto can spawn only one connections pool, that is using one port

13

Testcontainers for Elixir has module Testcontainers.Ecto. The purpose of this is to start a container, override ecto repo config and shutdown repo and container on system/test exit. The module will temporarely override the configured port with the dynamic host port of the container. So technically (not tested) if you have multiple repos you can run postgres_container for each repo.

14

But im not fully sure if I understood the question 100%.

15

I meant to ask if it is possible to run two or more tests at the same, while both of them are using the same Repo module, but each of this tests is connected to it’s own testcontainer with db. I guest that it is not possible, since Ecto Repo is a singletone, right?

16

That is contradictory to the point of the library, which is to provide full container and service isolation for e2e testing.

17

Not really an answer to your exact question, but we had a similar issue of running tests in parallel in my last job, and we solved it by using a different database for each db connection in the pool.

I wrote about it a bit here - Experimenting with running mysql tests in parallel

Initially I was going to suggest configure option to switch server, but realized that configure hook only comes into play after it is connected to DB, so we can’t really use it to switch different db server running at different port

UPDATE: on second look, you should be able to configure to switch to a different DB server running on a different port for every connection (haven’t tested)

18

Maybe you could outline the problem a bit more ? In GitHub actions you can run the tests in partitions, effectively in parallell.

EDIT: You dont have to use partitioning on database name with Testcontainers. Its a new database for each time tests is run in GitHub actions. So Even if its the same database name in every parallell workflow run its different database server

19

Version 1.5.0 released

  • New contributor: @Argonus :partying_face:
  • Feature: Kafka container with embedded and external zookeeper support, and Kraft
  • Feature: Run code right just when container starts (useful for uploading startup scripts)
  • Improving documentation

Feedback and suggestions ia always welcomed!

1 Like
20

Version 1.6.0 released

Improved support for older elixir versions. Minimum required version is now 1.13. Kudos to @Argonus for this update.

Ryuk has been updated to latest version.