TL;DR: library to conveniently spawn throwaway docker containers for your integration/functional tests. Not stable yet. Written by an Elixir newbie, any feedback is super welcome!
Hello everyone
I’ve been studying Elixir in the past few months, and to get some practice I decided to try and develop a library that I felt could be useful for someone.
I developed something similar to the Testcontainers java library for elixir.
The idea is to make it as convenient as possible to launch and use docker containers within your integration/functional tests.
No need for external docker-compose.yml files, your dependencies are nicely and declaratively defined within your own tests.
Installation, features and basic usage are documented in the project’s README.
Note: while the functionality is there and it is quite comprehensively tested, this library hasn’t seen any real world usage yet, so it is to be considered not stable yet.
I hope someone will find it useful! If you’d like any functionality that is not implemented yet you are welcome to open an issue. As I mentioned I just started learning elixir, so if you have any feedback or suggestion on how to make the project better/more idiomatic, please let me know! Thanks a lot in advance
This is great, and something I think is very much needed in the ecosystem!
I believe that this has potential to be used beyond just starting containers from test. For example, we could use this to start/link the accompanying containers when the system starts, or to run containerized one-off commands.
I’ve been meaning to develop something like this myself as the part of the ci library, but it looks like excontainers already solves many of the things I planned.
The usage of ryuk for container cleanup is nice, I didn’t even know about it! My plan was to start a separate OS process (ideally a hidden Erlang node), which would roughly do the same thing. I still plan on trying this out, because I think it could solve a subtle race condition that exists with ryuk.
I wonder why is Docker.Api marked with @moduledoc false? This is something that I think could be useful to the general population, and I think it’s possibly the most important part of this library. Having a simple docker API wrapper would allow people to build various custom abstractions & docker flows.
The synchronous logic based on polling in exec_and_wait likely won’t work for the scenarios I have in mind. I’d probably aim for a reactive approach, i.e. a GenServer that receives response chunks from the start request, and stops once the exec command has finished. That way the client could also handle stdout/stderr in realtime, e.g. by logging the output. I’m not sure if this can be done with tesla, but I think that it should be possible with mint. I’ll try this out myself, because I need this for the ci library, but I can’t commit on time.
Hi @sasajuric , thanks a lot for your feedback and for having a look at the code!
I believe that this has potential to be used beyond just starting containers from test
I completely agree with you.
I started with a focus on the testcontainers usecase in order to deliver something potentially useful as soon as possible and get feedback from the community.
However my goal is to do as you suggest and evolve the library to eventually make it useful for more generic scenarios as well.
I think it could solve a subtle race condition that exists with ryuk
Is that race condition specific to your use case, or is it something I should care about for Excontainers too? In the latter case could you tell me more please?
I wonder why is Docker.Api marked with @moduledoc false?
Same reason as above - so far I tried to keep the scope as narrow as possible. But I do plan to work on making the library more generic as soon as I have time.
The synchronous logic based on polling in exec_and_wait likely won’t work for the scenarios I have in mind.
Yeah polling is definitely not optimal, but I thought it was an acceptable shortcut in the initial phases of development.
I plan to try and replace it with a more efficient reactive approach, which would also be useful for other use cases (e.g. waiting for a specific log to detect when the container is ready).
And by the way your CI library is awesome, if this is a feature that could help with your effort I would be happy to make it my priority for the next steps of Excontainers
If the beam node is taken down after docker run is invoked, but before the container is registered, the container will not be reaped.
Admittedly such scenario is not very likely, but given time it’s bound to happen at some point, and it may in turn lead to bugs which are hard to track.
The only solution I can think of is to run an external program which would be both, the starter of docker containers, and the “garbage collector”. When we want to start the container we ask the program to do this for us (which would obviously be wrapped under some nice API). That way, even if the beam node is taken down in the middle of the container start, the external program can still clean up all the resources. My idea is to try implementing this in Elixir, rather than go. If this works, we could discuss the option of merging it to excontainers.
No need to change your priorities for the moment My plan is to implement my own docker exec as a wrapper around OsCmd. This would give me all the features of OsCmd practically for free, most notably async & sync execution with reactive output capture and proper cleanup on early shutdown. As a consequence, such implementation would use CLI instead of the HTTP API, so I don’t think it makes sense to have it in excontainers.
You are right about the race condition, thanks for letting me know!
I’m not sure if this would fit your scenario, but perhaps a simpler solution could be to use something different than the container ID to register the container for reaping, so that we can set up the reaper before starting the container (Ryuk supports all kinds of filters – by name, label, etc.)
For example we could generate a unique identifier (e.g. a UUID) for the container, set the reaper to kill containers with label e.g. reaping_id=<my-id> and then start the container with the unique label.
My idea is to try implementing this in Elixir, rather than go. If this works, we could discuss the option of merging it to excontainers.
Sure! I’d be happy to make excontainers 100% elixir, even for containerized dependencies
My plan is to implement my own docker exec as a wrapper around OsCmd […]
Yeah I agree both that excontainers would not be a good fit for your usecase, and that bringing CLI-based functionalities in excontainers would not make much sense.
I’ll let you know if I make substantial progress on that front thanks again for your feedback!
Oh, I see now that in the code that filters are “lazy”. Yeah that’s a great idea that would definitely work! Maybe consider mentioning this approach in docs.
I’ll still give this a try, b/c I want explore some ideas, but it looks like using custom unique labels will work.
Hi @NobbZ , thanks for your feedback.
Looking back at what you wrote here, I’m starting to think that excontainers may actually not be a perfect fit for your needs.
Excontainers is mostly useful to spawn short-lived containers to allow running integration tests asynchronously and with good test isolation.
Using it to spawn long-running containers is possible, but not very straightforward.
Of course any suggestion to make it more suitable for your scenario is very appreciated
Some potential problems I see are:
we’d need a way to start containers and run migrations (if any) before the application starts, otherwise other processes that depend on these containers could start to break. This may not be a big deal in some cases, as everything could go back to normal once migrations are done, but even then it would still be ugly.
if you need the container to listen on a specific port on the host (eg. postgres on 5432) then it would be hard to handle multiple instances of your application (eg. one for your local server and the other to run tests / run a REPL). New instances of your app would try to spawn the container and fail due to the port being already taken.
In case of a long-running container shared by the whole test suite, you could start the container and run migrations in your test_helper.exs.
If you have containers under your supervision tree then it could be a little more tricky, as you’d need some way to execute migrations once the process (and hence the container) is started.
This is something that I’ll consider to add.
For now the easiest ways to start a permanent container are either to start it manually in your test_helper.exs or to place {Excontainer.Container, your_container_configuration} under the supervision tree of your app.
This is perfect for what I need, and pretty much anyone who uses stripity_stripe for using the Stripe API.
The recommended (er, at least when I was actively writing the tests) was to use stripe-mock and boot that up in Docker when running integration-level tests. Since I’m impatient I am all for high ROI, so I generally focus on integration level tests. There’s only so much I would like to create Mocks for an API with a huge footprint…
So I always manually run a short Bash script to boot it up and then run the tests, which isn’t that fun ;/
#!/bin/bash
docker run --rm -it -p 12111-12112:12111-12112 stripemock/stripe-mock:latest
This would be awesome to do all this automatically!
Can this be hooked up to a tag, so it only boots up if tagged tests are going to be run?
Anyways thanks for this package! I think it’s much needed.
I went to the official Testcontainers Slack and expressed an interest in starting an Elixir version, and their head of product linked me to this thread!
If there’s anything I can do to assist with this project, let me know!
Hi @Nezteb!
Sorry for the late reply - I’m glad you’re interested in this project
I think the library is currently fine for simple use-cases, but far from being as stable, powerful and flexible as Testcontainers.
As you can see from Github I’m currently not actively working on it, and I’m mostly taking care of reviewing and merging PRs - even though I have some ideas I’d love to eventually experiment on.
Do you have any specific objective or activity in mind that you’d like to tackle?
Some points that IMHO could be worth exploring are:
autogenerating the bulk of the docker integration logic by parsing the official openapi spec. I started doing some experiments here but it’s still a very incomplete draft for the moment
streaming logs from containers, which would be useful eg. when waiting for a container to start, since now the only option for that is polling by periodically executing a command. Streaming logs is rather easy, I successfully experimented it here but never ended up integrating it in excontainers.
tracking the state of containers reactively. This could be done by monitoring docker events but I feel that getting it right may not be easy.
supporting dependencies between containers, and possibly more complex setups involving eg. shared networks, volumes etc. I’m not sure what would be the best way to approach this, though ideally I think it’d be cool to try and integrate that as much as possible within the OTP model.
In any case, if you want to either hack on excontainers, fork it or start a new package from scratch, I’d be happy to support if that can be helpful
I have been doing some thinking of making a fork of testcontainers-ruby and «porting it» to Elixir. Its not a 1-1 port in any way possible. But i was thinkng that Ruby is the closest Elixir.
To do something like this i need to implement a core that can talk with docker. Im planning on making a repo that looks and works similar to how test containers is working. Maybe we can merge or even reuse alot of stuff inside the excontainers module. My plan is to add elixir as a language into the testcontainers ecosystem.
Maybe someone wants to participate ? Maybe its already done ?
@dallagi maybe we can collaborate on something. First ill look at what you have made. Then ill see how it stands up to testcontainers for ruby. Its a complex domain for sure
never mind, seems you already have it up and running. I can look into it and see if there is any room for improvements. I can implement Ceph container support in it, get that going
Havent gotten any feedback either on github issues or here in this thread. So i have decided to initiate a new project called testcontainers-elixir. Havent gotten any Feedback from testcontainers Slack #contributors channel yet but I suspect none will come. Basically no one cares unless it gains popularity. I made testcontainers-ceph, a Java community module and i want to implement this in elixir too. So that will be the «test» container for my new project since its not difficult to get going. Slow yes, alpine is faster lol.
Anyway, maybe we can join forces? Excontainers and testcontainers-elixir ? Or i could use code from excontainers if license allows for it. Ill use gleam anyway for the core docker api Communication.
Sorry for the late reply, I wasn’t able to give much attention to my email/GH notifications this past week.
Anyway, I think starting your own project was the correct choice here, as personally I wouldn’t be able to commit consistent time on a common project right now.
Of course you’re very welcome to take any code that may be useful for your project from excontainers.
To make that easier I just relicensed it to MIT, as the previous GPL license would probably have made it a bit awkward.
I attributed the parts of the code I reused and later modified. Thanks. I have a lot of spare energy for this kind of “off-work” work. But I hope someone else can join on it unless I get it good enough to be merged into testcontainers mainline org.
One idea if you can’t contribute to testcontainers-elixir directly is to pull the code, skim through it, and provide any feedback to @jarlah?
An even simpler idea would be to just try using testcontainers-elixir on a test project and let jarlah know what you liked/disliked from a developer experience (DX) perspective?
(The above are just ideas and not at all expected )
)
