Phoenix : options for in-memory database in `test` env

My team uses an umbrella project with, among other “standalone” applications, a Phoenix 1.3 app.

Running mix test from the umbrella root causes the phoenix app’s test to be run, meaning a postgres db has to be set up for every developer (including those not working on the phoenix app at all.)

This is a bit cumbersome, so I though about using an “in memory” / “install-less” DB for tests on a developer local machine by default (clearly, our CI server would still run the tests against a real postgres database.)

What would be options to do that, knowing that sqlite is not an option at the moment, since unfortunately (at the time of writing) the sqlite_ecto lib has not been upgraded to Ecto 2 (there are some efforts on this front, but it does not seem to work in my case ?)

Any other “light” DB that I should look into ?

2 Likes

I don’t think using a different database in dev & production is the solution - it definitely isn’t if you use any of the interesting features of Postgres.

A simple solution might be to use @moduletag :db in the test case modules that require database (can be further automated with templates), and running mix test --exclude db to skip the tests that require the database.
That might still lead to issues, if applications themselves require connection to database.

7 Likes

Yes, that’s typically the problem : phoenix tries to connect to a db to create a test schema behind the scenes.

1 Like

This is a really nice use case for postgresql in docker.

https://hub.docker.com/_/postgres/

Starting a server in a docker container is a single command.

3 Likes

I agree, that’s what I’m advising my peers to do at the moment.
But it is pretty annoying for those that never touch the web app.

1 Like

Just conditionally include the Repo in the supervision tree.

1 Like

Not sure I get your idea here… Should I try with one Repo for test and another Repo for dev ?

1 Like

This feels like a recipe for disaster IMHO. Having a different DB in dev vs prod is going to hamstring your webapp team by essentially locking them out of all of PostgreSQL’s more advanced features…that constitute the bulk of the selling points for using it.

If the mix test --exclude db solution doesn’t work I’d strongly suggest continuing to use PG locally. Between the Docker option and even https://postgresapp.com/ if you’re developing in OSX it shouldn’t be a huge commitment.

4 Likes

I pretty much agree with everything others have said. Using the same db for local dev, testing, and prod is the best solution.

If setting up dbs is challenging, consider simplifying it with e.g. docker. At work, we use dockerized DB for dev/tests, and even have some helper scripts to start them and conditionally prime them. As a result, setting up of the dev env is almost completely automated. In particular, no custom configuration (such as creating users, creating databases, permissions, etc.) is required.

1 Like

Another option is to add the following to your mix.exs:

aliases: ["test.dbless": "cmd --app app1 --app app2 test"]

Then you will run test on every given app individually, you can skip the Phoenix one if you wish to.

1 Like

I understand your concerns about differences in DB implementation.

However, I am at a point of the project where we don’t use the DB at all (not yet), but due to the strong expectation on phoenix side that a DB is running, it impacts everyone working on most of our projects, even though they don’t (and should not) care about a DB.

Any way, thanks for the comments.

1 Like