How to run migrations that are in a library?

Questions / Help
1

I have a library that I use in multiple projects. In a library I have migrations. The db structure for each project is identical. Hence, the migrations have been moved into the library.

How can I run the migrations of the library for a certain app?

2

You would still generate a migration in each project, but then inside that migration file you would call MyLibrary.Migrations.up and that function would have the migration code.

6 Likes
3

  • Why would I have to do that? No way to avoid it?

  • If not, will you show me a code?

4

DB’s schema state is effectively a singleton. You’ll want to manage its state only in one place and you get bonus points for that one being somewhat immutable. You don’t want to run mix deps.update xx and have it implicitly change your db state for new installations, which later migrations might depend on.

Imho the best example is oban.

3 Likes
7

Irrespective of your use case or whole requirements, one underlying assumption about migrations is that they are executed in a predictable and repeatable order. Hence they use timestamps as part of the name to maintain that ordering. The reason for this principle is that migrations often depend on an assumed state of the database. And that assumption holds because migrations are executed in a predictable order. Both up (mix ecto.migrate) and down (mix ecto.rollback).

So the advice to generate a migration in your host application (the one that uses your library), even if it does nothing other than call a function in your library application, is to supporting holding to the ordering principle and therefore support maintaining a predictable state of the database.

Oban is a library that has migrations that are required to be run by a host program. Like most libraries in Elixir it can be found on hex.

2 Likes
8

even if it does nothing other than call a function in your library application → how would the code look like for this?

And that would neccesitate hardcoding the number of a migration for each migrationNNN.ex in a migration file itself, right? If so, it’s not what I’m looking for.

9

One example is from a library I maintain. The library provides a mix task that when run, creates a migration in the host application. That migration calls a function in the library when the migration is run. You can see the code here. The timestamp for the migration is generated at the time of running the mix task.

I believe this is a quite common pattern in elixir libraries and most of the code in the link came from leaning on prior practise from other authors.

oban implements its migrations in a similar method to the above only you create the migration in your host application yourself. Its installation instructions are here. The timestamp of the migration is generated when you generate your migration.

3 Likes
10

I don’t understand. The migrations that are in the library will work the same way for a host app. Where do you say there’s an issue?

11

The basic contract is:

  1. mix ecto.migrate and mix ecto.rollback will set the database structure to a known state and are idempotent
  2. That all migrations run in a predictable and repeatable order.

The current implementations I’m aware of deliver on that contract by requiring the host application to be the single source of migrations, even when a migration invokes functions in a library. I’ve shown you two examples which operate this way. If you have another strategy to deliver the contract then thats cool too.

1 Like
12

In the host app, mix ecto.migrate will run migrations defined in the host app. It does not know about migrations in any library, so they won’t run.

13

That’s what my very first question was: how to run the migrations that are located in my library?

1 Like
14

@kip provided links to concrete examples here How to run migrations that are in a library?

Can you elaborate on what elements of that example you didn’t follow so that we can direct our answers better?