fireproofsocks

fireproofsocks

Seeds as migrations?

While working through a Docker build/deployment, it hit me that seeds are weird. Apart from requiring some extra setup (e.g. some variant of Migrations - Distillery Documentation), you can wind up in trouble if you accidentally run the seeds when you didn’t expect.

In some cases, the problems with seeds could be solved if they were treated as a migration – I even saw some Ruby Gems that do this, so other people have thought of this.

What are the thoughts about putting seeds into a migration? I don’t think there needs to be any consideration to rolling back a migration like that… the table structure or the table itself would just get altered or deleted.

Most Liked

shanesveller

shanesveller

Are you using Docker-Compose for your production deployments?

I’d probably use docker-compose run --rm myapp migrate and docker-compose run --rm myapp seed after setting up the ReleaseTasks module per the document you linked, plus the two migrate and seed Distillery commands. Obviously, this doesn’t automate the execution of the migrations/seeds as you bring up the application if you’re only typing docker-compose up [-d], but it’s a simple, documentable step that doesn’t leave you with the second container lingering.

If you are running docker-compose on your production deployment targets, hopefully you’re wrapping it with a Systemd unit or something, where you could add an ExecStartPost or similar hook.

I keep a skeleton repo to be a companion to blog posts and other open demonstrations, which has these implemented if you’d like to see a concrete example:

shanesveller

shanesveller

Personally, I also have to fight this attitude/approach at work and it’s a hill I’m willing to die on: this is an abuse of the migrations system. Just use a seeds file, or preferably a more robust vanilla-Elixir module that calls either raw Ecto or Context functions. Expend the modicum of effort to make sure they’re idempotent, and they’re just as safe to run and rerun as mix ecto.migrate can be.

Table/column/index definitions aren’t the same thing as prepopulated data and they shouldn’t be managed the same way. Data insertions in migrations should be minimal, and any data updates done in a migration should be as necessary due to column changes, and execute as quick as possible. More general data munging and sanitization are an operational task that can and should happen out of band.

The only benefit you get from using migrations for this is quick proof that they have been run in the past, and a shortcut to preventing them from running again. Proper idempotence is far more preferable.

You can’t really undo a single migration from history, without just creating an inverse migration, but you can easily clean up rows that are no longer necessary through normal means.

What happens as the data you seed evolves with the application? Do you really want a new migration every time a value changes or an attribute gets added? We should never be editing existing migration files that have been broadly applied. If you adhere to that, is there actual value being captured in all of the prior historical migrations, that aren’t already captured by your VCS data? It’s just cruft, as we often only care about the current/latest state of that info.

Eventually you’ll likely run into the scenario where not every environment gets all the same bits of prepopulated data, and the migrations system is going to be inflexible enough to make this feel pretty clunky.

fireproofsocks

fireproofsocks

We can’t simply use a seeds file because don’t know all the vendors beforehand. We might onboard new ones every couple weeks. So our options are:

Option One:

  1. Push out new code (a new module dedicated to handling the vendor-specific logic).
  2. Manually log in to our admin portal and create a new vendor record with the proper slug ID (so that the dynamic dispatch can resolve correctly to the new vendor module) OR have shell access to the server to run a prepared seed file.

Pros: no migration required
Cons: manual, prone to human errors (misspellings etc) OR it requires shell access to server.

Option Two:

  1. Push out new code (a new module dedicated to handling the vendor-specific logic).
  2. Include a migration that adds the proper record to the vendors table.
    Pros: automated, exact.
    Cons: ???

Keep in mind that in this simplified example, the “vendors” table is very narrow: it’s almost just a key/value store.


Enums

Another use case where “seed migrations” make sense: we have stopped using enums in our database columns because they don’t play nice with migrations. Some divisions “solve” this problem by letting the application restrict the column values. That causes headaches for BI because when they are doing any reporting on those databases, they have to crack open the application code to know what the allowable values are, e.g. status_id 1 means “active” and status_id 13 means “cancelled” etc. And god forbid you accidentally let a bad value in. It’s very opaque, there are no constraints inside the database, and you experience something akin to “vendor lock” with your application code so that even if you wanted to replace the application that sits on top of that database, you’d be stuck coding in the same ad-hoc ersatz enums somewhere inside the application. Boo.

So instead of using enums in the database, we create a dedicated table, e.g. “statuses” and then reference it via a foreign key (I know, it’s hardly revolutionary). It’s more work to set up, but it makes things more transparent for BI/reporting and it plays nice with migrations.

And that seems like a really good use-case for using a migration instead of a seed: the application code has specific logic built around those statuses. The application falls apart if you don’t have specific values in your database. And importantly, when you need to add a new status along with new behavior, a migration seems like a really clean way to introduce that change. Importantly if the enum columns played nice with migrations, then you’d modify the allowed values via a migration anyhow.

If you think of migrations as strictly defining database structure, that solution might annoy you, but if you think of migrations as managed database changes, it seems like a pretty viable solution.

Where Next?

Popular in Questions Top

Kurisu
For example for a current url like http://localhost:4000/cosmetic/products?_utf8=✓&query=perfume&page=2, I would like to get: ...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
jaysoifer
Is there a way to rollback a specific migration and only that one (“skipping” all the other ones)? Would mix ecto.rollback -v 200809061...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New

Other popular topics Top

9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
chrismccord
This release brings a number of exciting features, including integration with the new Phoenix LiveDashboard and Phoenix LiveView. There h...
New
RisingFromAshes
I’ve read in another post that it may be possible with a router helper - but I couldn’t find an appropriate one, and tbh, I’m still just ...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
klo
Got a question about when to concat vs. prepending items to list then reversing to achieve appending. So i know lists boil down to [1 | ...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New

We're in Beta

About us Mission Statement