fireproofsocks
Squashing Schema Migrations
Anyone who has worked on an old code-base is familiar with this problem: the migrations folder can get huge. The database your app started with may bear little resemblance to the database it uses now.
I’ve been reading about how to “squash” migrations in other frameworks… the idea is pretty simple: take a single snapshot of your database schema as the new starting point and trash all of your old migrations. It reminds me of the “squash and merge” option when you close a Github pull request.
How would one do this in Elixir and Ecto? Has anyone written an article or documentation page on this perhaps?
Thanks!
Most Liked
cblavier
Everything went smoothy, here is how I did it to squash everything until 2021.
MIX_ENV=test mix ecto.drop
MIX_ENV=test mix ecto.create
MIX_ENV=test mix ecto.migrate --to 20211223095738 # last 2021 migration
MIX_ENV=test mix ecto.dump -d priv/repo/schema-squash.sql
mkdir priv/repo/migrations_archive
for i in {2016..2021}; do mv repo/migrations/$i* priv/repo/migrations_archive; done
mix ecto.setup
And here is my mix alias in mix.exs:
defp aliases do
[
...,
"ecto.setup": [
"ecto.create",
"ecto.load -d priv/repo/schema-squash.sql -f --skip-if-loaded",
"ecto.migrate"
]
]
end
EDITED after @LostKobrakai hint
fireproofsocks
We have come up with the following strategy to deal with squashing migrations on a periodic basis.
- Run
mix ecto.dumpor the equivalent database command - Rename the generated
structure.sqlfile to something the better communicates that this represents a snapshot of the database structure at a certain point in time, e.g.snapshot-2023-11-10.sql - Remove the old migrations (celebrate if you wish)
- Create a new migration that references the dump file from above. This helps avoid frustration when developers unfamiliar with the process have to run yet-another-mix-command to bootstrap the database. They may run into problems getting the
psqlCLI installed, for example. You can write a simple migration that parses the dump file like this:
defmodule MyApp.LoadStructureSqlMigration do
use Ecto.Migration
require Logger
def change do
:core
|> :code.priv_dir()
|> Path.join("snapshot-2023-11-10.sql")
|> File.read!()
|> String.split(";\n")
|> Enum.each(fn statement ->
execute(statement)
end)
end
end
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance









