sezaru

sezaru

I’ve been wondering for some time now if there is a way to instruct ecto to run all, not yet applied, migrations inside a single transaction.

For example, let’s say I have two migrations:

20260406204201_remove_some_stuff.exs

20260406206213_add_some_stuff.exs

Now, let’s say that I push this to prod, ecto will create a transaction to run 20260406204201_remove_some_stuff.exs, commit it, and then create a new transaction for 20260406206213_add_some_stuff.exs.

Now, let’s imagine that for some reason 20260406206213_add_some_stuff.exs migration fails.

Now, I have a system that I can’t rollback to the last working version because 20260406204201_remove_some_stuff.exs removed some fields, so I can’t actually rollback it, and I can’t install the new version because I can’t finish the full migration.

Meaning that I will need to first create a new release with 20260406206213_add_some_stuff.exsfixed and I will have downtime during that period.

So, I was wondering, is there some way to make Ecto run ALL migrations inside the same migration?

If I could do that, then the example above would work just fine, since 20260406204201_remove_some_stuff.exs and 20260406206213_add_some_stuff.exs will run inside the same transaction, so even if 20260406206213_add_some_stuff.exs fails, 20260406204201_remove_some_stuff.exs will not be committed, so it would be safe to rollback to the older, working version.

Showing Posts 1 to 8

Asd

Asd

Migration is an operation which converts database from one valid state to another valid state (in both directions). If some migration leads to invalid state, it means that the migration is incorrect. You can work with incorrect migrations, but it is much harder thing to work with than the correct migrations

Calling multiple migrations inside one transaction is possible, you’d just have to merge the code of all of them into a single migration. Check out this guide for more info about it: GitHub - fly-apps/safe-ecto-migrations: Guide to Safe Ecto Migrations · GitHub

Schultzer

Schultzer

Nice, this is a timely question as I’m planning to tackle migration soon in my library and I didn’t know this was a thing, feels like this should be one transaction with multiple savepoints.

Any reason why you would’t want this behavior?

Asd

Asd

Performance. DDL transactions lock tables for the duration of the transaction.

sezaru

sezaru OP

Ah, yes, I was aware of ecto.dump, but unless I’m misunderstanding something, It will not fix the issue I’m having.

AFAIK, ecto.dump will always dump your current DB state, it can’t create a SQL file for a subset of migrations (In my example 20260406206213 and 20260406204201), so that’s only useful if you are applying the dump to a new, empty DB or to just be able to remove migrations files for already applied migrations.

mix ecto.migrate do have the options –log-migrations-sql and –log-migrator-sql which will output the SQL code, but you can’t get that without actually applying the migration to the DB, meaning that it doesn’t solve the issue too.

Too bad that we don’t have a dry-run or some way to extract the sql commands from the migrations to run it manually

dimamik

dimamik

If I were you, I’d probably make sure (and test) that either all migrations can be rolled back, you can adjust old migrations too if needed, OR replace all old migrations with one, idempotent, new migration.

Too bad that we don’t have a dry-run or some way to extract the sql commands from the migrations to run it manually

You could rollback to the latest migration version on prod with your test environment, and then re-run all migrations with log migrations sql. You’ll get raw sql.

Can you elaborate more on the problem you’re solving?

LostKobrakai

LostKobrakai

Tbh I don’t think even with the request implemented it would help much. If you’re relying on transactional rollback I’d strongly suggest this to be enforced by having all code in a single migration and not be relying on deployment order. Nothing will make sure 20260406204201_remove_some_stuff.exs won’t be rolled out individually besides manual “making sure”, which is not a great state to be in.

If you really need distinct migrations you’re probably better off making individual migrations in a way they do not break the prev. version of your application, so you do not need to rollback anything to stay with a working application.

Asd

Asd

The link I’ve shared is not only about ecto.dump and if you want to merge two migrations into a single one, the most part of the advise still applies

dimitarvp

dimitarvp

Two options:

  1. Accept multiple migrations and extra work to make sure each is non-breaking and is deployed together with code that also makes use of the new schema / new data;
  2. Make one big migration.

#1 could be viewed as safer but it’s absolutely introducing code churn – you’ll write interim code to work with the interim state of the schema and the data… and you’ll be throwing it away next week.

#2 requires more planning and more testing.

I’d usually vote for #2 but I’ve done #1 more times than it due to various business requirements and limitations.

— All posts loaded —

Where Next? Top

Trending in Questions Top

RSP87
I’m working on a project that simulates the bumbl example in the programming phoenix book. It acts almost like an email client. We have a...
New
nseaSeb
Hello, I know there is an approach for handling lists that allows for optimized traversal, but I can’t recall the specific method (somet...
New
RemyXRenard
I’m seeing that a list inside a Kino.DataTable will be interpreted as a charlist, even if the Kino.configure() is set to charlists: :as_l...
New
velrest
So my question is quite simple and i have found no conclusive answer on forum, google or AI. Should we use :erlang.float for Integer to ...
New
brecabral
Documentation While reading the Scoped Routes section, I noticed that the documentation currently refers to a problem without explainin...
New
samoloth
Hi, I’ve just set up an application with ash_authentication. There is only magic link strategy for now, so there is no confirmation add o...
New
FlyingNoodle
If a change or preparation module uses Ash.Changeset.get_argument/2 or Ash.Query.get_argument/2 (or any of the other get_argument functio...
New

Other Trending Topics Top

mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
marciok
Hi there! We created Gust: A task orchestrator inspired by Airflow. For those who have never heard about Aiflow, it’s a Python-based wor...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Dmk
Xamal is a deployment tool for Elixir apps that deploys native releases to bare metal servers over SSH. It’s a port of GitHub - basecamp/...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
webofbits
With AI doing more of the implementation work, I’ve been wondering how much coding I should deliberately keep doing myself. My main conc...
#ai
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews