Removing a duplicate migration

Questions / Help
1

I was having trouble getting an Oban migrated on a staging server, and along the way created two versions of the same migration. Now, I’m kind of stuck because this has been deployed, but the deployment can’t finish because the migration won’t run because of the duplication.

  {:down, 20221214223233, "add_oban_jobs_table"},
  {:down, 20221110193829, "update_product"},
  {:down, 20221108223233, "add_oban_jobs_table"},

Dropping the database would be a last resort in this case due to the complexity of seeding/setting up everything.

I’m running all these operations through the console via Ecto.Migratior eg: Ecto.Migrator.run(MyApp.Repo, path, :up, all: true).

Without knowing much about how migrations work I feel like my best bet is getting rid of 20221214223233 somehow.

2

Hi @travisf I would deploy a new version of your application that simply removes the extra migration file.

3

Thanks. Maybe this is a codedeploy issue? Running a new deployment I’m still getting the error: ** (Ecto.MigrationError) migrations can't be executed, migration name add_oban_jobs_table is duplicated

Running Ecto.Migrator.migrations/1 still shows the duplicate.

I’m going to try and manually deploy skipping codedeploy all together.

4

This is not advice for what to do in your specific case, just a tidbit that might be useful.

AFAICT, Ecto tracks migrations in the schema_migrations table:

mydb=> select *  from schema_migrations;
    version     |     inserted_at
----------------+---------------------
 20220604234935 | 2022-09-23 00:44:37
...
 20221206180706 | 2022-12-14 06:43:47
 20221216075959 | 2022-12-16 19:09:52
(8 rows)

and deleting the last row (version 20221216075959 / inserted_at 2022-12-16 19:09:52) would make mix ecto.migrations think that migration 20221216075959 hasn’t run.

I don’t know know what is happening in your specific case and how you ended up with a duplicate migration, so I am not advising that you delete any entries in your schema_migrations, just thought this might be a useful FYI.

5

Well I’m flummoxed. The code-deployments failed (although they seemed to get everything deployed, the part that failed was the migration). So I manually deployed a tarball release, same issue. I’ve also gone in and deleted the last migration version as @markmark206 mentioned, (I’ve double checked that it’s gone).

Server has been restarted.

When I run Ecto.Migrator.migrations/1 I’m still met with the same list of migrations as before (both of the faulty duplicates). Where is migrations/1 pulling this data from?

6

In another (not Elixir) deployment system I’ve seen weird behavior like this happen when a “release” copied a whole directory over top of an existing directory - files that were removed in the new one were ignored and stuck around!

Something similar could be going on if your deployment is copying priv/repo/migrations over old versions…

1 Like
7

This may have been the fix, I ended up pretty much blowing away whole project on the server and redeploying. I was able to remove the old migration but I’m still having issues with migrations that are listed as down, but actually up?

 {:down, 20221110193829, "update_product"},
  {:down, 20221108223233, "add_oban_jobs_table"},
  {:up, 20220729153610, "** FILE NOT FOUND **"},
  {:down, 20220208223202, "add_custom_override_to_addresses"},

If I run Ecto.Migrator.run(PacksizeData.Repo, path, :up, all: true) I get this error:

 alter table addresses
** (Postgrex.Error) ERROR 42701 (duplicate_column) column "override" of relation "addresses" already exists

I notice that there is a ** FILE NOT FOUND ** line, I don’t know if it’s causing problems in this case, cruising through the historical migrations I see a few other instances of that, all up, never caused any issues.

8

That list gets data from two places:

  • the migration files in priv/repo/migrations
  • the rows of schema_migrations in the DB

The ** FILE NOT FOUND** entries will always be :up, because they represent rows in schema_migrations that don’t have a corresponding file in priv/repo/migrations.

The pickaxe could help you find that file; run git log -S 20220729153610 to see when a file with that timestamp in the name was added/removed.

As to the old migration from February (20220208223202) that’s :down, I’m mystified how that happened and wasn’t failing before…

9

I’ll take a look… I think 20220208223202 is one I manually rolled back using Ecto.Migrator.run/4 somewhere in this process.