dcrck

dcrck

Ecto Changeset: modify column type from :string to {:array, :string}

Hi,

I’d like to change a database column’s type from :string to {:array, :string}. Migration should work as follows: "my_value" -> ["my_value"].

I’ve looked in the Ecto documentation, and I tried doing the migration via

modify :column_name, {:array, :string}, from: :string

But that doesn’t work. I’ve also seen from the Postgres documentation that I can use the USING option to indicate how to do the conversion, and the error message when I try to run the migration indicates as much:

hint: You might need to specify "USING column_name::character varying(255)[]"

I don’t see how to specify this option in the modify function, though.

What’s the most straightforward way to handle this conversion? Am I missing something obvious? Do I need to use raw Postgres, or something more complicated? Thanks!

First 10 of 13 Posts Switch mode

aesmail

aesmail

I’m assuming the migration isn’t working because it’s not able to convert the current values in the column to the new type.

You might want to do this in multiple migrations:

  • A migration to create a new column with type {:array, :string} and copy all the old column values to the new column as a list of strings.
  • A migration to delete the old column.
  • A migration to rename the new column to be the name of the deleted column.
dcrck

dcrck OP

Thanks for the suggestions. After sleeping on it, I ended up deciding to pluralize the column name. So this simply became a matter of moving the values from one column to the other.

Here’s what I ended up doing:

defmodule MyApp.Repo.Migrations.ConvertColumnToList do
  use Ecto.Migration

  alias MyApp.{Repo, MyTable}
  import Ecto.Query

  def change do
    alter table :my_table do
      add :list_column_name, {:array, :string}, default: []
    end

    flush()

    Repo.all(MyTable)
    |> Enum.each(fn row ->
      row
      |> MyTable.changeset(%{ name: row.name, list_column_name: [ row.column_name ]})
      |> Repo.update()
    end)

    alter table :my_table do
      remove :column_name
    end

  end
end

LostKobrakai

LostKobrakai

You should try to not use Repo in migrations. You don’t want to fall in a situation, where you need a migration to be applied to be able to successfully start the repo, while the migration needs the repo to be applied.

The other reason is that the repo is started by your application, so your whole application will be started up – which might have other sideeffects – just for migrations to be applied. While without the repo it’s enough for your app to be loaded (for the credentials), but only ecto needs to be started.

dcrck

dcrck OP

Okay, is there a way to solve this problem without using Repo?

I see Repo used in Programming Ecto when changing both the table structure and the data, as is the case here.

LostKobrakai

LostKobrakai

There’s execute for executing arbitrary sql in migrations.

dimitarvp

dimitarvp

Yep, better use raw SQL, otherwise you might find yourself in a situation when the runtime doesn’t allow you to use Elixir code to do or rollback a migration.

dcrck

dcrck OP

Thanks @dimitarvp and @LostKobrakai, here’s what I ended up settling on:

defmodule Database.Repo.Migrations.StringToListMigration do
  use Ecto.Migration

  def change do
    execute(&string_to_list/0, &list_to_string/0)
  end

  defp string_to_list() do
    repo().query!("alter table my_table alter column_name type varchar(255)[] using array[column_name], alter column_name set default '{}';", [], [log: :info])
  end

  defp list_to_string() do
    repo().query!("alter table my_table alter column_name type varchar(255) using column_name[1], alter column_name drop default;", [], [log: :info])
  end
end

I do the renaming / other cleanup operations in a separate migration.

dimitarvp

dimitarvp

Mostly good, but remove the repo().query!(...) stuff from the front. The logging level has a default so you shouldn’t worry about it. Just use execute with both strings as they are and nothing else.

dcrck

dcrck OP

I see.

Actual final solution:

defmodule Database.Repo.Migrations.StringToListMigration do
  use Ecto.Migration

  def change do
    execute(string_to_list(), list_to_string())
  end

  defp string_to_list() do
    "alter table my_table alter column_name type varchar(255)[] using array[column_name], alter column_name set default '{}'"
  end

  defp list_to_string() do
    "alter table my_table alter column_name type varchar(255) using column_name[1], alter column_name drop default"
  end
end
samcdavid

samcdavid

For my situation, using alter table profiles alter ethnicity type varchar(255)[] using string_to_array(ethnicity, ','), alter ethnicity set default '{}' on a PostgreSQL database separated my string into a list for me.

Likewise, if you need to rollback you can do alter table profiles alter ethnicity type varchar(255) using array_to_string(ethnicity, ','), alter column_name drop default

Where Next?

Trending in Questions Top

stjefim
Hello! Suppose you are building workflow (order / task / payment) processing system with the following requirements: Each workflow con...
New
jonnycharles
I’m in search of an Elixir library that offers PDF generation capabilities similar to Ruby’s Prawn. While there have been discussions abo...
New
spammy
I’m looking to build a personal workflow to quickly deploy web applications written in elixir/phoenix, for local consumption (ie not on t...
New
silverdr
Using Phoenix.LiveView.TagEngine as an EEx.Engine is deprecated! To compile HEEx, use Phoenix.LiveView.TagEngine.compile/2 instead. Sta...
New
dli
Before I dive in myself, did anyone successfully sprinkle Hologram into their existing LiveView app? Looking for hints regarding: Addi...
New
bottlenecked
Hi all, I wanted to ask how the community is dealing with post-release steps. Today we have Ecto migrations, which make sure that the db...
New
michallepicki
I am using Oban and occasionally, shortly after a deployment, a handful of jobs can fail because of dependency on other parts of the syst...
New

Other Trending Topics Top

JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve. They are GUI (Emerge) and State management (S...
New
ausimian
Emily is an Elixir library that runs Nx computations on Apple’s MLX. Install it as the default Nx backend and Nx, defn, Axon, Nx.Serving,...
New
type1fool
I just stumbled on a newly redesigned elixir-lang.org. :tada: It looks like @Software_Mansion did the work, and I think it is generally a...
New
akoutmos
@hugobarauna and I (Alex Koutmos) have been hard at work on writing a book on Nerves that takes you from simply blinking LEDs to building...
New

We're in Beta

About us Mission Statement