How to autogenerate new primary key in Ecto when inserting a changed changeset?

I can’t work out how Ecto works with autogenerating a new primary key when inserting a changed changeset. I run afoul of its not-null and unique constraints and I’ve probably just stared myself blind at this point. Hoping y’all have some opinions and knowledge I can lean into.

Given a simple schema like this:

defmodule Switch do
  use Ecto.Schema
  import Ecto.Changeset

  schema "switches" do
    field :switch_id, :string
    field :state, :boolean, default: false

    timestamps(updated_at: false, type: :utc_datetime_usec)

  def changeset(schema \\ %__MODULE__{}, params) do
    |> cast(params, [:switch_id, :state])
    |> validate_required([:switch_id])

I now want a context module that lets me:

  • Get Switch changesets (so a page can put them into forms).
  • Insert a changed changeset such that it becomes a new row (thereby “soft-deleting” the previous row).

Below is how I want the context module to work, but since it doesn’t work it means I’m misunderstanding something about how Ecto treats nil values and autogenerating keys.

defmodule Switches do
  import Ecto.Changeset
  import Ecto.Query
  alias Switch

  # Data Retrieval

  def get_switch(switch_id) do
    from(switch in Switch)
    |> where([switch], switch.switch_id == ^switch_id)
    |> order_by(desc: :id)
    |> limit(1)

  # Validation & Insertion

  def prepare_switch(switch_id) do
    case get_switch(switch_id) do
      nil -> Switch.changeset(%{switch_id: switch_id})
      switch -> Switch.changeset(switch, %{})

  def insert_switch(changeset) do

  def change_switch(data \\ %Switch{}, params) do
    changeset = Switch.changeset(data, params)

    if changeset.changes == %{} do
      changeset |> put_change(:id, nil)

With this, a new switch can be inserted:

iex> prepare_switch("foo")
     |> insert_switch()
   __meta__: #Ecto.Schema.Metadata<:loaded, "switches">,
   id: 1,
   switch_id: "foo",
   state: false,
   inserted_at: ~U[2024-07-08 16:24:59.157137Z]

But I run afoul of constraint errors when inserting a change:

iex> prepare_switch("foo")
     |> change_switch(%{state: true})
     |> insert_switch()
** (Postgrex.Error) ERROR 23502 (not_null_violation) null value in column "id" of relation "switches" violates not-null constraint
    table: switches
    column: id
Failing row contains (null, foo, f, 2024-07-08 16:24:59.157137).

Now, I thought setting id to nil in the change_switch function would cause Ecto to generate a new ID, but it seems to take the nil as a literal null-insertion. Whoops, my bad.

But then how are we supposed to create new entries from an existing changeset?

The migration, if that's relevant
  def change do
    create table(:switches) do
      add :switch_id, :string, null: false
      add :state, :boolean, default: false, null: false

      timestamps(updated_at: false, type: :utc_datetime_usec)

Okay, now that I spend some time debug in full detail I can see it doesn’t work to put_change(:id, nil) because that puts the nil into changes, but the below code does work because it now sets data id to nil:

  def change_switch(data \\ %Switch{}, params) do
    changeset = Switch.changeset(data, params)

    if changeset.changes == %{} do
      |> Map.put(:id, nil)
      |> change()
      |> Switch.changeset(params)

That’s not so bad, but I guess I’ve been too busy looking for something more along the lines of Ecto.Changeset functions.

Is messing with normal, and I’m just staring at something that’s fine to do? It’s hard for me to tell if this is clever or ugly hacking :sweat_smile: What do you think?

I‘d be explicit and build a changeset for a new row adding the data of the existing record as changes – truly constructing a new record – instead of trying to strip an existing record from being identified as already existing.


I do like the sound of that. Your reply led me to this solution, is that the direction you had in mind?

      {_, switch_id} = fetch_field(changeset, :switch_id)
      Switch.changeset(%Switch{switch_id: switch_id}, changeset.changes)

If the changes include all the fields you need to set on the new record.

I went back to the “trying to strip an existing record” well, and came away with this snippet:

  Map.from_struct( |> Map.drop([:id]))
|> Switch.changeset(changeset.changes)

It reconstructs changeset’s data without the unique constraint properties, and then uses that as the base for re-applying the changes.

I know it goes against @LostKobrakai’s good suggestion (I too like explicit) but just mentioning it here as I explore how different patterns work.

Is it not easier to do that logic inside of a custom changeset function? For example you can have changeset_new besides your changeset function.