How to autogenerate new primary key in Ecto when inserting a changed changeset?

Questions / Help
,
1

I can’t work out how Ecto works with autogenerating a new primary key when inserting a changed changeset. I run afoul of its not-null and unique constraints and I’ve probably just stared myself blind at this point. Hoping y’all have some opinions and knowledge I can lean into.

Given a simple schema like this:

defmodule Switch do
  use Ecto.Schema
  import Ecto.Changeset

  schema "switches" do
    field :switch_id, :string
    field :state, :boolean, default: false

    timestamps(updated_at: false, type: :utc_datetime_usec)
  end

  def changeset(schema \\ %__MODULE__{}, params) do
    schema
    |> cast(params, [:switch_id, :state])
    |> validate_required([:switch_id])
  end
end

I now want a context module that lets me:

  • Get Switch changesets (so a page can put them into forms).
  • Insert a changed changeset such that it becomes a new row (thereby “soft-deleting” the previous row).

Below is how I want the context module to work, but since it doesn’t work it means I’m misunderstanding something about how Ecto treats nil values and autogenerating keys.

defmodule Switches do
  import Ecto.Changeset
  import Ecto.Query
  alias Switch

  # Data Retrieval

  def get_switch(switch_id) do
    from(switch in Switch)
    |> where([switch], switch.switch_id == ^switch_id)
    |> order_by(desc: :id)
    |> limit(1)
    |> Repo.one()
  end

  # Validation & Insertion

  def prepare_switch(switch_id) do
    case get_switch(switch_id) do
      nil -> Switch.changeset(%{switch_id: switch_id})
      switch -> Switch.changeset(switch, %{})
    end
  end

  def insert_switch(changeset) do
    Repo.insert(changeset)
  end

  def change_switch(data \\ %Switch{}, params) do
    changeset = Switch.changeset(data, params)

    if changeset.changes == %{} do
      changeset
    else
      changeset |> put_change(:id, nil)
    end
  end
end

With this, a new switch can be inserted:

iex> prepare_switch("foo")
     |> insert_switch()
 %Switch{
   __meta__: #Ecto.Schema.Metadata<:loaded, "switches">,
   id: 1,
   switch_id: "foo",
   state: false,
   inserted_at: ~U[2024-07-08 16:24:59.157137Z]
 }}

But I run afoul of constraint errors when inserting a change:

iex> prepare_switch("foo")
     |> change_switch(%{state: true})
     |> insert_switch()
** (Postgrex.Error) ERROR 23502 (not_null_violation) null value in column "id" of relation "switches" violates not-null constraint
    table: switches
    column: id
Failing row contains (null, foo, f, 2024-07-08 16:24:59.157137).

Now, I thought setting id to nil in the change_switch function would cause Ecto to generate a new ID, but it seems to take the nil as a literal null-insertion. Whoops, my bad.

But then how are we supposed to create new entries from an existing changeset?

The migration, if that's relevant 
  def change do
    create table(:switches) do
      add :switch_id, :string, null: false
      add :state, :boolean, default: false, null: false

      timestamps(updated_at: false, type: :utc_datetime_usec)
    end
  end
2

Okay, now that I spend some time debug in full detail I can see it doesn’t work to put_change(:id, nil) because that puts the nil into changes, but the below code does work because it now sets data id to nil:

  def change_switch(data \\ %Switch{}, params) do
    changeset = Switch.changeset(data, params)

    if changeset.changes == %{} do
      changeset
    else
      changeset.data
      |> Map.put(:id, nil)
      |> change()
      |> Switch.changeset(params)
    end
  end

That’s not so bad, but I guess I’ve been too busy looking for something more along the lines of Ecto.Changeset functions.

Is messing with changeset.data normal, and I’m just staring at something that’s fine to do? It’s hard for me to tell if this is clever or ugly hacking :sweat_smile: What do you think?

3

I‘d be explicit and build a changeset for a new row adding the data of the existing record as changes – truly constructing a new record – instead of trying to strip an existing record from being identified as already existing.

5 Likes
4

I do like the sound of that. Your reply led me to this solution, is that the direction you had in mind?

      {_, switch_id} = fetch_field(changeset, :switch_id)
      Switch.changeset(%Switch{switch_id: switch_id}, changeset.changes)
5

If the changes include all the fields you need to set on the new record.

6

I went back to the “trying to strip an existing record” well, and came away with this snippet:

struct(
  Switch, 
  Map.from_struct(changeset.data) |> Map.drop([:id]))
|> Switch.changeset(changeset.changes)

It reconstructs changeset’s data without the unique constraint properties, and then uses that as the base for re-applying the changes.

I know it goes against @LostKobrakai’s good suggestion (I too like explicit) but just mentioning it here as I explore how different patterns work.

7

Is it not easier to do that logic inside of a custom changeset function? For example you can have changeset_new besides your changeset function.

1 Like
8

Hm, yes. I vaguely feel a schema is for storage-details and not the context that entity lives in, and in this case the business-logic-context is that switching creates an audit-trail of Switch rows. Thus I kept away from imbuing the changeset with such knowledge.

But I see your point. In your suggestion, do you imagine an external module calling schema.new_changeset directly, or would it call via the context module? (in that case: would there be corresponding two change functions in context, or still just one and it calls one of the changeset functions…?)

Or, I’m just as happy to hear if you don’t use context modules, I’m happy to learn and deepen my understanding of the different ways is preferred to be solved.

9

It highly depends on how your boundary logic is structured, but I see nothing wrong with calling such changeset functions directly. For example for phoenix forms, I would have a separated changeset for the form that I would call directly from phoenix, but this is due the fact that my only interest was data validation. In your case, if you do actual DB operations, it would be optimal to have them located in your context, as to not leak your Repo and custom queries everywhere.

1 Like