Repo.preload on structs mixes up data

Questions / Help
1

Given the following schema’s

defmodule EctoTest.User do
  use Ecto.Schema
  @primary_key {:id, Ecto.UUID, autogenerate: true}
  schema "users" do
    field :name, :string
    has_many :duties, EctoTest.Duty
  end
end

defmodule EctoTest.Duty do
  use Ecto.Schema
  @primary_key {:id, Ecto.UUID, autogenerate: true}
  schema "duties" do
    field :name, :string
    belongs_to :role, EctoTest.Role, type: Ecto.UUID
    belongs_to :user, EctoTest.User, type: Ecto.UUID
  end
end

defmodule EctoTest.Role do
  use Ecto.Schema
  @primary_key {:id, Ecto.UUID, autogenerate: true}
  schema "roles" do
    field :name, :string
    has_many :permissions, EctoTest.Permission
  end
end

defmodule EctoTest.Permission do
  use Ecto.Schema
  schema "permissions" do
    field :name, :string
    belongs_to :role, EctoTest.Role, type: Ecto.UUID
  end
end

Let’s say we have the following, which works as expected:

EctoTest.User
|> EctoTest.Repo.get(user_id)
|> EctoTest.Repo.preload(duties: [role: [:permissions]])

Now, if we do:

user = EctoTest.Repo.get(EctoTest.User, user_id)
role1 = EctoTest.Repo.get(EctoTest.Role, role1_id)
dynamic_duty1 = %EctoTest.Duty{name: "Dynamic duty 1", user: user, role: role1}
user_with_a_dynamic_duty = %{user | duties: [dynamic_duty1]}

user_with_a_dynamic_duty
|> EctoTest.Repo.preload(duties: [role: [:permissions]])
|> IO.inspect

This still works as expected (the correct permission is loaded)

%EctoTest.User{__meta__: #Ecto.Schema.Metadata<:loaded, "users">,
 duties: [%EctoTest.Duty{__meta__: #Ecto.Schema.Metadata<:built, "duties">,
   id: nil, name: "Dynamic duty 1",
   role: %EctoTest.Role{__meta__: #Ecto.Schema.Metadata<:loaded, "roles">,
    id: "597302aa-7abd-499d-bcea-c3c7f3e1a025", name: "Role 1",
    permissions: [%EctoTest.Permission{__meta__: #Ecto.Schema.Metadata<:loaded, "permissions">,
      id: 105, name: "Permission 1",
      role: #Ecto.Association.NotLoaded<association :role is not loaded>,
      role_id: "597302aa-7abd-499d-bcea-c3c7f3e1a025"}]}, role_id: nil,
   user: %EctoTest.User{__meta__: #Ecto.Schema.Metadata<:loaded, "users">,
    duties: [], id: "1ba5ac9c-85be-497c-ae5f-6dbf68837e72", name: "User"},
   user_id: nil}], id: "1ba5ac9c-85be-497c-ae5f-6dbf68837e72", name: "User"}

But now, if we would have multiple dynamic duties, pointing to different roles and permissions:

user = EctoTest.Repo.get(EctoTest.User, user_id)
role1 = EctoTest.Repo.get(EctoTest.Role, role1_id)
role2 = EctoTest.Repo.get(EctoTest.Role, role2_id)
dynamic_duty1 = %EctoTest.Duty{name: "Dynamic duty 1", user: user, role: role1}
dynamic_duty2 = %EctoTest.Duty{name: "Dynamic duty 2", user: user, role: role2}
user_with_dynamic_duties = %{user | duties: [dynamic_duty1, dynamic_duty2]}

user_with_dynamic_duties
|> IO.inspect
|> EctoTest.Repo.preload(duties: [role: [:permissions]])
|> IO.inspect

This results in the following two IO.inspects. The first one is as expected, but the second one not.

Notice that the duty list now contains two times “Dynamic duty 1”, pointing to “Role 1”, pointing to “Permission 1” instead of the expected “Dynamic duty 2”, pointing to “Role 2”, pointing to “Permission 2”.

%EctoTest.User{__meta__: #Ecto.Schema.Metadata<:loaded, "users">,
 duties: [%EctoTest.Duty{__meta__: #Ecto.Schema.Metadata<:built, "duties">,
   id: nil, name: "Dynamic duty 1",
   role: %EctoTest.Role{__meta__: #Ecto.Schema.Metadata<:loaded, "roles">,
    id: "9533b6b6-c7ba-4a46-b8a1-89558eedd203", name: "Role 1",
    permissions: #Ecto.Association.NotLoaded<association :permissions is not loaded>},
   role_id: nil,
   user: %EctoTest.User{__meta__: #Ecto.Schema.Metadata<:loaded, "users">,
    duties: [], id: "ca7f7c8a-b87d-4643-b867-52575f1a8e64", name: "User"},
   user_id: nil},
  %EctoTest.Duty{__meta__: #Ecto.Schema.Metadata<:built, "duties">, id: nil,
   name: "Dynamic duty 2",
   role: %EctoTest.Role{__meta__: #Ecto.Schema.Metadata<:loaded, "roles">,
    id: "a1fee564-fcaa-4a0d-b810-32628c7b330b", name: "Role 2",
    permissions: #Ecto.Association.NotLoaded<association :permissions is not loaded>},
   role_id: nil,
   user: %EctoTest.User{__meta__: #Ecto.Schema.Metadata<:loaded, "users">,
    duties: [], id: "ca7f7c8a-b87d-4643-b867-52575f1a8e64", name: "User"},
   user_id: nil}], id: "ca7f7c8a-b87d-4643-b867-52575f1a8e64", name: "User"}

Second IO.inspect, containing two times the first duty, role and permission.

%EctoTest.User{__meta__: #Ecto.Schema.Metadata<:loaded, "users">,
 duties: [%EctoTest.Duty{__meta__: #Ecto.Schema.Metadata<:built, "duties">,
   id: nil, name: "Dynamic duty 1",
   role: %EctoTest.Role{__meta__: #Ecto.Schema.Metadata<:loaded, "roles">,
    id: "546c111e-250f-4169-b907-19a4132992fd", name: "Role 1",
    permissions: [%EctoTest.Permission{__meta__: #Ecto.Schema.Metadata<:loaded, "permissions">,
      id: 117, name: "Permission 1",
      role: #Ecto.Association.NotLoaded<association :role is not loaded>,
      role_id: "546c111e-250f-4169-b907-19a4132992fd"}]}, role_id: nil,
   user: %EctoTest.User{__meta__: #Ecto.Schema.Metadata<:loaded, "users">,
    duties: [], id: "3157ddfa-07b0-4c93-8b11-4dc7b1d1dfe9", name: "User"},
   user_id: nil},
  %EctoTest.Duty{__meta__: #Ecto.Schema.Metadata<:built, "duties">, id: nil,
   name: "Dynamic duty 2",
   role: %EctoTest.Role{__meta__: #Ecto.Schema.Metadata<:loaded, "roles">,
    id: "546c111e-250f-4169-b907-19a4132992fd", name: "Role 1",
    permissions: [%EctoTest.Permission{__meta__: #Ecto.Schema.Metadata<:loaded, "permissions">,
      id: 117, name: "Permission 1",
      role: #Ecto.Association.NotLoaded<association :role is not loaded>,
      role_id: "546c111e-250f-4169-b907-19a4132992fd"}]}, role_id: nil,
   user: %EctoTest.User{__meta__: #Ecto.Schema.Metadata<:loaded, "users">,
    duties: [], id: "3157ddfa-07b0-4c93-8b11-4dc7b1d1dfe9", name: "User"},
   user_id: nil}], id: "3157ddfa-07b0-4c93-8b11-4dc7b1d1dfe9", name: "User"}

If we construct dynamic duties as follows:

%EctoTest.Duty{name: "Dynamic duty 1", user: user, role: role1, role_id: role1.id}

Instead of

%EctoTest.Duty{name: "Dynamic duty 1", user: user, role: role1}

This all works as expected.

My conclusion so far

As far as I can understand from this,

  • Ecto supports two ways of determining the role id during preloading: Either through the association (duty.role.id) or trough the field (duty.role_id). It seems that it doesn’t work properly if it is determined through the association.

  • I’m asking preload to preload permissions ( duties: [role: [:permissions]]). Should it touch the role then?

  • Also, i’m not sure if it’s even supported though associations, or even if this whole trickery with dynamic (in-memory-generated) duties is supported in this way.

I’ve created https://github.com/mkarnebeek/ecto_preload_bug where you can just run mix test which does the same as described here and experiment with putting IO.inspect in various places

As a permanent solution, i’m thinking about never instantiating dynamic duties using the struct syntax, but always through a function on that schema. So that can make sure that role_id is filled when specifying a role, but i wouldn’t be surprised that something like this is already present in Ecto (cast ?)

Thanks for the help!

2 Likes
2

I’ll try to look into it over the weekend. In general, though, preload was never intended to work on hand-crafted data, so it doesn’t really surprise me something’s off. That said, it definitely shouldn’t be producing bad data.

2 Likes