Preloading nested associations

Questions / Help
1

There seem to be a lot of questions regarding this, but no one actually answers the actual question I have.

I have some nested “objects” that are representad by Ecto schemas, each in its own database table and joined together with foreign keys. The structuere is something like the following (about 3 levels deep, with has_many relations):

%Project{
  tables: [
    %ProjectTable{
      columns: [
        %TableColumn{
          ...
        },
      ]
    },
    ...
  ]
}

I want to put everything in a form using Phoenix, ecto and some custom components I’ve written (similar to inputs_for), but for that I need to be able to preload the associations in the changesets. Everything is easy if one wants to preload a number of associations in structures, but I can’t find anything that works on chnagesets, and I wouldn’t like to manually deconstruct and reconstruct changesets for this.

Is there any way of being able to preload this without writing too much code of my own?

2

Not sure exactly what you mean by this but you need to set up your changesets for each schema then you can just call the parent one to get the whole tree:

defmodule Project do
  def changeset(project, attrs) do
    project
    |> cast(attrs, [...])
    |> cast_assoc(:project_tables)
  end
end

defmodule ProjectTable do
  def changeset(project_table, attrs) do
    project_table
    |> cast(attrs, [...])
    |> cast_assoc(:table_columns)
  end
end

defmodule TableColumn do
  def changeset(table_column, attrs) do
    table_column
    |> cast(attrs, [...])
  end
end

then

Project.get(1)
|> Repo.preload([project_tables: :table_columns])
|> Project.changeset(%{...})
3 Likes
3

I did try exactly what you’re saying, but the code you’ve shown only the preloads in the :project_tables and not in the subsequent associations. I’ll put a reproducible example here.

4

That’s very strange. It doesn’t load :table_columns or ones below that? And you’re doing [project_tables: :table_columns] and not [:project_tables, :table_columns]? Also, you have a changeset/2 on the ones below that take schema and attrs?

5

Ok, maybe I wasn’t explicit enough. The problem happens when I attempt to create child (or grandchild) elements with the new :sort_param and :drop_param functionality. Even if I preload the parent, when I create a new child, the child will not have its associations preloaded (these would be the grandchildren). If I want to use something like <nested_inputs_for/> with the child, then the component will raise an error because the association of the child (which contiains the grandchildren) will not be a list but instead will be an #<AssociationNotPreloaded ...> thing…

To solve this, I have to walk down the associations chain through the changesets and manually preload everything in the data. The naive way fires up an enormous amount of queries for the general case and optimizing it is not simple (I’d have to keep a map from the “path” of the value to the data itself, preload everything at once and then rebuild the changeset “tree” from the preloaded data). The code to do this is certainly not trivial, and I think it whould be something that should be handled by Ecto itself. Since Ecto set itself to handle creation and deletion of nested associations, I think it should be responsible for making sure those newly created values are preloaded whenever we wanted them to be peloaded in the parent.

I believe I might be the only one having this problem because I might be the one building a UI which is effectively a nested tree 4 levels deep.

The project itself is a data capture platform (think RedCap but in which I try to solve some problems with RedCap). The user can create projects, a project can contain a number of tables, a table can contain a number of (sorted) columns, and columns may be enumerations which may contain a number of sorted items. On the UI side, this works by having a number of nested components that work like <nested_inputs_for/> inside a LiveView. So far, Ecto has served me well, but I think that working with Ecto changesets that represent forms that are nested so deeply might be stretching it a bit much (?).

I’ll try to come up with a general solution and maybe submit it to Ecto as PR.

6

TL;DR, you might find this helpful: GitHub - woylie/ecto_nested_changeset: Helpers for manipulating nested Ecto changesets

If you are talking about the children of new records, then you just need to “build” them with Ecto.build_assoc.

The Form struct comes with an index field and you can use that to build up compound keys to easily find the node you want to update. It is a bit of extra code, yes, but that is where that lib might come in handy. I haven’t actually used it myself but it’s by (at)woylie so it’s gotta be good :sweat_smile: I also had a 4-deep tree in a form and it’s really not too bad updating it manually though I do agree would be nice if Ecto came with something itself!

I may still be missing a piece of what you’re looking for, though. It would def help to share code if you can!

1 Like