Where do you preload associations?

Simple question re. code organization: do you do preloads in context, or in controller/liveview?

Say a basic function in context looks like this

def get_class!(id) do
    Repo.get!(Class, id)
    |> Repo.preload(:users)
end

But basically sometimes I need the association, sometimes I don’t.

So would you typically do get_class!(id) |> Repo.preload(:users) in controller/liveview? (I have a feeling that basically I should never call Repo… from controller/liveview)

Or would have separate get_class!/1 and get_class_with_assoc!/1 in context?

Or am I overthinking and it doesn’t really matter? Thank you.

3 Likes

I avoid calling repo from the controllers. My usual pattern is to do this in the context:

def get_class!(id, preloads \\ []) do
  Repo.get!(Class, id)
  |> Repo.preload(preloads)
end
7 Likes

Oooo, I’ve never thought about sending a preloads as a parameter. The only thing I don’t like this is that this gives LiveViews/Controllers some schema knowledge. I usually go the get_posts_with_user route. What’s your mileage been like with the extra argument, @stefanchrobot? I do like it in theory but I’ve only been thinking about it for 5 mins.

I’m fine with the coupling, because the schema knowledge is already there - whenever you access associations via the root schema, you introduce the coupling:

def show(conn, %{"id" => id} = params) do
  {:ok, foo} = Foo.get_foo_with_bar(id)
  # The coupling is in foo.bar
  render(conn, "show.json", foo: foo, bar: foo.bar)
end

In theory one could return {foo, bar} from the context, but I don’t think it’s worth it. Plus in reality, most of the time, the bar is linked with that specific foo either way and not just some free-floating entity.

In my mind, coupling when reading data structures and coupling when querying are different but actually, not really :thinking: That’s cool, thanks for the answer! I’m gonna try it out.

Another way to do it is to not to use preload altogether. Just create views that include the association at the db level and treat them as separate schemas in ecto. So you will have the following schemas:

  • class, which maps to the table
  • class_with_users, which maps to the view that joins class and users

Then you just have different context functions as usual.

I would say that preloading in the controller is not bad and you should not worry about doing it.
If my controller needs to preload two associations because they are going to be displayed in a template this is a presentation concern. If the template change, I may not need this preloads, or I may need different ones.

There always is the option of passing a preload list to the context as was already mentioned in this tread. I agree with your concern that this would couple the controller with the schema and data model.
I think that at that point we are building an abstraction over Ecto just to avoid using the Repo module in the controller. We may as well call Repo.preload directly and remove an indirection layer.

In my opinion we should focus more about extracting business logic into the context than in blindly following dogmas such as “never use the Repo from the controller”.

My preferred way to doing this is having different functions for building the query in the context (those functions should have some real business logic that makes their extraction worthy).
Then, the controller can use those functions from the context and its own functions (for presentation concerns) to build the desired query and trigger it using the Repo.
For example the controller may want to select a subset of the fields since they are the only ones required in the template, or preload some associations that are required for presentation. As long as the real business logic is extracted properly, this should be fine.

If you follow this pattern you may see that some of the queries that you build in controllers may have common points such as preloads. This is not bad per se, since tomorrow you can modify the controller or the template and know that you are not breaking any other part of the application. At that point, you will have enough information to decide if extraction is worthy or not.

Locality of Behaviour is very valuable, and we usually don’t think about it much.

4 Likes

Could you please elaborate on this? Maybe short example in (pseudo)code? Not sure I fully understand.

A view is a virtual table, see the postgresql doc here

If you define a view that flattens the foreign key association, you can just treat it as a normal table in ecto (you cannot insert into it though)

interesting. and you need to create the view using a migration with something like create view (:classes_with_users)? Is that supported in ecto? Or do I need to create that manually directly in Postgres?

3 Likes

This is a generally hard problem IMHO. One of my great open source regrets is not having enough time to make the Dataloader pattern more accessible outside of GraphQL because it was designed to address the following tension:

On the one hand: Presentation / API layers tend to want / need a lot of flexibility in terms of asking for information
On the other hand: Access control, filtering rules, and other data fetching code is often fraught with business logic, and this lives in the contexts.

If you allow the controllers to just pass in preload parameters you skip the business logic. It’s a tough problem. Two ways come to mind for how to solve this:

The dataloader pattern

Basically you create some sort of mediating entity that your business logic can hook into to enforce rules, and your “client” code can use to compose requests to fetch stuff. This works really well in Absinthe (GraphQL) because there is an actual query document to wire Dataloader into. It’s less ergonomic in traditional controllers and ends up being super callback heavy (at least today).

CQRS style Read Models

You create dedicated database views / tables where you can just query them in a simple way and the data has already been written such that it’s impossible to query data you shouldn’t see. This can quickly add a lot of boilerplate.

4 Likes

It depends - the need for preloads “sometimes” could mean you have two concepts and therefore should have two functions.

I have a with_assoc(schema, assoc) function in each context that just passes the association directly to the Repo.preload(assoc) function.

  def with_assoc(book, assoc), do: Repo.preload(book, assoc)

I use it directly in the controller/live_view directly

    book = Books.get_book_from_slug!(book_slug) |> Books.with_assoc([:author, :draft])

When I want to use the same logic somewhere else, I create a function called get_book_from_slug_with_author_draft(book) and continue.

Gives flexibility without sacrificing readability.

3 Likes