marick

marick

Ecto preload conventions

Are there coding/naming conventions that make it more likely that when Ecto-using code is given an Animal object, it’s an Animal object with all the right associations preloaded?

My inclination is just to always preload all the Animal’s fields. (In my case, I don’t yet have trees with more than two meaningful levels.) I’m curious what other people do.

Most Liked Responses

xpg

xpg

That’s a really interesting question. I don’t have an answer for you, but I can share my situation with you. I am working with a code-base that has around 50 database tables, many of them related to a few central tables.
That has lead to a situation where I have a few models that have large number of associations (even some join_through associations).

To begin with, I tried to use the lazy preload approach, i.e. only preload what I need. It quickly turned out that there were preloads that I always used, so I ended up with a “default_preloads” function for each model. Next, I identified that in many places I preloaded the same associations recursively (i.e. [assoc_A: [sub_assoc_A: [:field, :field]]]), which lead to yet a number of preload functions.
This has become somewhat of a mess, where its difficult to know if preloading for a given associations has been performed or not. The result is that almost each function working on the data starts by preloading what it needs, and often preloads more than it needs. Also, once you start preloading an association, you need to consider how deep do you do the preloading.
When introducing new developers to the code base, they have a difficult time figuring out what is supposed to be preloaded when. That is not a good situation.

The approach I am trying to take now is to identify which associations can really be considered as a required part of the model, and are so often needed that they may as well be loaded from the beginning.
The remaining associations I consider removing from the model, and load using explicit loading functions, and pass along on the side of the model struct. This way it becomes much more clear what data a function needs, and I can avoid having pure-functional functions suddenly needing to perform database reads in order fetch the required data.

Imagine having a blog system, where it is possible to add comments. The comments for each blog post do not really need to be loaded as part of the blog post itself. Of course having them as an association makes it easy to load them when needed. However, once the blog post has 10 more associations it becomes tricky to figure out what is preloaded and what should be preloaded.

I’m not sure that the approach I am trying to take is the right one, but I can see that as our code base (and database structure) has evolved over time, things become difficult to manage.

axelson

axelson

Scenic Core Team

I think this makes a lot of sense. It sounds similar to the approach of Domain Driven Design where you define an “Aggregate” which is:

[[Aggregate]] A cluster of associated objects that are treated as a unit for the purpose of data changes. External references are restricted to one member of the AGGREGATE, designated as the root. A set of consistency rules applies within the AGGREGATE’S boundaries.

So for each “Aggregate” that you identify you’d always have the same preloaded data. Yes it won’t always be the absolute most performant (since you might preload data you don’t need), but the maintainability will be much higher.

This is an approach I would like to move to, but since I’m using Absinthe/GraphQL/Dataloader so heavily I am generally not using preloads.

Where Next?

Popular in Discussions Top

jeramyRR
This is an interesting article to read. Elixir’s performance, like usual, is excellent. However, it seems like the high CPU usage is co...
New
lucaong
Hello Elixir and Nerves community, I have been working for a while on an open-source embedded key-value database for Elixir, that I call...
230 13924 124
New
CharlesO
Erlang :list.nth simple, but 1 - based nth(1, [H|_]) -> H; nth(N, [_|T]) when N > 1 -> nth(N - 1, T). Elixir Enum.at … coo...
New
tmbb
This is a post to discuss the new Phoenix LiveView functionality. From Chris’s talk, it appears that they generate all HTML on the serve...
342 18146 126
New
sergio
There’s a new TIOBE index report that came out that shows Elixir is still not in the top 50 used languages. It also goes on to call Elix...
New
hazardfn
I suppose this question is effectively hackney vs. ibrowse but we are at a point in our project where we have to make a choice between th...
New
und0ck3d
Hello everyone! A few days ago I’ve created a topic here about how people were creating CMSs with Elixir and Phoenix. I’ve been studying...
New
pdgonzalez872
If this has been asked here before, please point me to where it was asked as I didn’t find it when I searched the forum. Maybe a mailing ...
New
kostonstyle
Hi all How can I compare haskell with elixir, included tools, webservices, ect. Thanks
New
paulanthonywilson
I like Umbrella projects and pretty much always use them for personal Elixir stuff, especially Nerves things. But I don’t think this is ...
New

Other popular topics Top

sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
boundedvariable
I am going through the kafka architecture. All the features what the kafka is providing are already in Erlang. I would like hear your opi...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New

We're in Beta

About us Mission Statement