tielur

tielur

Assumptions:

  • All Repo calls are isolated to the context’s within your application

I have lots of context methods that several parts of my phoenix applications use. Depending on what I’m doing with resources I may need different relationships preloaded on it. Is it common to just preload all resources on the context methods with whatever everything needs access to?

Or are people using Repo.preload outside of context’s to load up the additional data they may need when and where they need it?

What I have right now is lots of data that is being queried and loaded and sometimes that data isn’t even being used. A few alternatives I’ve thought about over the past few weeks:

  • Add additional public functions for each different way data is being used. I’m not a huge fan of this idea only because the surface area of the public API into your system becomes large. Though it’s tame-able with using ecto composable queries to reuse a lot of internal building of the queries.

    • Users.get_user(id)
    • Users.get_user_with_accounts(id)
    • Users.get_user_with_accounts_and_posts(id
  • Expose public composable functions that the “clients”(code needing the data) put together with whatever data they need. This is nice in that the clients are able to ask for only what data they need, what I don’t like is now you need to pass the query back to the context for it to execute ( assuming that all Repo calls are isolated to the context modules).

    • id |> Users.with_id() |> Users.get()
    • id |> Users.with_id() |> Users.with_accounts() |> Users.get()
    • id |> Users.with_id() |> Users.with_accounts() |> Users.with_posts() |> Users.get()

    Where with_id/1, with_accounts/0, and with_posts/0 are all composable queries that return the ecto query and Users.get/1 would take a query and basically call Repo.one

  • I haven’t looked that far into GraphQL but it would be interesting to see if that concept where the client could ask for what it needs and the server would just hand it back could be brought into this situation. Where my “client” another module could give a graphql query to my context and it would give back what it’s asking for. I’m not sure if this is possible now?

How are you handling this in your elixir applications?

Showing Posts 1 to 5

tielur

tielur OP

Another idea would be to allow the passing in of the Keyword list of preloads into the public context methods. Something like:

def get_user(id, preloads \\[])
  Repo.get(User, id)) |> Repo.preload(preloads) 
end

So I could get the accounts with:
Users.get_user(id, [:accounts]

or the accounts and posts with:
Users.get_user(id, [:accounts, :posts])

josevalim

josevalim

Creator of Elixir

Yes. However, keep in mind you can always preload after the fact too, so you can do this:

User.get_user!(...) |> Blog.preload_posts_for_user(...) |> Other.preload_another_for_user(...)
LostKobrakai

LostKobrakai

I feel like managing preloads or more generally flexible querying is certainly one of the more complex parts related to contexts. I’d also say you’re likely to not get black or white answers to the options you gave, but it’s probably more of a “use a solution that suits the requirements”.

One idea I want to propose is of having an aggregate root. This would mean a context does only handle data from one or few root datatype(s), like e.g. an Order, while access to child data (like e.g. OrderAddress or OrderLine) is transparently handled by the context either by upfront loading everything or loading on demand. From the callers perspective there’s nothing extra to do. This is probably the most simple and sane approach.

Your options about having an API for the caller to select what’s needed to be preload doesn’t really seem desirable to me. E.g. why should Users know about posts (besides that I don’t really like Users as a context name). A more domain focused api would rather look like:

user = Accounts.get_user(id)
posts = Blog.get_posts(user)

Given that I’d refrain from preloading in ecto queries in favor of preloading via second queries as much as possible (preloads and filtering often don’t go well together) this will result in two queries to the db just like a single call to a domain api would as well.

The last part I want to cover is flexible querying of data in general. There you have basically two options, like you seem to have already discovered based on your proposed solutions:

  1. Find a way to have a custom data structure represent the querying options available:
    This can be a function name or e.g. a keyword list/map passed as additional parameter.
  2. Have a composable API using the builder pattern, which builds up a data structure (personally I’d directly build up an ecto query) to be supplied to various context functions for actually querying the database. Given that both MyApp.Repo as well as MyApp.Context kinda implement a repository pattern it’s no wonder that the APIs might be similar even though on vastly different abstraction levels.

Also I’d like to leave this here, which might clear up what I don’t like about a Users context and how to make potentially very big context files smaller: http://devonestes.herokuapp.com/a-proposal-for-context-rules

cnck1387

cnck1387

As someone fairly new to Elixir / Phoenix, I think passing in the preloads as an optional list is a pretty clean approach with a nice balance between being flexible and explicit while reducing boilerplate.

But @josevalim brings up another good point too. If you have 1 get_user function that doesn’t preload, then you pipe in the preloads as needed in your “client” (usually controller) calls, is that better in the long run? It’s a little more boilerplate but it’s even more explicit and probably equally as flexible?

Would you say the no list argument while piping in the preloads as needed is more idiomatic Elixir?

The only problem I see then is you either need to expose Repo to your controller (partially defeating the purpose of contexts) so you can do the preloads there, or you wind up making a bunch of extra functions in your contexts for preloading?

baldwindavid

baldwindavid

I just released a package with the aim of providing a pattern and API exactly for this purpose if it is useful to anyone… TokenOperator - Dependency-free helper most commonly used for making clean keyword APIs to Phoenix context functions

The concept here is to use keywords that end up mapping to functions within the context.

— All posts loaded —

Where Next? Top

Trending in Questions Top

katta
I having some trouble figuring out if I have set myself too strict of standards for my production server. Currently I can handle 75% of r...
New
brecabral
Documentation While reading the Scoped Routes section, I noticed that the documentation currently refers to a problem without explainin...
New
achenet
Hello, I’m trying to build a basic Phoenix web-app, and I’d like to use Tailwind. However, when I launch mix phx.server, I get an error...
New
kpanic
Hi everyone, I am toying with the idea of building a “match maker” for giving personal help to people that wants to start coding. I sta...
New
Cxx-mlr
I’m working on a small exercise involving update_in/3, and I came up with this solution: data = %{ name: "Periodic Table", category:...
New
ChrisAmelia
I’ve got trouble wrapping my head around the order in which functions are called in this snippet (from Phoenix’s authentication): toke...
New
dillonoconnor
Is there any way to avoid the Hologram compiler running when using iex? It seems like the front-end code could potentially be disregarded...
New

Other Trending Topics Top

GenericJam
Edit: 2026 May 15 - This post is archived. Mob is alive!! Main docs: mob v0.7.11 — Documentation A bit of explanation for the slightly c...
New
mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
garrison
Hobbes is a low-level distributed database for the Elixir programming language. Hobbes provides a simple, safe, and scalable storage lay...
New
budgie
A little off-topic, but I feel like people here have a good head on their shoulders. I used to be quite good at making software. Was luc...
New
KristerV
Hey. Is there anyone here who creates agents in their apps? Not talking about using agents, but creating them. I’m finding it pretty diff...
New
mcass19
ExRatatui lets you cook up rich terminal UIs in Elixir, powered by Rust’s ratatui via Rustler NIFs. Build interactive terminal applicatio...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews