How best to write context functions for more complex cases?

I’m not sure there’s much consense of how exactly to write that portion of you codebase.

The best way to prepare for future requirements is make it easy to throw away the current code and replace it wholesale (https://www.youtube.com/watch?v=1FPsJ-if2RU). There’s no way to know what future requirements will be, so trying to cater to them is guesswork at best.

On a general note I’d always suggest to start with less abstraction (less complex parameters) and more distinct functions than the other way round. It leads to the above, but also means you discover useful abstractions rather than imagine them to be useful. Simpler more distinct functions should help against “just let this existing function do one more thing”.

Your example would lend itself to extacting common query manipulating functions into their own module. My suggestion for learning how to do layering in code (without necessarily buying into buzzword architecture) would be buying “grokking functional programming”. It has a few great chaptures on how to build larger stuff out of smaller pieces and how those layers should depend on each other (or not).

It definitely felt like I was writing an unnecessarily general get_post/2 function. In reality I currently only need to be able to do the following.

Posts.get_post(
        post_id,
        preload_user: [:id, :avatar, :username],
        preload_replies: true,
        preload_subreplies: true,
        with_reply_count: true
      )

I got tempted…

Maybe silly, but thinking of a good name for the function that would be equivalent to

Posts.get_post(
        post_id,
        preload_user: [:id, :avatar, :username],
        preload_replies: true,
        preload_subreplies: true,
        with_reply_count: true
      )

threw me off.

If it were me I would create my queries separately to my context functions and build out the semantic context functions using the queries and schema modules.

In my query module I would expose those preload functions and let the caller (the context module) decide what they need by chaining them together. I would allow specifying options to those query functions (like fields to return), and perhaps some sort and aggregate helpers also.

Context should be high level enough that for callers (eg LiveView or controller actions) it would not matter if your entire backend storage layer changed. Generally I think of context methods as orchestrating actions. If they involve multiple resources or a mix of ecto, external apis, sending email or pubsub notifications it shouldn’t matter.

Like so?

@doc """
Gets data of a post that is necessary for rendering  a dialog component.
  
## Options
  ...all (preload) options...
  ...preloading is not optional anymore, since the function is now specialized...

"""
  def get_post_for_dialog(id, opts \\ []) do
    from(p in Post, where: p.id == ^id)
    |> Posts.Query.add_reply_count()
    |> Posts.Query.preload_user(opts)
    |> Posts.Query.preload_replies(opts)
    |> Posts.Query.preload_subreplies(opts)
    |> Repo.one()
  end

Or without passing the opts to get_post_for_dialog? Like this. So reading the function becomes more semantic (explains more completely what to expect the function to do/return)?

  def get_post_for_dialog(id) do
    from(p in Post, where: p.id == ^id)
    |> Query.add_reply_count()
    |> Query.preload_user(fields: [:id, :avatar, :username])
    |> Query.preload_replies(with_user?: true, with_subreply_count?: true)
    |> Query.preload_subreplies(with_user?: true)
    |> Repo.one()
  end

Edit: I use a UI concept in the function name. Is that cursed?

We are doing something similar at work, but we generate all the functions and they follow the same naming convention.

The problem with

  def get_post(id, opts \\ []) do
    from(p in Post, where: p.id == ^id)
    |> add_reply_count(opts)
    |> preload_user(opts)
    |> preload_replies(opts)
    |> preload_subreplies(opts)
    |> Repo.one()
  end

is that you can’t reuse it for other contexts, as it is too specific for the Post schema.

We have something like this in all contexts:

use MyApp.Context,
    queries: MyApp.Queries.ActionQueries,
    schema: MyApp.Schemas.Action

And it generates def get_action(action_id, opts \\ []) and def list_actions(opts \\ []) and some other stuff based on the schema name.

That makes a lot of sense, though that’s easier to do when the code is quicker to write and easier to read, eg. using something like EctoSparkles — ecto_sparkles v0.3.0 (shameless plug) for joins/preloads which is less verbose than the standard ecto syntax.

Pretty much either of those is ok, and as @egze provided.

The key is to be declarative, the what and not the how or exposing the implementation. So specifying the fields or aggregates or order you want is fine.

But also think beyond just Ecto concerns in your contexts. You may also notify on certain events using Phoenix pub sub topics, send emails or orchestrate additional activities via Oban, call web hooks for integrations and so on.

Some other things to think about…

Think beyond CRUD concepts. If you had say a ticketing system you would have semantic actions like open ticket, close ticket.

Think about access control enforcement in your contexts.

Aim for thin controllers and live views and fat contexts where the guts of your application and logic lives.

I typically create functions that are tailored to their business purpose—or, ahem context—as opposed to generic functions that take all sorts of parameters. In your example you have way too many permutations that I imagine you aren’t going to be using even half of (maybe only 2 or 3?). Write getters that describe the purpose you are getting them for (and I’m now seeing @adw632 has just said the same thing as I’m writing this).

I also sometimes forgo preloads and just run separate queries from separate context functions. Contrary to popular belief, running a single query isn’t guaranteed to be more efficient than running multiple. Not only is this true of SQL its especially true in Ecto since a single query will return a single, denormalized table which must be reduced in Elixir. This is slooow for larger datasets. In any event, this is totally situational but can work well in some cases. I would say Posts and Replies is one of these situations depending on how you have your data set up. Like, if creating a new reply is done through Posts.update_post(params) where there are cast_assocs all the way down, then obviously you need to preload. But if you have more specific, business-named functions like Posts.reply(user, post, params) or Replies.reply_to_post(user, post, params) then you aren’t gaining much from preloading (if you’re doing the multiple-query kind) other than being able to do @post.replies instead of just @replies.

I’m glad you pointed this out. The thing to avoid is 1+N queries. Using 2 or 3 queries to resolve each level is a lot less data repetition.

Query strategy is important. If you’re returning join IDs then you have already lost and just multiplied the left table data by the number of rows on the right just to get their ID. I also dislike assembling and passing lots of IDs into the second or third level query and prefer to scope the second or third level query on the prior queries constraint. I remember dealing with certain ORMs back in the day that would use the ID strategy, it’s ok with small queries but less than ideal with larger ones.