1player

1player

A design pattern for more flexible context APIs

A question I had when first learning contexts and Ecto was how to expand the default context API to support more flexible queries. Usually your auto-generated context modules look like:

defmodule App.Accounts do
  def list_users()
  def get_user(id)
  def get_user!(id)

  ...
end

Which soon becomes a little restrictive when you need to query your database with different filters. How do I find a user by email? How do I list only admin users? So one would create helper functions such as:

defmodule App.Accounts do
  def list_users()
  def list_admin_users()
  def get_user(id)
  def get_user!(id)
  def get_user_by_email(email)
  def get_user_by_email!(email)

  ...
end

As you can imagine, creating a function for each possible query type doesn’t scale very well, and if you introduce GraphQL to your application, where queries tend to be both user-generated and composable (find a user by email which is an admin), this way of organising your context API gets problematic.

Thankfully, the dataloader library, which is often used in conjuction with (Absinthe) GraphQL, shows an interesting pattern that can be used in all cases, not only in the GraphQL context.

dataloader mandates the creation of a query/2 function, that takes a queryable (i.e. an Ecto schema or a query) and a list of filters we want to apply to that queryable. Basically we want to take a list of filters, and build a query out of it.

def query(User, opts \\ [])
  # apply filters listed in opts and return an Ecto query
end

We can use this function to make all our context functions accept a list of filters than can be composed together:

defmodule App.Accounts do
  def query(User = queryable, opts \\ []) do
    Enum.reduce(queryable, opts, fn 
      {:email, email}, query -> 
        where(query, [u], u.email == ^email)

      {:admin?, admin?}, query -> 
        where(query, [u], u.admin? == ^admin?)

      {:in_group, group_name}, query -> 
        query
        |> join(:inner, [u], g in assoc(u, :group))
        |> where(g.name == ^group_name)
    end)
  end)

  def list_users(opts \\ []) do
    User
    |> query(opts)
    |> Repo.all()
  end

  def get_user(opts \\ []) do
    User
    |> query(opts)
    |> Repo.one()

  ...
end

All the possible filters are now encapsulated in this query function, they’re composable because Ecto is awesome, and our context API remains slim and neat. Want to find an admin user by email? Accounts.get_user(email: "foo@example.com", admin?: true). And bonus point, your context module is ready to be integrated with your GraphQL server.

An astute reader might notice that some of the filters tend to be very similar: we often want to filter a field by value (i.e. field == something), we often want to give our context API a way of ordering the results or limiting the number of rows returned. It would be nice to generalise some of these filters in a shared module. What I tend to do in my code is change the query function like this:

  def query(User = queryable, opts \\ []) do
    Enum.reduce(queryable, opts, fn 
      {:in_group, group_name}, query -> 
        query
        |> join(:inner, [u], g in assoc(u, :group))
        |> where(g.name == ^group_name)

      filter, query -> Helpers.default_query_filters(filter, query)
    end)
  end)

# in some other helper module...

def default_query_filters({:preload, preloads}, query) do
  preload(query, ^preloads)
end

def default_query_filters({:limit, n}, query) do
  limit(query, [q], ^n)
end

def default_query_filters({field, value}, query) when is_atom(field) do
  where(query, [q], field(q, ^field) == ^value)
end

The query function in the context now only contains filters and queries specific to the module, whereas the common functionality such as preloading or filtering a field by value are held by a helper module, since the logic is always the same no matter the queryable. And all this work lets us express complex queries very simply: Accounts.list_users(in_group: "Friends", admin?: true, preload: :group)

I’ve never seen this pattern in the wild, and I’ve been using it for a year at work with great success. I hope this helps somebody, and if someone has more free time than me, I imagine the default_query_filters function could be implemented in a library available for everybody to use.

Most Liked

baldwindavid

baldwindavid

This is similar to what my token_operator (GitHub - baldwindavid/token_operator: Helper to make clean keyword APIs to Phoenix context functions · GitHub) package does though a bit more opinionated and probably easier to understand.

It is for this reason that I’ve mostly moved away from using the pattern in the controller in favor of explicit function references.

# controller
Calendar.list_reservations([
  &Calendar.filter_reservations_by_room(&1, rooms),
  &Calendar.filter_reservations_by_day(&1, datetime),
  &Calendar.preload_reservation_owner_and_company/1,
  &Calendar.preload_reservation_room_and_location/1
])

In the context, I just run each of those functions in succession.

# context
def list_reservations(queries \\ []) do
  Reservation
  |> (fn query -> Enum.reduce(queries, query, & &1.(&2)) end).()
  |> Repo.all()
end

Okay, really I wrap that ugly reduce part in a utility, but you get the idea.

baldwindavid

baldwindavid

I haven’t actually had the need for that, but assume will at some point. If you’re talking about doing it with the pattern using explicit function references, perhaps something like:

# controller
Syndication.list_feeds([
  &Syndication.filter_and_sort_feeds_by(&1, params)
])
# context
def filter_and_sort_feeds_by(query, attrs \\ %{}) do
  attrs
  |> Map.take(["type", "occurred_at", "resource_type", "sort_by"])
  |> Enum.map(fn {key, value} ->
    case key do
      "type" -> &filter_feeds_by_type(&1, value)
      "occurred_at" -> &filter_feeds_by_occurred_at(&1, value)
      "resource_type" -> &filter_feeds_by_resource_type(&1, value)
      "sort_by" -> &order_feeds_by(&1, value)
    end
  end)
  |> Enum.reduce(query, & &1.(&2))
end

That provides the ability to use it just like the other filters. For something that custom though, maybe simpler to just have a dedicated function that actually hits the repo.

# controller
Syndication.filter_and_sort_feeds(params)
# context
def filter_and_sort_feeds(attrs \\ %{}) do
  attrs
  |> Map.take(["type", "occurred_at", "resource_type", "sort_by"])
  |> Enum.map(fn {key, value} ->
    case key do
      "type" -> &filter_feeds_by_type(&1, value)
      "occurred_at" -> &filter_feeds_by_occurred_at(&1, value)
      "resource_type" -> &filter_feeds_by_resource_type(&1, value)
      "sort_by" -> &order_feeds_by(&1, value)
    end
  end)
  |> Enum.reduce(Feed, & &1.(&2))
  |> Repo.all()
end
OvermindDL1

OvermindDL1

This is how I’ve been programming my big project to access an old DB with lots and lots of joins to do even trivial lookups. ^.^;

I haven’t really generalized any of the lookups as if it’s simple like that I’ll just do it on-site instead, the actual ‘query’ function holds anything that’s not generic.

Where Next?

Popular in Guides/Tuts Top

bluegene
Hi guys, I’ve been on a personal journey to learn Elixir for the past two years. During this journey I’ve been using the spaced repetiti...
New
anuragg
Hi everyone! I’m the founder of Render, a new cloud provider with native support for Elixir. When we launched Elixir support the most po...
New
zookzook
If you want to use the MongoDB in your next Killer-App-Project, but you did not dare ask because otherwise many would advise you to use P...
New
smpallen99
Did you know that IO.inspect/2 returns the the first argument and accepts a label option as a second argument. This makes it a perfect to...
New
yeshan333
vfox (version-fox) is a popular general version management tool write in Go, and the plug-in mechanism uses Lua to achieve extensibility....
New
dkuhlman
For those of you who might be interested in using ZeroMQ in Elixir, I converted the Erlang examples in the ZeroMQ “zguide” to Elixir. I ...
New
ob1
Recently we partitioned a table with 28 million rows by its ULID to improve query speeds and reduce query timeouts. To do this I ended up...
New
njwest
In the process of developing a Phx-based multiplayer experience, I found myself with so many browser tabs open with Elixir gaming resourc...
New
kevinlang
Hey all, With Phoenix 1.6 just around the corner, I figured I’d make a tutorial on how to add Bulma to a new Phoenix 1.6 project. By lev...
New
KoviRobi
Hi, I’ve written the following to debug function calls, not sure if it’s useful for anyone else, and if so should I put it somewhere? i...
New

Other popular topics Top

danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 29603 241
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
johnnyicon
Hi all, I’ve just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I’m trying to use Postgres...
New
nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
boundedvariable
I am going through the kafka architecture. All the features what the kafka is providing are already in Erlang. I would like hear your opi...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New

Latest on Elixir Forum

We're in Beta

About us Mission Statement