I use something like that. I forget where I got most of the code from,
or at least the scaffolding of the idea.
Here’s a rough dump of it that I might as well share. I can not promise
to the quality of it, use at your own risk. It’s not perfect, it’s probably not
even good really, it’s a crutch. It’s kind of like a poormans GraphQL mixed
with a less expressive Ecto.Query. The worst of all worlds.
Usage looks roughly like:
Posts.list_posts(
by_category: "unhinged-rants",
posted_between: {last_year, now},
assocs: [:author, :comments]
)
It lets you define broad ranges of queries in a pretty flexible manner.
It doesn’t cover everything, you sometimes need to write
custom one-shots. You can also use the Post.Queries module directly to
pipe that into Repo.exists() etc.
What I do like about it is, since the API is basically defined as functions,
you get pretty good code suggestions, as well as “no matching function” errors,
including the list of possible choices, when you muck up. You can also make
your API as confusing as you like, with assocs: [:authors, {:comments, magic_number_8}, "desc"] etc, since you’re just pattern matching on something.
(More practically, I have a few fields that can be true | false | [true, false] or date | {date_start, date_end} and the interface feels pretty
fluid).
Managing nested associations can be awkward, it’s best to set a hard limit.
Maybe getting comment authors via a post is ok, but getting comment author
likes is a bit “too” nested. As I said, it is also kind of a poormans
everything and you will definitely hit queries that are painful to run, or
unexpectedly limit join results because of a where clause.
I think running something like GraphQL as your internal API might be the best
bet in the end. I find controllers/views often have pretty
fluid-but-also-duplicated query requirements and you just end up needing
something that supports that. I am not sure if Absinthe really supports being
used as an internal API though. The docs mention one way with
absinthe_phoenix, but it’s pretty slim and absinthe_phoenix's docs seem to
be more about exposing a GraphQL API via Phoenix? Truthfully I just haven’t had
the time to run through it though.
Anyway, here’s the code, may contain gore, viewer discretion is advised.
The macro, which you use to define your “queryable” terms.
defmodule YourApp.Queryable do
@moduledoc """
Helper macros for defining a "Queries" interface.
Call Queryable.setup() then
```
Queryable.has_filter(:filter_name, fn query, filter_value ->
query
|> where ...
end)
```
```
Queryable.has_assoc(:parts, fn query ->
query
|> preload(:parts)
end)
```
which produces
```
Queries.filter_name(query, val)
Queries.with_parts(query)
```
and
```
list_x(filter_name: 1, assocs: [:parts])
```
"""
@doc """
Required to setup option chaining functions.
Accepts an optional "preflight" function for altering options, defaults to
```
fn opts -> opts end
```
"""
defmacro setup() do
f =
quote do
fn opts -> opts end
end
quote_setup(f)
end
defmacro setup(func) do
quote_setup(func)
end
defp quote_setup(preflight_opts) do
quote do
def apply_opts(query, opts) do
opts = unquote(preflight_opts).(opts)
# force assocs to be done first so we can rely on "has_named_binding"
# in queries if we want to do checks only if the data was requested.
opts =
case Keyword.has_key?(opts, :assocs) do
true ->
{assocs, opts} = Keyword.pop(opts, :assocs)
[{:assocs, assocs} | opts]
false ->
opts
end
do_filter(query, opts)
end
defp do_filter(query, []) do
query
end
defp do_filter(query, [{:assocs, assocs} | rest]) do
query
|> do_assoc(assocs)
|> do_filter(rest)
end
defp do_assoc(query, []) do
query
end
end
end
# func = {:fn [line: 4] [...]}
defmacro has_filter(name, func) when is_atom(name) and is_tuple(func) do
# has_filter(:by_id, fn q, id -> where(q, id: ^id))
quote do
# def by_id(query, val)
def unquote(name)(query, val) do
unquote(func).(query, val)
end
# def do_filter(query, [{:by_id, val} | rest])
defp do_filter(query, [{unquote(name), val} | rest]) do
query
|> unquote(name)(val)
|> do_filter(rest)
end
end
# |> tap(fn m ->
# m
# |> Macro.to_string()
# |> IO.puts()
# end)
end
# {:fn [line: 4] [...]}
defmacro has_assoc(name, func) when is_atom(name) and is_tuple(func) do
# has_assoc(:seller, fn q, id -> ... from(..., preload: ...)
quote do
# the actual joiner
defp with_join(query, unquote(name)) do
unquote(func).(query)
end
# def with_seller(query)
def unquote(:"with_#{name}")(query) do
with_join(query, unquote(name))
end
# def do_assoc(query, [:seller | other])
defp do_assoc(query, [unquote(name) | rest]) do
query
|> with_join(unquote(name))
|> do_assoc(rest)
end
end
# |> tap(fn m ->
# m
# |> Macro.to_string()
# |> IO.puts()
# end)
end
end
Then define the “Queries” for a model:
defmodule YourApp.Posts.Queries do
import Ecto.Query, warn: false
import YourApp.Ecto.Query
alias YourApp.Posts.Post
require YourApp.Queryable, as: Q
@type query_options :: [
by_ids: [integer(), ...] | nil,
by_slug: String.t() | nil,
assocs: [:comments, :authors, ...] | nil,
by_search: String.t() | nil
]
def posts() do
base()
end
Q.setup()
Q.has_filter(:by_id, fn query, id when is_integer(id) or is_binary(id) ->
query
|> where([post: post], post.id == ^id)
end)
Q.has_filter(:by_slug, fn query, slug when is_binary(slug) ->
query
|> where([post: post], post.slug == ^slug)
end)
Q.has_filter(:by_search, fn
query, search ->
search =
search
|> to_tsquery_format()
query
|> where(
[post: p],
tsvector_search(
[
tsvector(p.title, weight: "A"),
tsvector(p.body, weight: "B")
],
tsquery(^search)
)
)
|> Ecto.Query.order_by(
[post: p],
{:desc,
tsvector_rank(
[
tsvector(p.title, weight: "A"),
tsvector(g.body, weight: "B")
],
tsquery(^search)
)}
)
end)
Q.has_assoc(:comments, fn query ->
if has_named_binding?(query, :comments) do
query
else
from([post: post] in query,
left_join: comments in assoc(group, :comments),
as: :comments,
preload: [comments: comments]
)
end
end)
Q.has_assoc(:author, fn query ->
if has_named_binding?(query, :author) do
query
else
from([post: post] in query,
left_join: author in assoc(post, :author),
as: :author,
preload: [author: author]
)
end
end)
defp base() do
from(_ in Post, as: :post)
end
end
And define an API inlet:
defmodule YourApp.Posts do
import Ecto.Query, warn: false
alias YourApp.Repo
alias YourApp.Posts.Post
alias YourApp.Posts.Queries
@spec list_posts(Queries.query_options()) :: [Post.t(), ...]
def list_posts(opts \\ []) do
Queries.posts()
|> Queries.apply_opts(opts)
|> Repo.all()
end
@spec get_post(integer() | String.t(), Queries.query_options()) ::
{:ok, Post.t()} | {:error, atom()}
def get_post(id, opts \\ []) when is_integer(id) or is_binary(id) do
Queries.posts()
|> Queries.by_id(id)
|> Queries.apply_opts(opts)
|> Repo.fetch_one()
end
end
I have also tried the forms:
Posts.posts()
|> Posts.by_category("x")
|> Posts.with_comments()
|> Posts.list()
and
Posts.list_posts(fn query ->
query
|> Post.by_category("x")
|> Post.with_comments()
end)
but have found the macro to be nicest to use in the end. I actually don’t
really love any of them though.
So to answer the original question, I write my preloaders in their own Queries module (generally) per-model and then write my “queries” directly in the view/controller, but they are abstracted behind the data context.
I kind of wrote that macro out one day, then just got on with using it. I have been meaning to return and polish it up, as I think it could probably be reasonably transportable between projects. It can run into namespace issues with stuff like sorting though, the developer needs to make decisions and be consistent with what “names” they give to query fields.
That’s cool, thanks for the answer! I’m gonna try it out.
I wonder if that’s always the case though? Isn’t ecto smart enough to combine 
