TokenOperator - Dependency-free helper most commonly used for making clean keyword APIs to Phoenix context functions

I just released the first version of TokenOperator - A dependency-free helper most commonly used for making clean keyword APIs to Phoenix context functions.

I would love feedback/questions on the pattern, the code, issues, better ways to accomplish, etc. This is my first Elixir package so there are probably issues. Thanks!

Here is a quick intro from the documentation:

One thing I’ve struggled with dealing with Phoenix contexts is knowing how to specify the queries to make from the controller. For example, say we want to see a list of blog posts. Sometimes we want that list paginated, sometimes only published, sometimes authors, sometimes with content, sometimes ordered by published date, etc.

We can always just create a bunch of functions on the context for every single variation. Here is an extremely contrived example for illustration:

Posts.list_published_posts_with_author_ordered_by_published_date_paginated(page: 7)

It would be nice to have a simple way to have an API with preset defaults similar to the following:

  filter: [:featured, :published],
  include: :author,
  paginate: true,
  page: 7,
  order_by: :publish_date

TokenOperator makes it easy to develop a keyword-based API such as this, using the keywords that make sense for your application. The most obvious use case relates to operating on an Ecto query, but it can operate on any token and has no dependencies.


Now that makes me wonder how developers usually address these cases without a library.

I had the same question and just didn’t find a lot out there. There was this blog post, but I feel like it couples Ecto to the controller a bit…

There is a short thread on the topic in this forum… Phoenix Contexts - is it common to preload all resources on the context methods with whatever everything needs access to?

1 Like

What this library does is near identical to what I manually do in most of my projects

It’s not hard at all to do manually, though this library does make the pattern more obvious.


Glad to know this is similar to a pattern you might already be using. To your point, the entire library/helper is only about 20 LOC and can pretty easily be done inline. The value to me is definitely more about documenting a consistent pattern, a clean external API, and not recreating the wheel (even if easily recreated) in every context.

I suspect a lot of experienced Phoenix devs already have some sort of helper they are using that serves the same type of purpose.


Documentation and Clarity is very important. ^.^

1 Like

Just curious about the naming. What is meant by token here?

By token, I just mean a data structure that can be passed around and operated upon. Two common examples of tokens are an Ecto query and Plug.Conn. This post explains it much better than I can…

TokenOperator is actually pretty abstracted in that it doesn’t care what type of token you are passing around or what naming you use for the options. I chose an obvious use-case to explain how it can be used. I don’t think I did a particularly good job explaining what it is though.

I started out calling it MaybeQuery, dependent upon Ecto, and with hard-coded opinionated option names like filter, order_by, and preload. That would have been easier to explain, but less flexible. Instead, this allows you to configure your own API conventions and naming and use it beyond that example use case if one presents itself.


I should note that even with the controller/context use case that token structure might be an Ecto.Multi as opposed to an Ecto query.

Version 0.2.0 has been released. This release provides the ability to use functions with an arity of 1. Previously, it was required to reference a function with an arity of 2. This function will receive the token as the first argument and the opts with the second argument. For many usage scenarios, these options are unused, so this requirement has been removed.

Previously, you might have configured a function to filter published articles with the following:

def list_posts(opts \\ []) do
  |> TokenOperator.maybe(opts, :filter, published: &published/2)
  |> Repo.all()

defp published(query, _) do
  from(p in query, where: p.is_published)

That can be simplified to:

def list_posts(opts \\ []) do
  |> TokenOperator.maybe(opts, :filter, published: &published/1)
  |> Repo.all()

defp published(query) do
  from(p in query, where: p.is_published)

It’s a small change but these functions (e.g. published) probably already existed in your context without a second argument. They should be able to stay that way rather than having arguments dictated by this package unless needed for the use case.