Hello!
I wanted to share with you my work concerning a counter cache on a many_to_many association (User ↔ Team) and some questions that came to my mind while coding it.
Here is the code:
Schemas
defmodule MyApp.Accounts.Team do
use Ecto.Schema
schema "teams" do
field :name, :string
field :users_count, :integer, default: 0
timestamps()
many_to_many :users, User, join_through: TeamUser
end
end
defmodule MyApp.Accounts.User do
use Ecto.Schema
schema "users" do
field :email, :string
field :name, :string
timestamps()
many_to_many :teams, Team, join_through: TeamUser
end
end
The join schema:
defmodule MyApp.Accounts.TeamUser do
use Ecto.Schema
@primary_key false
schema "teams_users" do
belongs_to :team, Team
belongs_to :user, User
end
def changeset(%TeamUser{} = team_user, attrs) do
team_user
|> cast(attrs, [:team_id, :user_id])
|> validate_required([:team_id, :user_id])
|> unique_constraint(:team_id, name: "teams_users_team_id_user_id_index")
end
end
Some quick Repo helpers
defmodule Palapa.Repo do
use Ecto.Repo, otp_app: :my_app
require Ecto.Query
# Not sure about that
def increment(struct, field, value \\ 1) do
primary_key = Ecto.primary_key(struct)
struct.__struct__
|> Ecto.Query.where(^primary_key)
|> update_all([inc: [{field, value}]], returning: true)
|> case do
{1, updated_struct} -> {:ok, updated_struct}
_ -> {:error, struct}
end
end
def decrement(struct, field, value \\ -1) do
increment(struct, field, value)
end
end
Adding a User to a Team
Here I use Ecto.Changeset.prepare_changes/2 when incrementing the counter, as advised in the Ecto documentation
def add_user_to_team(%User{} = user, %Team{} = team) do
TeamUser.changeset(%TeamUser{}, %{user_id: user.id, team_id: team.id})
|> increment_counter_cache(team, :users_count)
|> Repo.insert
end
defp increment_counter_cache(changeset, struct, counter_name, value \\ 1) do
prepare_changes(changeset, fn prepared_changeset ->
prepared_changeset.repo.increment(struct, counter_name, value)
prepared_changeset
end)
end
Removing a User from a Team
Here’s the kicker. To add/remove a user from a team, I work directly on the join schema. From what I understood, I can’t use a changeset here as the join table has no id primary key to work with. Therefore, I have to use Repo.delete_all/2 or Ecto.Multi.delete_all/4 to delete the association directly in the join table. And since I can’t use a changeset, I can’t use Ecto.Changeset.prepare_changes/2 again to decrement the counter.
So here is my version using Ecto.Multi this time:
def remove_user_from_team(%User{} = user, %Team{} = team) do
team_user_query =
from tu in TeamUser,
where: tu.user_id == ^user.id and tu.team_id == ^team.id
Ecto.Multi.new
|> Ecto.Multi.delete_all(:team_user, team_user_query)
|> Ecto.Multi.run(:counter_cache_decrement, fn changes_so_far ->
# Avoids having a negative counter by checking if the row in the join table has actually been deleted
%{team_user: {deleted_entries_count, nil}} = changes_so_far
if deleted_entries_count > 0 do
Repo.decrement(team, :users_count)
else
{:ok, team}
end
end)
|> Repo.transaction
end
OK. There are probably some things to improve here, but that’s the global idea.
Newbie thoughts
I’m relatively new to Elixir. I know Ecto favors explicitness and that it shouldn’t be seen as an ORM. Coming from a Rails/ActiveRecord background, I’m probably biased. But still, having this counter cache working was not as straightforward/elegant as I thought.
Q1) It seems that the “official” way to implement a counter cache with Ecto is to do it manually through Ecto.Changeset.prepare_changes/2. In the Rails world, I think I only had to do something like has_many :users, through: :team_memberships, counter_cache: true and that’s it (yes, magic). Is a counter cache feature like this would be considered too opinionated or abstract to be part of Ecto?
Q2) I’ve read several blog posts and Ecto documentation where Ecto.Changeset.put_assoc/4 and the on_replace option are used to associate two existing records in a many_to_many association. But it doesn’t feel very appropriate for this use case as you have to preload the whole association first (which may be very big!) and then re-submit all the desired associated records. Therefore I decided to work directly on the join table to add/remove a user to a team. As @josevalim said in this Github issue :
Going through the join table is the way to go! put_assoc is meant to compute the diffs between values and keep transactional guarantees.
Maybe I’ve misread the docs but it feels like some functions are missing to Ecto for doing just a simple association on the join table, without preloading, and without having to maintain a join schema and having to deal directly with the user_id and team_id fields.
Q3) Why common Repo helpers like Repo.increment(struct, field, value), Repo.exists? aren’t available directly in Ecto?
Q4) What’s the real difference between a has_many through: association and a many_to_many association with a join schema (join_through option)?
Many thanks for your time and suggestions!
