Phoenix LiveView: Should I use Ecto.Changeset for many-to-many table or just enter IDs myself?

Needing advice since I’m pulling my hair out trying to set up what should be a very simple many-to-many table in a Phoenix LiveView App. Everything was going very smoothly until I got to table relationships.

WHAT WAS EASY: Setting up two related tables and their schemas. That all worked fine. Created the join table which is basically a table with the unique identifiers from both tables.

But I’ve spent hours trying to understand how to persist data into these databases, looking at documentation that seems to over-complicate something that is really simple:

Using the example from Elixir School, I think this is what I would need to add to my code:

movie = Repo.preload(movie, [:distributor, :characters, :actors])
movie_changeset = Ecto.Changeset.change(movie)
movie_actors_changeset = movie_changeset |> Ecto.Changeset.put_assoc(:actors, [actor])

I’m still not sure where I put that code, but I’m pretty sure I have to write another changeset method in my schema.

I ended up down a rabbit hole of whether to use put_assoc or cast_assoc. And then there’s all of the :on_replace options to choose from.

Is there a reason why I shouldn’t just grab the IDs and put them in the table myself? Is there something magical happening when I use build_assoc or changeset functions? It seems like it would be easier to simply populate the tables with the IDs myself.

I’ve read pages of documentation on Ecto relationships and I just don’t understand the reason for it being this complicated. We’re just joining tables using a simple unique identifier. I would really like to know if others have been down the path and just opted to do the relationships themself. Is there any amazing functionality I’m going to miss?
Thanks in advance!

3 Likes

That’s what I am doing, after wasting many hours in the docs.

In my opinion the Ecto docs are wrote for Elixir experienced users, that are familiar with how Ecto works, because a lot of implicit knowledge is assumed(by accident), therefore beginners will get lost very easily.

Bear in mind that I am not saying that this was done with intention, rather I believe that it’s hard to not fall in this trap when writing docs for advanced features…

I cannot also put a Merge request in place to improve them, because I don’t get what is wrong with them, and had not open an issue because I though that was a problem with me, but it seems that I am not alone.

@Exadra37 Sorry to hear that you went down the Ecto-docs rabbit hole too. I found it difficult to understand the subtle differences between the choices. The docs would say, “be sure to understand X before using Y” and I’d get lost in another set of docs.

When you switched to just managing the ID relationships yourself, did you keep the migration and schema formats as defined in the Ecto docs?

For example … in the Movie example, a Movie can have many characters but the characters are associated with 1 movie so the Character schema would look like this:

MIGRATION
def change do
create table (:characters) do
add :name, :string
add :movie_id, references (:movies)
end

SCHEMA
schema “characters” do
field :name, string
belongs_to :movie, Movie
end

How you describe it it looks like an has_many relation instead of many_to_many.


Yes, I kept everything in migrations as per the docs.


My use case:

So my case is that a video can belong to many users and a user can have many videos. I just never got to understand how to update the relation in one go in the Ecto way.

Let’s say that when a user adds a video I need to first insert the video in the database and then I associate it manually with that user, by inserting the record directly in the join through table.

I would expect to make it one go with Ecto built-ins, and I am pretty sure that it must be possible, but for me the issue is that the examples in the docs are just partial code that you cannot run, therefore I was never able to grasp them.

1 Like

Thank you @Exadra37 ! You confirmed my exact observations about trying to use the Ecto built-ins. And your example using the video is what I was thinking as well: create the entry, grab the ID and set the association(s) myself. I feel like I’ll also have more control with changes and deletes this way as well. Thanks again.

Whenever I can I prefer to use build_assoc instead of assigning id manually.

Have you read The Little Ecto Cookbook? It really helped me to understand the design decisions behind Ecto. It also describes a many_to_many—it’s related to polymorphism but should be the same.

Sorry I can’t give you concrete help because I’m still newish to Phoenix and have never written a many_to_many. I will say that I always try and avoid many_to_many if I can. If I can’t get away with using a json column (for something like tags, for example) then I prefer to use has_many with :through, much like in Rails. I find these much easier to reason about (rubocop-rails’ default actually bans has_and_belongs_to_many). To each their own, of course, just my 2 cents on that.

Otherwise, this is a very useful blog post on the differences between put_assoc and cast_assoc: Understanding Associations in Elixir's Ecto | AppSignal Blog

As for on_replace, I would default to :delete, that way you won’t end up with orphaned rows, otherwise go with :raise. It’s really nice that Ecto makes up think about this as it took several times of being in this situation in the past, having to spend hours doing stressful db surgery, before I started to think hard about it with Rails.

As to where to put the code, I don’t want to ramble on too much here without concrete example. I would definitely advise reading The Little Ecto Cookbook. I can absolute empathize that this stuff is frustrating at first, you just have to really work through it. If you are used to the world of Rails or Django, then Phoenix can be especially frustrating as it requires being much more explicit. What it’s actually doing is refusing to provide you with many well-known foot-guns that other frameworks just hand to you in the name of going fast up front (I actually feel that way about functional programming in general, now). It can be really frustrating at first, but but the payoff is worth it.

The one other thing I will say: just do whatever works for now because refactoring Elixir is surprisingly smooth. I did work-arounds like you described when I was learning but as I learned more, it was easy to refactor into something more idomatic.

And I would stick to guides and books over the docs to begin with.

1 Like

Sorry, I just realized that the book I was actually thinking of is Programming Ecto, which is not free, but it’ll teach you all you need to know. The Little Ecto Cookbook is also very great, but it is all about specific patterns. I read that first and it got me really excited about learning Ecto and motivated me to work through the initial hump.

1 Like

As it says in the docs you linked, the build_assoc is for has_many, not many_to_many:

Examples

If the relationship is a has_one or has_many and the primary key is set in the parent struct, the key will automatically be set in the built association:

And yes, build_assoc works like a charm in has_many, that was my previous use case where a user could have many videos, but the video could had only one user.

@Exadra37 I just got to the point where my form has generated the IDs for both items that go into the join-table, but I can’t figure out how to insert into that table.

In a many-to-many case where the table is movies_actors (where movies have many actors and actors are in many movies), how to do you actually enter the movie_id and actor_id into the movies_actors table? How do I create a record in movies_actors and set the record with these two ids?

As for the 1-to-many relationship, I saw an example where he just casted the id directly into the “many” table. So in the case of one user having many videos, then the video table will have a user_id since it can only have one user. I would just cast the id directly to the video. Doesn’t build_assoc basically just do that? In the end, we’re just assigning one user_ID to that movie.

Where I’m really struggling is when there is an ID pair in a totally separate table. I just can’t figure out how to get a handle onto that join-table and then cast and ID pair into a row. I’m wondering if I need to fully implement a schema and changeset for the join-table. Is that how you did it?

Thank you! Just downloaded that book. Will read more documentation. Sigh.

My code:

defmodule Rumbl.Repo.Migrations.CreateUser do

  use Ecto.Migration

  def change do
    create table(:users) do
      add :name, :string
      add :username, :string, null: false
      add :password_hash, :string

      timestamps()
    end

    create unique_index(:users, [:username])
  end
end

defmodule Rumbl.Repo.Migrations.CreateVideo do
  use Ecto.Migration

  def change do
    create table(:videos) do
      add :url, :string
      add :title, :string
      add :slug, :string
      add :description, :text
      add :user_id, references(:users, on_delete: :nothing)

      timestamps()
    end
    create index(:videos, [:user_id])

  end
end

defmodule Rumbl.Repo.Migrations.CreateUserVideos do
  use Ecto.Migration

  def change do
    create table(:user_videos) do
      add :user_id, references(:users, on_delete: :nothing)
      add :video_id, references(:videos, on_delete: :nothing)
      timestamps()
    end

    create index(:user_videos, [:video_id])
    create index(:user_videos, [:user_id])
    create unique_index(:user_videos, [:user_id, :video_id])
  end
end

### SCHEMAS ###

defmodule Rumbl.User do

  use Rumbl.Web, :model

  schema "users" do
    field :name, :string
    field :username, :string
    field :password, :string, virtual: true
    field :password_hash, :string
    # has_many :videos, Rumbl.Video
    many_to_many :videos, Rumbl.Video, join_through: Rumbl.UserVideos, on_replace: :delete
    has_many :annotations, Rumbl.Annotation

    timestamps()
  end

  def changeset(user, params \\ %{}) do
    user
    |> cast(params, [:name, :username], [])
    |> validate_length(:username, min: 1, max: 20)
    |> unique_constraint(:username)
  end

  def registration_changeset(user, params) do
    user
    |> changeset(params)
    |> cast(params, [:password], [])
    |> validate_required([:username, :password])
    |> validate_length(:password, min: 12, max: 256)
    |> _put_pass_hash()
  end

  defp _put_pass_hash(changeset) do
    case changeset do
        %Ecto.Changeset{valid?: true, changes: %{password: pass}} ->
          change(changeset, Argon2.add_hash(pass))
        _ ->
          changeset
      end
  end

end

defmodule Rumbl.Video do

  use Rumbl.Web, :model

  # The key to be used to extract the id we want to be used by the url helpers
  @derive {Phoenix.Param, key: :slug}

  schema "videos" do
    field :url, :string
    field :title, :string
    field :description, :string
    field :slug, :string
    field :soft_deleted, :boolean, default: false
    belongs_to :user, Rumbl.User
    many_to_many :users, Rumbl.User, join_through: Rumbl.UserVideos
    belongs_to :category, Rumbl.Category
    has_many :annotations, Rumbl.Annotation

    timestamps()
  end

  @required_fields [:url, :title, :description]
  @optional_fields [:category_id, :soft_deleted]

  @doc """
  Builds a changeset based on the `struct` and `params`.
  """
  def changeset(struct, params \\ %{}) do

    struct
    |> cast(params, @required_fields ++ @optional_fields)
    |> validate_required(@required_fields)
    # @IMPORTANT must come before `update_change/3`, otherwise the slug will not
    #            be fixed when we run Rumbl.ReleaseTasks.fix_video_records/0
    |> slugify_title()
    |> update_change(:url, &UtilsFor.Text.Trim.to_single_whitespace/1)
    |> update_change(:title, &UtilsFor.Text.Trim.to_single_whitespace/1)
    |> update_change(:description, &String.trim/1)
    # @TODO change constraint to the youtube video id
    |> unique_constraint(:slug)
    |> assoc_constraint(:category)
  end

  defp slugify_title(changeset) do
    if title = get_change(changeset, :title) do
      put_change(changeset, :slug, UtilsFor.Text.Slug.slugify(title, "-"))
    else
      changeset
    end
  end

end


### ADD VIDEO ###

defmodule Rumbl.Videos.Add.Video do

  def add_for(%Rumbl.User{} = user, %{} = params) do

    changeset = %Rumbl.Video{} |> Rumbl.Video.changeset(params)

    case Rumbl.Repo.insert(changeset) do
      {:ok, %Rumbl.Video{} = video} ->
        video
        # @TODO Use instead Ecto Associations?
        |> Rumbl.Videos.AssociateWithUser.Video.associate(user)

        {:ok, video}

      {:error, changeset} ->
        {:error, changeset, video}
    end
  end

end


### ASSOCIATE VIDEO ###

defmodule Rumbl.Videos.AssociateWithUser.Video do

  def associate(%Rumbl.Video{} = video, %Rumbl.User{} = user) do
    Rumbl.UserVideos.new_changeset(%{user_id: user.id, video_id: video.id})
    |> Rumbl.Repo.insert()
  end

end

Then I use it from the controller:

def create(conn, %{"video" => video_params}, user) do
    case Rumbl.Videos.Add.Video.add_for(user, video_params) do
      {:ok, %Rumbl.Video{} = video} ->
        Logger.info("Created new video with slug:#{video.slug} for user @#{user.username}:#{user.id}")

        conn
        |> redirect(to: watch_path(conn, :show, video))
      {:error, changeset, video} ->
        render(conn, "new.html", changeset: changeset)
    end
end

So, as you can see by the code the association for the video to the user is done by invoking Rumbl.Videos.AssociateWithUser.Video.associate(video, user) from Rumbl.Videos.Add.Video.add_for(user, params).

Hope that the code works better then 1000 words :wink:

3 Likes

@Exadra37 THANK YOU soooooo much! I’ve just spent hours battling changesets trying one last time to get them to work. No success. I’m done trying to use them! I’m going to follow your example. Thanks again for your help. Switching gears right now.

1 Like