How to implement a get_or_insert function for a many to many association

Questions / Help
, ,
1

I’m building a File Manager with a tagging system. I have a FileManagement context in a Phoenix application with File and Tag in a many to many association through FileTag.

When creating a File I always want at least 1 Tag associated to it, so I’ve implemented a get_or_insert_tag() function inside the File schema which currently looks like this:

# file_management/file.ex
schema "files" do
    field :file_size, :integer
    field :file_type, :string
    field :name, :string
    field :url, :string

    many_to_many :tags, Tag, join_through: FileTag

    timestamps()
  end

  @doc false
  def changeset(%File{} = file, attrs) do
    file
    |> cast(attrs, [:name, :url, :file_type, :file_size])
    |> validate_required([:name, :url, :file_type, :file_size])
    |> put_assoc(:tags, get_or_insert_tag(attrs), required: true)
  end

  defp get_or_insert_tag(%{tags: [name]}) do
    %Tag{name: name}
    |> Repo.insert( 
      on_conflict: [set: [name: name]], 
      conflict_target: :name
    )
  end

I based it on this: Many to many and upserts « Plataformatec Blog article, but I’m having some issues adapting it to this use case. When using the create_file/1 function and passing it a file like the following:

%{
    name: "nEEDza1", 
    file_size: 743686, 
    file_type: "jpg", 
    url: "https://s3-us-west-2.amazonaws.com/bucket-name-here/nEEDza1.jpg",
    download_count: 0,
    tags: [
      "etc"
    ],
  }

I get an error like this:

** (MatchError) no match of right hand side value: {:error, ecto.Changeset<action: :insert, changes: %{file_size: 943916, file_type: “png”, name: “5Sw5ZtG.png”, url: “https://s3-us-west-2.amazonaws.com/bucket-name-here/5Sw5ZtG.png”}, errors: [tags: {“is invalid”, [type: {:array, :map}]}], data: #Filix.FileManagement.File<>, valid?: false>}

The Tag is successfully being upserted, but the File isn’t liking the Tag it should be associated to. I get the feeling this is something simple that I’m missing - I’m not very familiar with Ecto yet, so any help would be appreciated.

1 Like
2

It is because get_or_insert_tag should returns a list of tags, but does not, it returns only one element… It would be better to treat tags as a string with comma separated tags.

You could probably get it working by wrapping result in a list. But concept should be to be able to pass multiple tags at once for a post (file in your case).

  defp get_or_insert_tag(%{tags: [name]}) do
    [%Tag{name: name}
    |> Repo.insert( 
      on_conflict: [set: [name: name]], 
      conflict_target: :name
    )]
  end

By the way, the final upserts example is like this.

  @doc false
  def changeset(%Post{} = post, attrs) do
    post
    |> cast(attrs, @required_fields ++ @optional_fields)
    |> validate_required(@required_fields)
    |> validate_length(:title, min: 1, max: 255)
    |> foreign_key_constraint(:user_id)
    |> foreign_key_constraint(:group_id)
    |> put_assoc(:tags, parse_tags(attrs))
  end
  
  defp parse_tags(attrs) do
    (attrs["tags"] || attrs[:tags] || "")
    |> String.split(",")
    |> Enum.map(&String.trim/1)
    |> Enum.reject(& &1 == "")
    |> insert_and_get_all()
  end
  
  defp insert_and_get_all([]) do
    []
  end
  defp insert_and_get_all(names) do
    now = NaiveDateTime.utc_now
    maps = Enum.map(names, &%{name: &1, inserted_at: now, updated_at: now})
    Repo.insert_all Tag, maps, on_conflict: :nothing
    Repo.all from t in Tag, where: t.name in ^names
  end
2 Likes
3

If You want to keep your code, try this (not tested)

  @doc false
  def changeset(%File{} = file, attrs) do
    file
    |> cast(attrs, [:name, :url, :file_type, :file_size])
    |> validate_required([:name, :url, :file_type, :file_size])
    |> put_assoc(:tags, get_or_insert_tags(attrs), required: true)
  end

  defp get_or_insert_tags(%{tags: names}) do
    names
    |> Enum.map(fn name ->
      %Tag{name: name}
      |> Repo.insert( 
        on_conflict: [set: [name: name]], 
        conflict_target: :name
      )
    end)
  end
1 Like
4

Wrapping the result in a list is returning the same error. At least for now I don’t have a use case where I’ll be creating a file with multiple tags, at least in the same transaction, so the final example in the upserts example didn’t fit very well.

Repo.insert/2 returns an {:ok, Ecto.Schema.t()} would I need to parse out the schema first?

5

In that case wouldn’t it be easier to have just a tag attribute inside the file schema?

And Yes, I forgot to strip :ok from insert response :slight_smile:

      {:ok, tag} = %Tag{name: name}
      |> Repo.insert( 
        on_conflict: [set: [name: name]], 
        conflict_target: :name
      )
      tag
1 Like
6

A separate service will be making POST requests to insert and associate Tags to existing Files. The primary use case is to associate a file to multiple related records in an external system like a CRM.

7

This worked! Thanks.

A follow up question, if I were to implement the tagging in an attribute like you mentioned, would I be able to easily add/remove a Tag via. REST callouts from a separate system?

8

It is easy because when You pass tags to your post changeset, it will automatically create (and delete!) the tags… removing unwanted, creating newly etc.

BTW I just suggested to have an attribute, because You said there would be hardly but one tag per file… Your solution is working fine, but You should not forget tags is a collection :slight_smile:

1 Like
9

Thanks for the help, I’ll experiment with a Tag attribute this week because it seems like a simpler implementation, but I think I like the idea of having a separate Tag schema and api since the CRM service will often want to associate a file to other CRM records.

10

I would also keep Tag as a separate schema…

Good luck with your CRM and happy coding.

11

@MrDoops I would highly recommend you check out the (free) eBook by Platformatec “What’s New in Ecto 2.1”
http://pages.plataformatec.com.br/ebook-whats-new-in-ecto-2-0

Chapter 9 goes into great depth about many-to-many relations and upserts and the examples even focus on creating tags. It goes from a very naive approach to a very sophisticated one at the end.

2 Likes
12

The article linked in the first post == Chapter 9 of the book :slight_smile: