The new(ish) project that I’m working on started with Ecto 2 so I had access to many_to_many, however I had issues actually using it so I ended up falling back to has_many for two reasons before, the first being that I needed the extra data that was stored on the join table, and second that I could not get it to work.

Now has_many is a read-only mapping, no being able to store back in to it, but that feels like it wastes code and effort that could be automatically managed, at least in more simple join tables without extra data, and many_to_many is touted at having writing capability. So now I have a model that does have a many_to_many mapping and also has no extra useful data on the join table (beyond timestamps for logging purposes), so with that, here is the main model:

defmodule MyServer.Tag do
  use MyServer.Web, :model

  schema "tags" do
    field :name, :string
    field :description, :string

    many_to_many :parents, MyServer.Tag,
      join_through: MyServer.Tag.Tag,
      join_keys: [child_id: :id, parent_id: :id],
      on_replace: :delete,
      on_delete: :delete_all

    timestamps()
  end
end

And the join table:

defmodule MyServer.Tag.Tag do
  use MyServer.Web, :model

  @primary_key false
  schema "tags_tags" do
    belongs_to :child, MyServer.Tag, primary_key: true
    belongs_to :parent, MyServer.Tag, primary_key: true

    timestamps()
  end
end

I was going through this a lot yesterday, refactoring, changing styles, and I am certain the above definitions are broken somehow. The concept is that there are Tags, Tags can also be tagged. For example any tag that is tagged with an ‘Issue’ tag can be used in the issue tracker on issues. Unsure if the style is useful over a simple string mapping but I’ve not done this style before and it seems it could be useful so…

Now, say a tag is being created, we get something like:

iex> tag_params = %{"description" => "Testing", "name" => "Test", "parents" => ["1"]}
%{"description" => "Testing", "name" => "Test", "parents" => ["1", "4"]}

Say from a web form, it should create a new tag that is also tagged with the tags with the IDs of 1 and 4 (the creation form gets a list of IDs that the current user are allowed to attach to and assume that I’ve already checked that 1 and 4 are allowed here too, I have a very fine-grained permission system). What would be the way to attach it? My immediate thought would be:

changeset = Tag.changeset_assoc(%Tag{}, tag_params)
Repo.insert(changeset)

Where changeset_assoc/2 and changeset/2 are:

  def changeset(struct, params \\ %{}) do
    struct
    |> cast(params, [:name, :description])
    |> validate_required([:name, :description])
    |> unique_constraint(:name)
    |> unique_constraint(:id, name: "tags_pkey")
  end


  def changeset_assoc(struct, params \\ %{}) do
    struct
    |> changeset(params)
    |> cast_assoc(:parents) #, with: &changeset_as_assoc(&1, &2))
  end

My immediate thought as to why that would work is because the primary_key on Tag is id of an integer, and I would think that it would be able to look up a prior-existing Tag with that ID (or error if it did not exist). However, instead I get this error:

** (exit) an exception was raised:
    ** (ArgumentError) argument error
        :erlang.apply("1", :id, [])
        (my_server) anonymous fn/1 in MyServer.ViewExtras.tag_input/4
        (elixir) lib/enum.ex:1184: Enum."-map/2-lists^map/1-0-"/2
        (my_server) web/view_extras.ex:34: MyServer.ViewExtras.tag_input/4
        (my_server) web/templates/tag/form.html.eex:22: anonymous fn/2 in MyServer.TagView.form.html/1
        (phoenix_html) lib/phoenix_html/form.ex:235: Phoenix.HTML.Form.form_for/4
        (my_server) web/templates/tag/form.html.eex:1: MyServer.TagView."form.html"/1
        (my_server) web/templates/tag/new.html.eex:3: MyServer.TagView."new.html"/1
        (my_server) web/templates/layout/app.html.eex:79: MyServer.LayoutView."app.html"/1
        (phoenix) lib/phoenix/view.ex:335: Phoenix.View.render_to_iodata/3
        (phoenix) lib/phoenix/controller.ex:641: Phoenix.Controller.do_render/4
        (my_server) web/controllers/tag_controller.ex:1: MyServer.TagController.action/2
        (my_server) web/controllers/tag_controller.ex:1: MyServer.TagController.phoenix_controller_pipeline/2
        (my_server) lib/my_server/endpoint.ex:1: MyServer.Endpoint.instrument/4
        (my_server) lib/phoenix/router.ex:261: MyServer.Router.dispatch/2
        (my_server) web/router.ex:1: MyServer.Router.do_call/2
        (my_server) lib/my_server/endpoint.ex:1: MyServer.Endpoint.phoenix_pipeline/1
        (my_server) lib/plug/debugger.ex:93: MyServer.Endpoint."call (overridable 3)"/2
        (my_server) lib/my_server/endpoint.ex:1: MyServer.Endpoint.call/2
        (plug) lib/plug/adapters/cowboy/handler.ex:15: Plug.Adapters.Cowboy.Handler.upgrade/4

Uh, hrmm, well it tried to call “1” as a function, in that case I bet that it is actually assuming that it is a map, well lets try this as the changeset function instead:

parents_params = Map.get(tag_params, "parents", [])
parents_ids = Enum.map(parents_params, fn t ->
  String.to_integer(t)
end)
parents = parents_ids |> Enum.map(fn t -> %{id: t} end)
changeset = Tag.changeset_assoc(%Tag{parents: []}, put_in(tag_params["parents"], parents))

And try it… and changeset has an error:

iex> changeset
#Ecto.Changeset<action: nil,
  changes: %{description: "Testing", name: "Test",
    parents: [#Ecto.Changeset<action: :insert, changes: %{},
      errors: [name: {"can't be blank", []},
       description: {"can't be blank", []}], data: #my_MyServer.Tag<>,
      valid?: false>,
     #Ecto.Changeset<action: :insert, changes: %{},
      errors: [name: {"can't be blank", []},
       description: {"can't be blank", []}], data: #my_MyServer.Tag<>,
      valid?: false>]}, errors: [], data: #my_MyServer.Tag<>, valid?: false>}

Hmm, it seems that it is trying to insert new tags as parents instead of re-using existing ones, so I guess it does not try to lookup existing ones at all… So instead what if I lookup the Tags and put ‘those’ into the table, changing the entire changeset lookup into:

parents_params = Map.get(tag_params, "parents", [])
parents_ids = Enum.map(parents_params, fn t ->
  String.to_integer(t)
end)
parents = Repo.all(from t in Tag, where: t.id in ^parents_ids)
changeset = Tag.changeset_assoc(%Tag{parents: []}, put_in(tag_params["parents"], parents))

And give this a try:

** (Ecto.CastError) expected params to be a map, got: `%my_MyServer.Tag{__meta__: #Ecto.Schema.Metadata<:loaded, "tags">, description: "Parent tag for Issues", id: 1, inserted_at: #Ecto.DateTime<2016-07-05 21:27:32>, name: "Issues", parents: #Ecto.Association.NotLoaded<association :parents is not loaded>, updated_at: #Ecto.DateTime<2016-07-05 21:27:32>}`
    (ecto) lib/ecto/changeset.ex:326: Ecto.Changeset.do_cast/4
    (my_server) web/models/tag.ex:22: my_MyServer.Tag.changeset/2
    (ecto) lib/ecto/changeset/relation.ex:98: Ecto.Changeset.Relation.do_cast/5
    (ecto) lib/ecto/changeset/relation.ex:284: Ecto.Changeset.Relation.map_changes/7
    (ecto) lib/ecto/changeset.ex:533: Ecto.Changeset.cast_relation/4
    (my_server) web/controllers/tag_controller.ex:28: my_MyServer.TagController.create/2
    (my_server) web/controllers/tag_controller.ex:1: my_MyServer.TagController.action/2
    (my_server) web/controllers/tag_controller.ex:1: my_MyServer.TagController.phoenix_controller_pipeline/2
    (my_server) lib/my_server/endpoint.ex:1: my_MyServer.Endpoint.instrument/4
    (my_server) lib/phoenix/router.ex:261: my_MyServer.Router.dispatch/2
    (my_server) web/router.ex:1: my_MyServer.Router.do_call/2
    (my_server) lib/my_server/endpoint.ex:1: my_MyServer.Endpoint.phoenix_pipeline/1
    (my_server) lib/plug/debugger.ex:93: my_MyServer.Endpoint."call (overridable 3)"/2
    (my_server) lib/my_server/endpoint.ex:1: my_MyServer.Endpoint.call/2
    (plug) lib/plug/adapters/cowboy/handler.ex:15: Plug.Adapters.Cowboy.Handler.upgrade/4
    (cowboy) src/cowboy_protocol.erl:442: :cowboy_protocol.execute/4

So it really badly wants a map, not a struct, lets try casting the Tag into a map then:

parents_params = Map.get(tag_params, "parents", [])
parents_ids = Enum.map(parents_params, fn t ->
  String.to_integer(t)
end)
parents = Repo.all(from t in Tag, where: t.id in ^parents_ids)
  |> Enum.map(fn t -> Map.from_struct(t) end)
changeset = Tag.changeset_assoc(%Tag{parents: []}, put_in(tag_params["parents"], parents))

And try this:

** (exit) an exception was raised:
    ** (Ecto.ConstraintError) constraint error when attempting to insert struct:

    * unique: tags_lower_name_index

If you would like to convert this constraint into an error, please
call unique_constraint/3 in your changeset and define the proper
constraint name. The changeset defined the following constraints:

    * unique: tags_pkey
    * unique: tags_name_index

        (ecto) lib/ecto/repo/schema.ex:403: anonymous fn/4 in Ecto.Repo.Schema.constraints_to_errors/3
        (elixir) lib/enum.ex:1184: Enum."-map/2-lists^map/1-0-"/2
        (ecto) lib/ecto/repo/schema.ex:393: Ecto.Repo.Schema.constraints_to_errors/3
        (ecto) lib/ecto/repo/schema.ex:193: anonymous fn/11 in Ecto.Repo.Schema.do_insert/4
        (ecto) lib/ecto/association.ex:931: Ecto.Association.ManyToMany.on_repo_change/4
        (ecto) lib/ecto/association.ex:231: anonymous fn/7 in Ecto.Association.on_repo_change/6
        (elixir) lib/enum.ex:1623: Enum."-reduce/3-lists^foldl/2-0-"/3
        (ecto) lib/ecto/association.ex:228: Ecto.Association.on_repo_change/6
        (elixir) lib/enum.ex:1623: Enum."-reduce/3-lists^foldl/2-0-"/3
        (ecto) lib/ecto/association.ex:194: Ecto.Association.on_repo_change/3
        (ecto) lib/ecto/repo/schema.ex:527: Ecto.Repo.Schema.process_children/4
        (ecto) lib/ecto/repo/schema.ex:595: anonymous fn/3 in Ecto.Repo.Schema.wrap_in_transaction/6
        (ecto) lib/ecto/adapters/sql.ex:472: anonymous fn/3 in Ecto.Adapters.SQL.do_transaction/3
        (db_connection) lib/db_connection.ex:973: DBConnection.transaction_run/4
        (db_connection) lib/db_connection.ex:897: DBConnection.run_begin/3
        (db_connection) lib/db_connection.ex:671: DBConnection.transaction/3
        (my_server) web/controllers/tag_controller.ex:34: my_MyServer.TagController.create/2
        (my_server) web/controllers/tag_controller.ex:1: my_MyServer.TagController.action/2
        (my_server) web/controllers/tag_controller.ex:1: my_MyServer.TagController.phoenix_controller_pipeline/2
        (my_server) lib/my_server/endpoint.ex:1: my_MyServer.Endpoint.instrument/4

Well it took it, but then it tried to create a new Tag with that data, having the database throw a constraint error, meaning I should change the :index constraint to have the name “tags_lower_name_index”, fixed, but this obviously does not work either. Maybe it will take a changeset instead, might have special code to handle that:

parents_params = Map.get(tag_params, "parents", [])
parents_ids = Enum.map(parents_params, fn t ->
  String.to_integer(t)
end)
parents = Repo.all(from t in Tag, where: t.id in ^parents_ids)
  |> Enum.map(fn t -> Ecto.Changeset.change(t) end)
changeset = Tag.changeset_assoc(%Tag{parents: []}, put_in(tag_params["parents"], parents))

And trying this:

** (Ecto.CastError) expected params to be a map, got: `#Ecto.Changeset<action: nil, changes: %{}, errors: [], data: #my_MyServer.Tag<>, valid?: true>`
    (ecto) lib/ecto/changeset.ex:326: Ecto.Changeset.do_cast/4
    (my_server) web/models/tag.ex:22: my_MyServer.Tag.changeset/2
    (ecto) lib/ecto/changeset/relation.ex:98: Ecto.Changeset.Relation.do_cast/5
    (ecto) lib/ecto/changeset/relation.ex:284: Ecto.Changeset.Relation.map_changes/7
    (ecto) lib/ecto/changeset.ex:533: Ecto.Changeset.cast_relation/4
    (my_server) web/controllers/tag_controller.ex:28: my_MyServer.TagController.create/2
    (my_server) web/controllers/tag_controller.ex:1: my_MyServer.TagController.action/2
    (my_server) web/controllers/tag_controller.ex:1: my_MyServer.TagController.phoenix_controller_pipeline/2
    (my_server) lib/my_server/endpoint.ex:1: my_MyServer.Endpoint.instrument/4
    (my_server) lib/phoenix/router.ex:261: my_MyServer.Router.dispatch/2
    (my_server) web/router.ex:1: my_MyServer.Router.do_call/2
    (my_server) lib/my_server/endpoint.ex:1: my_MyServer.Endpoint.phoenix_pipeline/1
    (my_server) lib/plug/debugger.ex:93: my_MyServer.Endpoint."call (overridable 3)"/2
    (my_server) lib/my_server/endpoint.ex:1: my_MyServer.Endpoint.call/2
    (plug) lib/plug/adapters/cowboy/handler.ex:15: Plug.Adapters.Cowboy.Handler.upgrade/4
    (cowboy) src/cowboy_protocol.erl:442: :cowboy_protocol.execute/4

Nope. So how is this supposed to be able to set an existing tag as a parent of a tag? Maybe this ‘with:’ call on cast_assoc/4 might help, lets try to define something for it:

def changeset_assoc(struct, params \\ %{}) do
  struct
  |> changeset(params)
  |> cast_assoc(:parents, with: &changeset_as_assoc(&1, &2))
end

def changeset_as_assoc(struct, params \\ %{}) do
  my_MyServer.Repo.get!(from(my_MyServer.Tag), params.id) |> changeset()
end

After a quick experimenting I find that params is what each cell in the parents list is, so I give it the map from before since it will still not pass through the individual number strings from the web form (why do I have to massage these…):

parents_params = Map.get(tag_params, "parents", [])
parents_ids = Enum.map(parents_params, fn t ->
  String.to_integer(t)
end)
parents = parents_ids |> Enum.map(fn t -> %{id: t} end)
changeset = Tag.changeset_assoc(%Tag{parents: []}, put_in(tag_params["parents"], parents))

And then give it a try:

** (exit) an exception was raised:
    ** (RuntimeError) attempting to cast or change association `parents` from `my_MyServer.Tag` that was not loaded. Please preload your associations before casting or changing the struct
        (ecto) lib/ecto/changeset/relation.ex:65: Ecto.Changeset.Relation.load!/2
        (ecto) lib/ecto/repo/schema.ex:424: anonymous fn/4 in Ecto.Repo.Schema.surface_changes/4
        (elixir) lib/enum.ex:1623: Enum."-reduce/3-lists^foldl/2-0-"/3
        (ecto) lib/ecto/repo/schema.ex:413: Ecto.Repo.Schema.surface_changes/4
        (ecto) lib/ecto/repo/schema.ex:170: Ecto.Repo.Schema.do_insert/4
        (ecto) lib/ecto/association.ex:931: Ecto.Association.ManyToMany.on_repo_change/4
        (ecto) lib/ecto/association.ex:231: anonymous fn/7 in Ecto.Association.on_repo_change/6
        (elixir) lib/enum.ex:1623: Enum."-reduce/3-lists^foldl/2-0-"/3
        (ecto) lib/ecto/association.ex:228: Ecto.Association.on_repo_change/6
        (elixir) lib/enum.ex:1623: Enum."-reduce/3-lists^foldl/2-0-"/3
        (ecto) lib/ecto/association.ex:194: Ecto.Association.on_repo_change/3
        (ecto) lib/ecto/repo/schema.ex:527: Ecto.Repo.Schema.process_children/4
        (ecto) lib/ecto/repo/schema.ex:595: anonymous fn/3 in Ecto.Repo.Schema.wrap_in_transaction/6
        (ecto) lib/ecto/adapters/sql.ex:472: anonymous fn/3 in Ecto.Adapters.SQL.do_transaction/3
        (db_connection) lib/db_connection.ex:973: DBConnection.transaction_run/4
        (db_connection) lib/db_connection.ex:897: DBConnection.run_begin/3
        (db_connection) lib/db_connection.ex:671: DBConnection.transaction/3
        (my_server) web/controllers/tag_controller.ex:34: my_MyServer.TagController.create/2
        (my_server) web/controllers/tag_controller.ex:1: my_MyServer.TagController.action/2
        (my_server) web/controllers/tag_controller.ex:1: my_MyServer.TagController.phoenix_controller_pipeline/2

Although the changeset shows no errors the actual Repo.insert call failed by saying the parents association is not loaded, yet how could it be loaded if the Tag is new and does not exist to ‘be’ loaded… Maybe it is talking of the one I load in changeset_as_assoc, lets change it to:

def changeset_as_assoc(struct, params \\ %{}) do
  my_MyServer.Repo.get!(from(my_MyServer.Tag, preload: :parents), params.id) |> changeset()
end

And give this a try:

iex> Repo.insert(changeset)
{:error,  #Ecto.Changeset<action: :insert,
  changes: %{description: "Testing", name: "Test",
    parents: [#Ecto.Changeset<action: :insert,
      changes: %{description: "Parent tag for Issues", id: 1,
        inserted_at: #Ecto.DateTime<2016-07-05 21:27:32>, name: "Issues",
        updated_at: #Ecto.DateTime<2016-07-05 21:27:32>},
      errors: [id: {"has already been taken", []}], data: #my_MyServer.Tag<>,
      valid?: false>,
     #Ecto.Changeset<action: :insert, changes: %{}, errors: [],
      data: #my_MyServer.Tag<>, valid?: true>]}, errors: [],
  data: #my_MyServer.Tag<>, valid?: false>}}

So, it still seems to be trying to insert a new parent instead of just joining it.

Stopping here for now as I kept having more of the same issues and had the same issues when trying to update a parent tag listing when both the child and parent tags already exist by updating the tag.parents list in a huge variety of ways, including put_assoc/4 all to no avail

So I have got to be a missing something obvious or documented somewhere or in the Ecto codebase that I must be missing, so could someone show me the code needed to insert a new tag with parents (it works if parents is empty) and to update an existing tag with parents, in all cases the parents already exist and cannot be dynamically created? >.>