Delete many_to_many via parent

What is the de-facto/simplest way of deleting a single join entry via the parent?

I was hoping to use a changeset, to add a no assoc constraint on a different association, so deleting a tag for example can only happen when there are no associated comments. But I’m coming a little unstuck.

def remove_tag(post, tag) do
  post
  |> change
  |> no_assoc_constraint(:comments)
  |> put_assoc(:tags, #?????)
  |> Repo.update
end