Ecto custom type issue: "value `[list_of_ids]` in `where` cannot be cast to type {:in, :id} in query"

Questions / Help
,
1

In my project, ids are just auto incremented ids managed by the database. When showing an id to the user, it is encoded using Hashids. I have a helper module that just wraps it to avoid the boilerplate of passing the salt to every function.

Until now I’ve been manually converting the ids from integers to hashes and vice versa every time a hash was provided and an id was to be shown.

I wanted to avoid that, so I wrote a custom Ecto Type:

defmodule Embers.Hashid do
  @behaviour Ecto.Type

  alias Embers.Helpers.IdHasher

  def type(), do: :id

  def cast(int) when is_integer(int) do
    {:ok, IdHasher.encode(int)}
  end

  def cast(id) when is_binary(id) do
    {:ok, id}
  end

  def cast(_) do
    :error
  end

  def dump(id) when is_binary(id) do
    case IdHasher.decode(id) do
      id when is_integer(id) -> {:ok, id}
      _ -> :error
    end
  end

  def load(id) when is_integer(id) do
    {:ok, IdHasher.encode(id)}
  end

end

It should cast integers into hashes, convert integers from the db into hashes, and convert hashes to integers before putting them in the db.

The type is used in the schemas via the @primary_key module attribute:
@primary_key {:id, Embers.Hashid, autogenerate: true}

For single entities, without preloads, it works, but when trying to preload an association Ecto blows with this error:

[error] Task #PID<0.11545.0> started from #PID<0.11541.0> terminating
** (Ecto.Query.CastError) deps/ecto/lib/ecto/association.ex:623: value `["KV3A8", "Ja344", "8n43V", "304Ko"]` in `where` cannot be cast to type {:in, :id} in query:

from r0 in Embers.Reactions.Reaction,
  where: r0.post_id in ^["KV3A8", "Ja344", "8n43V", "304Ko"],
  order_by: [asc: r0.post_id],
  select: {r0.post_id, r0}

    (elixir 1.10.4) lib/enum.ex:2111: Enum."-reduce/3-lists^foldl/2-0-"/3
    (elixir 1.10.4) lib/enum.ex:1520: Enum."-map_reduce/3-lists^mapfoldl/2-0-"/3
    (elixir 1.10.4) lib/enum.ex:2111: Enum."-reduce/3-lists^foldl/2-0-"/3
    (ecto 3.0.7) lib/ecto/repo/queryable.ex:132: Ecto.Repo.Queryable.execute/4
    (ecto 3.0.7) lib/ecto/repo/queryable.ex:18: Ecto.Repo.Queryable.all/3
    (ecto 3.0.7) lib/ecto/repo/preloader.ex:188: Ecto.Repo.Preloader.fetch_query/8
    (ecto 3.0.7) lib/ecto/repo/preloader.ex:119: Ecto.Repo.Preloader.preload_assoc/10
    (elixir 1.10.4) lib/task/supervised.ex:90: Task.Supervised.invoke_mfa/2
    (elixir 1.10.4) lib/task/supervised.ex:35: Task.Supervised.reply/5
    (stdlib 3.13) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
Function: &:erlang.apply/2
    Args: [#Function<8.84726962/1 in Ecto.Repo.Preloader.maybe_pmap/4>, [{{:assoc, %Ecto.Association.Has{cardinality: :many, defaults: [], field: :reactions, on_cast: nil, on_delete: :nothing, on_replace: :raise, owner: Embers.Posts.Post, owner_key: :id, queryable: Embers.Reactions.Reaction, related: Embers.Reactions.Reaction, related_key: :post_id, relationship: :child, unique: true, where: []}, {0, :post_id}}, nil, nil, []}]]

I’m clueless on what could cause that error. This issue seemed similar, but the error I’m getting mentions casting, not dumping.

2

I don’t know either, but wondering if you change this to:

  def cast(other) do
    IO.inspect(other, label: "Casting Embers.HashID")
    :error
  end

You might at least see if this give you a clue?

3

After a lot of IO.ispecting I found out that the issue only happened when using preloads.
For some reason Ecto wasn’t using the custom type, so it was trying to cast the strings to :id, and that obviously failed. It expected the ids to be integers, not strings.
I finally got to this part of the Ecto.Schema.belongs_to docs:

:type - Sets the type of automatically defined :foreign_key . Defaults to: :integer and can be set per schema via @foreign_key_type

So I tried and configured the associations as:
belongs_to(:field, SchemaModule, type: Embers.Hashid)
and now everything works fine.

That… made sense, but the documentation is split between Ecto.Type and Ecto.Schema, so it wasn’t clear from the start how everything fits together and the error wasn’t helpful at all.

The Programming Phoenix book does use a custom type as primary key, that’s where I got the @primary_key {:id, Embers.Hashid, autogenerate: true} bit, but it does not mention anything about associations with custom types. I couldn’t find references to it in the Programming Ecto book, either.

I guess this is an opportunity to improve Ecto.Type or Ecto.Schema docs.
EDIT: PR sent :slight_smile:

3 Likes