Ecto.Schema `any` type?

Questions / Help
1

I’m currently using Ecto in a project for data validation and rendering into Phoenix only (i.e., only using Ecto.Schema and Ecto.Changeset). This works well, but my schema allows for one field to be any valid term object, constrained by the value of another field. From what I can gather in the docs, Ecto.Schema requires a type, but my schema is not as rigid as this requirement.

Any thoughts on how to handle this?

2

Finally, schemas can also have virtual fields by passing the virtual: true option. These fields are not persisted to the database and can optionally not be type checked by declaring type :any .

A bit further in the docs.

1 Like
3

What value would you store in the database for this ‘any’ column?

4

That’s exactly what I needed. Helps to read everything. Thanks!

5

The original poster sounded like they weren’t expecting to persist this data, just interacting with changesets.

But if necessary, it’s possible to make it work - set up an underlying column that accepts a binary value (some DBs will call that a BLOB, not all) and then write an Ecto type that uses Erlang Term Format:

  defmodule CustomTypes.Term do
    @behaviour Ecto.Type
    @impl true
    def type, do: :binary

    # used by Changeset.cast
    @impl true
    @spec cast(term()) :: {:ok, term()}
    def cast(bin), do: {:ok, bin}

    # DB -> struct
    @impl true
    @spec load(binary()) :: {:ok, term()}
    def load(bin), do: {:ok, bin |> :erlang.binary_to_term()}

    # struct -> DB
    @impl true
    @spec dump(term()) :: {:ok, binary()}
    def dump(bin), do: {:ok, bin |> :erlang.term_to_binary()}

    @impl true
    @spec embed_as(atom()) :: :self
    def embed_as(_), do: :self

    @impl true
    @spec equal?(term(), term()) :: boolean()
    def equal?(term1, term2), do: term1 == term2
  end

Some very large caveats:

  • DO NOT use this if the field can be written to by users. Deserializing user-controlled ETF is potentially Very Bad for security

  • DO NOT use this for historical data storage with structs; deserializing those when the underlying definition has changed can cause errors. If you must retain this data for a long time, normalize everything to basic types (maps, tuples, etc) with stable definitions

  • DO NOT use this if you need to query the data from SQL. It’s probably possible to write a PL/SQL function that parses ETF, but needing to query is a good sign this is not the right format for your application.

We use this at work for database persistence of the state of GenServers that only live for a single day; saves writing a lot of serialization / deserialization code.

2 Likes