Dynamic Embeds in Ecto

Hey guys. I know this is an old topic, but as I write more and more complex application with Elixir and Ecto I feel like we really need a way to let developers use dynamic embeds.

Few words about our use case, we have a Transport schema which stores various common fields and settings of a transport. But depending on transport type (eg. Twillio and Facebook Messenger) settings can be very different and also there are DB constraints that should be in place for those settings.

We do work around this issue with an application logic which takes params for the embedded schema (which defined as :map type on parent changeset) and validates/casts them if embedded changeset is valid or properly adds errors to the parent otherwise. Here are some code:

A function that shows how dynamic changeset works in our case:

  defp cast_provider_settings(changeset, provider_field, provider_settings_field) do
    with {:ok, provider} <- fetch_change(changeset, provider_field),
         {:ok, settings} <- fetch_change(changeset, provider_settings_field),
         provider_settings_changeset = Provider.settings_changeset(provider, settings),
         {:ok, valid_settings} <- Validator.fetch_valid_attrs(provider_settings_changeset) do
      put_change(changeset, provider_settings_field, valid_settings)
    else
      :error ->
        changeset

      {:error, :not_found} ->
        changeset

      {:error, %{valid?: false} = settings_changeset} ->
        put_embedded_error(changeset, provider_settings_field, settings_changeset)
    end
  end

Here is how you can add an error to an embedded changeset defined as map:

  defp put_embedded_error(changeset, embed_field, embedded_changeset) do
    embedded_type =
      {:embed,
       %Ecto.Embedded{
         cardinality: :one,
         field: embed_field,
         on_cast: nil,
         on_replace: :raise,
         owner: %{},
         related: Transport,
         unique: true
       }}

    %{
      changeset
      | changes: Map.put(changeset.changes, embed_field, embedded_changeset),
        types: Map.put(changeset.types, embed_field, embedded_type),
        valid?: false
    }
  end

(Notice that you can’t override types and leave embedded changeset in Ecto Schema where :map type was defined because you would get a cast error. Ecto.Changeset does use pre-compiled type information when insert happens so overriding only helps when you use functions like traverse_errors/2.)

And even if you do that, there is a lot of issues that persist here. The main one right now for us is constraints - they are lost when embedded schema turned into a map and moving them manually to parent doesn’t make sense (error field would point to a wrong direction).

Other ways to hack around:

1. Define multiple schemas per database entity (or even combine that with PostgreSQL table inheritance). This one looks weird for me because when I fetch data back from DB I do want to see only one kind of schema. Data that I want to put there should be exactly what I get back.
2. Do not use dynamic embeds. This option looks poor because there is sooo many use cases where dynamic embed makes perfect sense.

As a very raw suggestion how we can deal with that:

1. We might add a :changeset type for Ecto.Schema.
2. It’s application responsibility to actually implement logic how embedded changeset gets there, on which fields it’s resolved, etc. (I don’t think that Ecto needs to add any kind of magic here.)
3. Repo operations should take care of changesets in :changeset fields in a same way as they would do with usual embedded schema.

OR

1. Make Ecto use type information from changeset (removing the calls to Schema.__*__ functions) which is not straightforward and would make changesets structs much bigger. (See this issue.)