How to validate JSON, and especially nested JSON objects, with Schemaless Ecto changesets?

Questions / Help
,
1

Currently I’m trying to use the Ecto changesets for validating a map, or actually HTTP response body as JSON decoded into a map.

According to the docs it is possible to use schemaless changesets in order to do the normalization/validation of this data. However, there is no mention or examples on how to use this when JSON objects are nested (or nested maps, after decoding the incoming JSON).

A trivial example would be this book map

%{
  "author": "B. Tate, S. DeBenedetto",
  "title": "Programming Phoenix LiveView"
}

for which a setup like below works perfectly fine:

defmodule Book do
  defstruct [:tile, :author]
  @types %{title: :string, author: :string}

  import Ecto.Changeset

  def changeset(%__MODULE__{} = book, params) do
    {book, @types}
    |> cast(params, Map.keys(@types))
    |> validate_required([:title, :author])
  end
end

However, for the case of a nested structure, where a person has a collection of books (library), e.g.:

%{
  name: "Sil van Brummen"
  library: [
    %{
      author: "B. Tate, S. DeBenedetto",
      title: "Programming Phoenix LiveView"
    },
    %{
      author: "C. McCord, B. Tate and J. Valim",
      title: "Programming Phoenix 1.4"
    }
  ]  
}

I’ve extended the code snippet to include a person struct with a name and library (note the library being of type {:array, :map}).

defmodule Person do
  defstruct [:name, :library]
  @types %{name: :string, library: {:array, :map}}

  import Ecto.Changeset

  def changeset(%__MODULE__{} = person, params) do
    {person, @types}
    |> cast(params, Map.keys(@types))
    |> validate_required([:name, :library])
  end

end

defmodule Book do
  defstruct [:tile, :author]
  @types %{title: :string, author: :string}

  import Ecto.Changeset

  def changeset(%__MODULE__{} = book, params) do
    {book, @types}
    |> cast(params, Map.keys(@types))
    |> validate_required([:title, :author])
  end
end

Running below code in iex with my setup does not validate the books, it only checks if the books are of type map.

Person.changeset(%Person{}, %{name: "Sil van Brummen", library: [%{title: "Programming Phoenix LiveView", author: "B. Tate, S. DeBenedetto"}, %{title: "Programming Phoenix 1.4", author: "C. McCord, B. Tate and J. Valim"}]})

Is it possible to define a nested schema structure where the library is of type {:array, Book} such that both the person and all books will be validated? And if it is possible, how would this nested schemaless setup look like?

For context; my ultimate goal is to be able to normalize/validate a nested map where the library consists of various books, identified by type and each book type has their own structure.

%{
  name: "Sil van Brummen"
  library: [
    %{
      type: "coding",
      author: "B. Tate, S. DeBenedetto",
      title: "Programming Phoenix LiveView",
      language: "elixir"
    },
    %{
      type: "coding",
      author: "C. McCord, B. Tate and J. Valim",
      title: "Programming Phoenix 1.4",
      language: "elixir"
    },
    %{
      type: "cooking",
      author: "Marcella Hazan",
      title: "The Essentials of Classic Italian Cooking",
      kitchen: "Italian" 
    }
  ]  
}

Cheers,
Sil

2 Likes
2

This does not answer your question regarding using Ecto.Changesets but maybe this can help with validation: GitHub - mhanberg/schematic: 📐 schematic

2 Likes
3

If you’re already declaring structs for the contents, I’d recommend using embedded_schema and embeds_many.

2 Likes
4

Thanks for the suggestion Kevin, maybe I’ll use this when I can’t sort it out with Ecto.

From a learning/understanding perspective I would rather figure out how to do this with Ecto and the schema validation they provide.

5

I’ve tried the embedded_schema with embeds_many option, but this seems to be limiting to embedding structs of a single type. In my use case the array would contain various struct types, e.g.

%{
  type: "coding",
  language: "Elixir",
  title: "",
  author: ""
}
%{
  type: "cooking",
  kitchen: "Italian",
  title: "",
  author: ""
}

The embeds_many relation still is of only a single type, so below example does not work

defmodule Person do
  use Ecto.Schema
  import Ecto.Changeset

  alias ApiValidation.Coding
  alias ApiValidation.Cookbook

  @primary_key false
  embedded_schema do
    field :name, :string
    embeds_many :library, Coding # Single struct type allowed
  end

  def changeset(data, params) do
    data
    |> cast(params, [:name])
    |> cast_embed(:library, with: &defer/2)
    |> validate_required([:name])
  end

  def defer(data, %{type: type} = params) when type == "coding" do
    Coding.changeset(data, params)
  end

  def defer(data, %{type: type} = params) when type == "cooking" do
    Cookbook.changeset(data, params)
  end
end

I hoped to achieve more flexibility by using the schemaless changesets since the types can be defined inside the function rather than on module level. But this still requires tackling the nesting issue.

1 Like
6

Haven‘t used it myself, but maybe this library is what you‘d need?

1 Like
7

Nice find! I’ll take a look at this library, it seems to do exactly what I’m trying to achieve :slight_smile: