How to handle setup for a form with recursive data using `with_cte`?

I have a recursive set of data using with_cte and using the docs I have been able to pull all rows I need from the database, but it is returned as a flat list. How would you take deeply nested data like this and handle changesets and forms to update this data?

You cannot, as DBs are returning such data as a flat lists. So you need to do post-processing on your own to achieve what you want.

Okay, lets say this is what I get returned to me using the recursive query:

[
  %Portal.Field{
    __meta__: #Ecto.Schema.Metadata<:loaded, "fields">,
    children: #Ecto.Association.NotLoaded<association :children is not loaded>,
    handle: "root",
    id: "4234a8d8-29e7-40ca-b286-0dd14af5ffd3",
    inserted_at: ~N[2021-03-31 02:53:21],
    order: nil,
    parent: #Ecto.Association.NotLoaded<association :parent is not loaded>,
    parent_id: nil,
    updated_at: ~N[2021-03-31 02:53:21]
  },
  %Portal.Field{
    __meta__: #Ecto.Schema.Metadata<:loaded, "fields">,
    children: #Ecto.Association.NotLoaded<association :children is not loaded>,
    handle: "child_one",
    id: "0d196017-2a71-45a0-82e7-e159330f530b",
    inserted_at: ~N[2021-03-31 02:53:21],
    order: 0,
    parent: #Ecto.Association.NotLoaded<association :parent is not loaded>,
    parent_id: "4234a8d8-29e7-40ca-b286-0dd14af5ffd3",
    updated_at: ~N[2021-03-31 02:53:21]
  },
  %Portal.Field{
    __meta__: #Ecto.Schema.Metadata<:loaded, "fields">,
    children: #Ecto.Association.NotLoaded<association :children is not loaded>,
    handle: "child_two",
    id: "e9da58af-bc0d-4667-92f1-55c759f2d7c9",
    inserted_at: ~N[2021-03-31 02:53:21],
    order: 1,
    parent: #Ecto.Association.NotLoaded<association :parent is not loaded>,
    parent_id: "4234a8d8-29e7-40ca-b286-0dd14af5ffd3",
    updated_at: ~N[2021-03-31 02:53:21]
  },
  %Portal.Field{
    __meta__: #Ecto.Schema.Metadata<:loaded, "fields">,
    children: #Ecto.Association.NotLoaded<association :children is not loaded>,
    handle: "child_three",
    id: "c4edd898-5dc4-4eb5-ab07-7ecd2be685ac",
    inserted_at: ~N[2021-03-31 02:53:21],
    order: 2,
    parent: #Ecto.Association.NotLoaded<association :parent is not loaded>,
    parent_id: "4234a8d8-29e7-40ca-b286-0dd14af5ffd3",
    updated_at: ~N[2021-03-31 02:53:21]
  },
  %Portal.Field{
    __meta__: #Ecto.Schema.Metadata<:loaded, "fields">,
    children: #Ecto.Association.NotLoaded<association :children is not loaded>,
    handle: "child_four",
    id: "1241cb4a-75d8-4fa6-879d-5e83bc741bb3",
    inserted_at: ~N[2021-03-31 02:53:21],
    order: 0,
    parent: #Ecto.Association.NotLoaded<association :parent is not loaded>,
    parent_id: "0d196017-2a71-45a0-82e7-e159330f530b",
    updated_at: ~N[2021-03-31 02:53:21]
  },
  %Portal.Field{
    __meta__: #Ecto.Schema.Metadata<:loaded, "fields">,
    children: #Ecto.Association.NotLoaded<association :children is not loaded>,
    handle: "child_five",
    id: "3e620503-7980-447f-93b4-4b4696236839",
    inserted_at: ~N[2021-03-31 02:53:21],
    order: 0,
    parent: #Ecto.Association.NotLoaded<association :parent is not loaded>,
    parent_id: "1241cb4a-75d8-4fa6-879d-5e83bc741bb3",
    updated_at: ~N[2021-03-31 02:53:21]
  },
  %Portal.Field{
    __meta__: #Ecto.Schema.Metadata<:loaded, "fields">,
    children: #Ecto.Association.NotLoaded<association :children is not loaded>,
    handle: "child_six",
    id: "a3317355-bb3a-4a7f-b970-fc45491e0f9e",
    inserted_at: ~N[2021-03-31 02:53:21],
    order: 1,
    parent: #Ecto.Association.NotLoaded<association :parent is not loaded>,
    parent_id: "1241cb4a-75d8-4fa6-879d-5e83bc741bb3",
    updated_at: ~N[2021-03-31 02:53:21]
  }
]

How would I be able to build out the children values of the root Struct. I tried doing it using an Enum.reduce, but when I tried to set the children key I would get an argument error.

You can start by grouping your list by parent_id, this will give You the list of children per id.

map = Enum.group_by(list, & &1.parent_id)

Then iterate over root elements, the one with parent_id is_nil… and use recursion to build children.

Something like this (not tested)

  def build_children(nil, _map), do: nil
  def build_children(%{id: id}, map) do
    case map |> Map.get(id) do
      nil -> nil
      list -> Enum.map(list, &Map.put(&1, :children, build_children(&1)))
    end
  end