How to properly handle nested associations in forms with manual data manipulation

Hello and welcome!

Without seeing your actual form it sounds like that from the user’s perspective they are adding “panels” to the configuration, not “panel groups.” You would have a much easier time if also associated Panel straight to Configuration and used that association for your forms. You can still have the panel_groups association which, with the proper preloads, will be auto-magically grouped for you.

Something like this:

source "configurations" do
  has_many :pannel_groups, PannelGroup
  has_many :panels, through: pannel_groups
end

def changeset(configuration, attrs) do
  configuration
  |> cast(attrs, [...])
  |> cast_assoc(:panel_groups)
  |> cast_assoc(:panels)

(note the lack of required: true on the cast_assocs)

I’m not sure i understand how this would help? This would make my panels one level less deep but it looks like it won’t solve my problems with put_assoc since i would still have to use it to update the configuration.

I think i may have to separate my panel_groups into a state of its own. Manipulate the data there and then when i submit the form, put_assoc once the state of the panel_groups into the configuration. But then i would loose the validation i need to do on my panels.

Still would love to know the best practice for this kind of stuff.

Thinking about it, having a separate state would not work i think, because i need Ecto validation on the panels for the width and other fields. So i need to update the form in the liveview.

If this help, here what my schemas looks like. I trimmed them down to keep only the relevant stuff.

defmodule Order do
  schema "orders" do
    field(:configuration_type, Ecto.Enum, values: [:custom, :kit])
    has_one(:custom_configuration, Orders.CustomConfiguration)
    has_one(:kit_configuration, Orders.Configuration)
  end

  @required ~w[configuration_type]a

  def changeset(order, attrs) do
    status = Map.get(attrs, "status")

    order
    |> cast(attrs, @required)
    |> maybe_validate_required(status)
    |> cast_assoc(:order_kit_configuration,
      with: &KitConfiguration.changeset(&1, &2, status)
    )
    |> cast_assoc(:order_custom_configuration,
      with: &CustomConfiguration.changeset(&1, &2, status)
    )
  end

  def maybe_validate_required(changeset, :draft), do: changeset
  def maybe_validate_required(changeset, _), do: validate_required(changeset, @required)
end

defmodule CustomConfiguration do
  schema "order_custom_configurations" do
    field(:panels_count, :integer)
    field(:panels_height, :decimal, default: Decimal.new(90))

    belongs_to(:order, Orders.Order)

    has_many(:panel_groups, Orders.PanelGroup, on_replace: :delete)
  end

  @required ~w[panels_count panels_height]a
  def changeset(custom_configuration, attrs, status \\ nil) do
    custom_configuration
    |> cast(attrs, @required)
    |> maybe_validate_required(status)
    |> validate_number(:panels_count, greater_than_or_equal_to: 1, less_than_or_equal_to: 4)
    |> validate_number(:panels_height, greater_than_or_equal_to: 60, less_than_or_equal_to: 120)
    |> cast_assoc(:panel_groups,
      with: &CustomConfigurationPanelGroup.changeset(&1, &2, status),
      on_replace: :delete
    )
  end

  def maybe_validate_required(changeset, :draft), do: changeset
  def maybe_validate_required(changeset, _), do: validate_required(changeset, @required)
end

defmodule PanelGroup do
  schema "order_custom_configuration_panel_groups" do
    field :background_position, :string, default: "0% 0%"
    
    has_many :panels, Orders.CustomConfigurationPanel, foreign_key: :group_id, on_replace: :delete
    
    belongs_to :custom_configuration, Orders.CustomConfiguration
    belongs_to :image, Image
  end

  @required ~w[image_id]a
  @optional ~w[background_position]a
  def changeset(group, attrs, status \\ nil) do
    group
    |> cast(attrs, @required ++ @optional)
    |> maybe_validate_required(status)
    |> cast_assoc(:panels,
      with: &CustomConfigurationPanel.changeset(&1, &2, status),
      on_replace: :delete
    )
  end

  def maybe_validate_required(changeset, :draft), do: changeset
  def maybe_validate_required(changeset, _), do: validate_required(changeset, @required)
end

defmodule Panel do
  schema "order_custom_configuration_panels" do
    field(:width, :decimal, default: Decimal.new(72))

    belongs_to(:group, CustomConfigurationPanelGroup)

    field(:index, :integer)

    timestamps()
  end

  @required ~w[width finish index]a
  def changeset(panel, attrs, status \\ nil) do
    panel
    |> cast(attrs, @required)
    |> maybe_validate_required(status)
    |> validate_number(:width, greater_than_or_equal_to: 1, less_than_or_equal_to: 72)
  end

  def maybe_validate_required(changeset, :draft), do: changeset
  def maybe_validate_required(changeset, _), do: validate_required(changeset, @required)
end

I can take a closer look at this later and would help too if you can share the form.

If you make it one less level deep you wouldn’t need put_assoc, you can just |> cast_assoc(:panels) using the panel’s changeset. This would just be for the form itself. This assumes a panel would have a panel_group_id when adding it to the order/configuration. Again, this would only be for the form, you can still have the panel_group association which you can preload along with their panels and then all the panels would be in their proper groups.

I’m saying this based off the information that you are trying to manually sort them into groups.

I tinkered a bit around with the idea you had of having the panels one levels less deep via has_many :panels, through: panel_groups on my Configuration. This seems promising! But then i realize it would not work since :through associations are read-only

From the ecto docs:

Note :through associations are read-only. For example, you cannot use Ecto.Changeset.cast_assoc/3 to modify through associations.

My form is quite big so i can’t really show it here but i will show a trimmed down version that hopefully get the big picture.

First let me explain what i’m trying to do.

I have a sidebar with a long vertical form in it with multiple sections for the order. There is a “Panel” section where the user choose the number of panel and then choose the dimensions and finish and other properties for each panel. At this point, there is no concept of group. The user is just configuring panel individually.

Let say he choose 4 panel. Then, on the right part of the screen he will see 4 empty panels with the right dimensions.

[ empty ] [ empty ] [ empty ] [ empty ] 

In reality here we have 4 panel group with each 1 panel inside.

Then the user can select 1 or more panels and assign an image to it. This is where the group come in play. The image is attached to the group not the panel. Let say the user select the two first panel and assign an image to it. Now the interface look something like this,

[ image spanning two panels] [empty] [empty]

Again in reality we have now 1 group with two panels and 2 groups with each 1 panel.

I will admit, this form is more complex than anything i had to do form wise before and got me quite struggling. I did some weird workaround to make it work regarding CSS.

<.form for={@form} phx-submit="submit_order" phx-change="validate_order">
  <%!-- SIDEBAR --%> 
  # Bunch of other sections for the order...

  # The user can choose between a predefined configuration or a custom one.
  <.input type="select" field={@form[:configuration_type]} options={configuration_type_options()} />
  
  # Here is the part for the panels. The user is indeed not aware of groups of panel he just configure each
  # panels. Their dimensions and other properties..
  <%= if Changeset.get_field(@form.source, :configuration_type) == :custom do %>
    <.inputs_for :let={custom_configuration_form} field={@form[:order_custom_configuration]}>
      <.input type="select" field={custom_configuration_form[:panels_count]} phx-change="update_panels" />
      <.input type="number" field={custom_configuration_form[:panels_height]} />
      
      <.inputs_for :let={panel_group_form} field={custom_configuration_form[:panel_groups]}>
        <.input type="hidden" field={panel_group_form[:image_id]} value={Changeset.get_field(panel_group_form.source, :image_id)} />
        
        <.inputs_for :let={panel_form} field={panel_group_form[:panels]}>
          <.input type="hidden" field={panel_form[:index]} value={Changeset.get_field(panel_form.source, :index)} />
          <.input type="number" field={panel_form[:width]} />
        </.inputs_for>
      </.inputs_for>
    </.inputs_for>
  <% end %>
  
  # Bunch of other sections for the order 
  <%!-- SIDEBAR END --%>

  <%!-- PANEL CONFIGURATOR --%>
  <%= if Changeset.get_field(@form.source, :configuration_type) == :custom do %>
    
    # I have a hook that is reponsible for handling the positioning of the image in the panel group.
    <div phx-hook="orderPanelConfigurator" id="panel_configurator">
      <% panels_height = get_panels_height(@form) %>
      
      <div class="grid">
        <%= for {group, index} <- Enum.with_index(get_panel_groups(@form))do %>
          <% panels = Changeset.get_assoc(group, :panels) %>
          <% image_id = Changeset.get_field(group, :image_id) %>

          <div class="panel-group" style={get_panel_group_style(group, @form)} data-group-index={index}>
            <%= for panel <- panels do %>
              <% panel_width = get_panel_width(panel) %>
              <% index = Changeset.get_field(panel, :index) %>

              # Image is displayed via background-image url
              <div
                class={get_panel_class(@selected_panels, panel, group)}
                style={"flex: #{panel_width}; aspect-ratio: #{panel_width}/#{panels_height};"}
                phx-click="select_panel"
                phx-value-index={if image_id, do: -1, else: index}
              >
                # Stuff inside here that is not important.
              </div>
            <% end %>
          </div>
        <% end %>
      </div>
    </div>
  <% end %>
  <%!-- PANEL CONFIGURATOR --%>
</.form>

To me, my data structure make sense. I tried other approach of having the panels directly on the configuration. Then they could maybe have a group, maybe not. But having them always in a group and having the image on the group is what made more sense in the end.

My struggle is with manipulating the groups in the context of the form really. How can i move the panel around, changing in which group they belong without using put_assoc multiple times? What would be the right tool here? I want to keep the validation on the panels so that when the user enters an invalid number for the width in the sidebar for example, the UI reflect the error and set the input red.

Ah crap, I totally forgot you can’t manipulate through associations. Sorry about that!

I’m confused by this part:

At this point, there is no concept of group.

But then you say:

The image is attached to the group not the panel.

Is this a completely different step or do you mean that there is no concept as far as the user is concerned at this point? Assuming it is just from the user’s perspective, I would still build a group any time a panel is added using inputs_for as normal, even if it’s not visible. But yes, sorry if that isn’t what you’re talking about.

As for not being able to call put_assoc multiple times, it looks like you are trying to manipulating the same changeset after it’s been validated. The handle_event that is doing the updates should be constructing a brand new changeset from params. Can you share what you have there?

As for updating nested changeset, it’s just generally a pain. This library may be of interest:
https://github.com/woylie/ecto_nested_changeset

Lastly, for a bit of nitpicky advice, your order changeset function would be a bit more robust if you didn’t use Map.get on params and instead do this:

  def changeset(order, attrs) do
    changeset = cast(order, attrs, @required)
    status = get_change(changeset, :status)

    changeset
    |> maybe_validate_required(status)
    |> ...

Sorry if this still isn’t super helpful, it is quite complicated and hard to tell without seeing everything!

Sorry that was unclear. Yes, i meant that at this point from the user perspective in the UI there is no concept of group but the there is in the data structure.

This opened my eyes. I wasn’t working with the form params for the grouping operation. I was only working with the changeset ( source ) of the form. Updating the data with put_assoc. I changed my approach and now i’m only working with the form params. Accessing them via socket.assigns.form.params, updating them manually when i group/ungroup panels and not touching the changeset at all. This seems to work like a charm. The only problem i have is that, on the first mount, form.params is empty until a phx-change is triggered at least once. I worked around that by having a hidden input with a js hook that trigger a change on mount. A bit sketchy but seems to work just fine.

Haven’t read through the whole thread yet, but FWIW this is precisely what phoenix_ecto does in to_form:

https://github.com/phoenixframework/phoenix_ecto/blob/76e5b7f8286c6212db5cabb68934c0ee3758d667/lib/phoenix_ecto/html.ex#L171-L176