How to properly handle nested associations in forms with manual data manipulation

Phoenix Forum Questions / Help
, ,
1

I’m struggling with the proper way to handle nested associations in a Phoenix form when I need to manually manipulate form data.

Let me explain,
The structure of my data look like this

%Order{
  configuration: %Configuration{
    panel_groups: []
  }
}

I’m building a UI where you can select the number of panels you want, then you can group and ungroup panels. For example, when a user select panels and clicks a “group panels” button, I want to take the selected panels and group them.

First i select that i want 4 panels:

%Order{
  configuration: %Configuration{
    panel_groups: [
      %PanelGroup{image_id: some_id, panels: [%Panel{}]},
      %PanelGroup{image_id: some_id, panels: [%Panel{}]},
      %PanelGroup{image_id: some_id, panels: [%Panel{}]},
      %PanelGroup{image_id: some_id, panels: [%Panel{}]}
    ]
  }
}

Then i group some panels together:

%Order{
  configuration: %Configuration{
    panel_groups: [
      %PanelGroup{image_id: some_id, panels: [%Panel{}, %Panel{}]},
      %PanelGroup{image_id: some_id, panels: [%Panel{}, %Panel{}]}
    ]
  }
}

I used get_assoc and put_assoc everywhere to get and update the data in my form. Here is a pseudo example of grouping panels together.

%{source: changeset} = form 

configuration = Changeset.get_assoc(form.source, :configuration)
current_panel_groups = Changeset.get_assoc(configuration, :panel_groups)
new_panel_groups = function_that_rebuild_the_panel_groups()
custom_configuration = Changeset.put_assoc(configuration, new_panel_groups)
changeset = Changeset.put_assoc(changeset, configuration)
to_form(changeset, action: :update)

This works fine when creating a new order, but I started facing issues when adding the possibility to update an existing order:

  1. Somewhere in my page i need to display the panels. I made a function that takes the form and return the panel_groups using get_assoc, In the new form i didn’t had problem, but when editing an existing order, when I use get_assoc after a put_assoc, I get both the old panel groups (with action: :replace) and the new ones (with action: :insert). As a quick workaround i decided to filter out the :replace ones, but this feels wrong and naive.
  2. This doesn’t happen in a new form but when editing, If I try to perform a second put_assoc operation (e.g., ungroup panels, then group different ones), I get this error
cannot replace related %PanelGroup{} This typically happens when you are calling put_assoc/put_embed with the results of a previous put_assoc/put_embed/cast_assoc/cast_embed operation, which is not supported. You must call such operations only once per embed/assoc, in order for Ecto to track changes efficiently

So i’m starting to understand that put_assoc is maybe not the right choice here. What would be the best way to manipulate form data manually?

PS: The reason i’m using a form and not just a state using a map where i can manipulate freely my data is that the panels stuff is just a small part of my form and i didn’t want to have multiple state/source-of-truth and also i need validation

2

Hello and welcome!

Without seeing your actual form it sounds like that from the user’s perspective they are adding “panels” to the configuration, not “panel groups.” You would have a much easier time if also associated Panel straight to Configuration and used that association for your forms. You can still have the panel_groups association which, with the proper preloads, will be auto-magically grouped for you.

Something like this:

source "configurations" do
  has_many :pannel_groups, PannelGroup
  has_many :panels, through: pannel_groups
end

def changeset(configuration, attrs) do
  configuration
  |> cast(attrs, [...])
  |> cast_assoc(:panel_groups)
  |> cast_assoc(:panels)

(note the lack of required: true on the cast_assocs)

3

I’m not sure i understand how this would help? This would make my panels one level less deep but it looks like it won’t solve my problems with put_assoc since i would still have to use it to update the configuration.

I think i may have to separate my panel_groups into a state of its own. Manipulate the data there and then when i submit the form, put_assoc once the state of the panel_groups into the configuration. But then i would loose the validation i need to do on my panels.

Still would love to know the best practice for this kind of stuff.

4

Thinking about it, having a separate state would not work i think, because i need Ecto validation on the panels for the width and other fields. So i need to update the form in the liveview.

If this help, here what my schemas looks like. I trimmed them down to keep only the relevant stuff.

defmodule Order do
  schema "orders" do
    field(:configuration_type, Ecto.Enum, values: [:custom, :kit])
    has_one(:custom_configuration, Orders.CustomConfiguration)
    has_one(:kit_configuration, Orders.Configuration)
  end

  @required ~w[configuration_type]a

  def changeset(order, attrs) do
    status = Map.get(attrs, "status")

    order
    |> cast(attrs, @required)
    |> maybe_validate_required(status)
    |> cast_assoc(:order_kit_configuration,
      with: &KitConfiguration.changeset(&1, &2, status)
    )
    |> cast_assoc(:order_custom_configuration,
      with: &CustomConfiguration.changeset(&1, &2, status)
    )
  end

  def maybe_validate_required(changeset, :draft), do: changeset
  def maybe_validate_required(changeset, _), do: validate_required(changeset, @required)
end

defmodule CustomConfiguration do
  schema "order_custom_configurations" do
    field(:panels_count, :integer)
    field(:panels_height, :decimal, default: Decimal.new(90))

    belongs_to(:order, Orders.Order)

    has_many(:panel_groups, Orders.PanelGroup, on_replace: :delete)
  end

  @required ~w[panels_count panels_height]a
  def changeset(custom_configuration, attrs, status \\ nil) do
    custom_configuration
    |> cast(attrs, @required)
    |> maybe_validate_required(status)
    |> validate_number(:panels_count, greater_than_or_equal_to: 1, less_than_or_equal_to: 4)
    |> validate_number(:panels_height, greater_than_or_equal_to: 60, less_than_or_equal_to: 120)
    |> cast_assoc(:panel_groups,
      with: &CustomConfigurationPanelGroup.changeset(&1, &2, status),
      on_replace: :delete
    )
  end

  def maybe_validate_required(changeset, :draft), do: changeset
  def maybe_validate_required(changeset, _), do: validate_required(changeset, @required)
end

defmodule PanelGroup do
  schema "order_custom_configuration_panel_groups" do
    field :background_position, :string, default: "0% 0%"
    
    has_many :panels, Orders.CustomConfigurationPanel, foreign_key: :group_id, on_replace: :delete
    
    belongs_to :custom_configuration, Orders.CustomConfiguration
    belongs_to :image, Image
  end

  @required ~w[image_id]a
  @optional ~w[background_position]a
  def changeset(group, attrs, status \\ nil) do
    group
    |> cast(attrs, @required ++ @optional)
    |> maybe_validate_required(status)
    |> cast_assoc(:panels,
      with: &CustomConfigurationPanel.changeset(&1, &2, status),
      on_replace: :delete
    )
  end

  def maybe_validate_required(changeset, :draft), do: changeset
  def maybe_validate_required(changeset, _), do: validate_required(changeset, @required)
end

defmodule Panel do
  schema "order_custom_configuration_panels" do
    field(:width, :decimal, default: Decimal.new(72))

    belongs_to(:group, CustomConfigurationPanelGroup)

    field(:index, :integer)

    timestamps()
  end

  @required ~w[width finish index]a
  def changeset(panel, attrs, status \\ nil) do
    panel
    |> cast(attrs, @required)
    |> maybe_validate_required(status)
    |> validate_number(:width, greater_than_or_equal_to: 1, less_than_or_equal_to: 72)
  end

  def maybe_validate_required(changeset, :draft), do: changeset
  def maybe_validate_required(changeset, _), do: validate_required(changeset, @required)
end
5

I can take a closer look at this later and would help too if you can share the form.

If you make it one less level deep you wouldn’t need put_assoc, you can just |> cast_assoc(:panels) using the panel’s changeset. This would just be for the form itself. This assumes a panel would have a panel_group_id when adding it to the order/configuration. Again, this would only be for the form, you can still have the panel_group association which you can preload along with their panels and then all the panels would be in their proper groups.

I’m saying this based off the information that you are trying to manually sort them into groups.

6

I tinkered a bit around with the idea you had of having the panels one levels less deep via has_many :panels, through: panel_groups on my Configuration. This seems promising! But then i realize it would not work since :through associations are read-only

From the ecto docs:

Note :through associations are read-only. For example, you cannot use Ecto.Changeset.cast_assoc/3 to modify through associations.

My form is quite big so i can’t really show it here but i will show a trimmed down version that hopefully get the big picture.

First let me explain what i’m trying to do.

I have a sidebar with a long vertical form in it with multiple sections for the order. There is a “Panel” section where the user choose the number of panel and then choose the dimensions and finish and other properties for each panel. At this point, there is no concept of group. The user is just configuring panel individually.

Let say he choose 4 panel. Then, on the right part of the screen he will see 4 empty panels with the right dimensions.

[ empty ] [ empty ] [ empty ] [ empty ]

In reality here we have 4 panel group with each 1 panel inside.

Then the user can select 1 or more panels and assign an image to it. This is where the group come in play. The image is attached to the group not the panel. Let say the user select the two first panel and assign an image to it. Now the interface look something like this,

[ image spanning two panels] [empty] [empty]

Again in reality we have now 1 group with two panels and 2 groups with each 1 panel.

I will admit, this form is more complex than anything i had to do form wise before and got me quite struggling. I did some weird workaround to make it work regarding CSS.

<.form for={@form} phx-submit="submit_order" phx-change="validate_order">
  <%!-- SIDEBAR --%> 
  # Bunch of other sections for the order...

  # The user can choose between a predefined configuration or a custom one.
  <.input type="select" field={@form[:configuration_type]} options={configuration_type_options()} />
  
  # Here is the part for the panels. The user is indeed not aware of groups of panel he just configure each
  # panels. Their dimensions and other properties..
  <%= if Changeset.get_field(@form.source, :configuration_type) == :custom do %>
    <.inputs_for :let={custom_configuration_form} field={@form[:order_custom_configuration]}>
      <.input type="select" field={custom_configuration_form[:panels_count]} phx-change="update_panels" />
      <.input type="number" field={custom_configuration_form[:panels_height]} />
      
      <.inputs_for :let={panel_group_form} field={custom_configuration_form[:panel_groups]}>
        <.input type="hidden" field={panel_group_form[:image_id]} value={Changeset.get_field(panel_group_form.source, :image_id)} />
        
        <.inputs_for :let={panel_form} field={panel_group_form[:panels]}>
          <.input type="hidden" field={panel_form[:index]} value={Changeset.get_field(panel_form.source, :index)} />
          <.input type="number" field={panel_form[:width]} />
        </.inputs_for>
      </.inputs_for>
    </.inputs_for>
  <% end %>
  
  # Bunch of other sections for the order 
  <%!-- SIDEBAR END --%>

  <%!-- PANEL CONFIGURATOR --%>
  <%= if Changeset.get_field(@form.source, :configuration_type) == :custom do %>
    
    # I have a hook that is reponsible for handling the positioning of the image in the panel group.
    <div phx-hook="orderPanelConfigurator" id="panel_configurator">
      <% panels_height = get_panels_height(@form) %>
      
      <div class="grid">
        <%= for {group, index} <- Enum.with_index(get_panel_groups(@form))do %>
          <% panels = Changeset.get_assoc(group, :panels) %>
          <% image_id = Changeset.get_field(group, :image_id) %>

          <div class="panel-group" style={get_panel_group_style(group, @form)} data-group-index={index}>
            <%= for panel <- panels do %>
              <% panel_width = get_panel_width(panel) %>
              <% index = Changeset.get_field(panel, :index) %>

              # Image is displayed via background-image url
              <div
                class={get_panel_class(@selected_panels, panel, group)}
                style={"flex: #{panel_width}; aspect-ratio: #{panel_width}/#{panels_height};"}
                phx-click="select_panel"
                phx-value-index={if image_id, do: -1, else: index}
              >
                # Stuff inside here that is not important.
              </div>
            <% end %>
          </div>
        <% end %>
      </div>
    </div>
  <% end %>
  <%!-- PANEL CONFIGURATOR --%>
</.form>

To me, my data structure make sense. I tried other approach of having the panels directly on the configuration. Then they could maybe have a group, maybe not. But having them always in a group and having the image on the group is what made more sense in the end.

My struggle is with manipulating the groups in the context of the form really. How can i move the panel around, changing in which group they belong without using put_assoc multiple times? What would be the right tool here? I want to keep the validation on the panels so that when the user enters an invalid number for the width in the sidebar for example, the UI reflect the error and set the input red.

7

Ah crap, I totally forgot you can’t manipulate through associations. Sorry about that!

I’m confused by this part:

At this point, there is no concept of group.

But then you say:

The image is attached to the group not the panel.

Is this a completely different step or do you mean that there is no concept as far as the user is concerned at this point? Assuming it is just from the user’s perspective, I would still build a group any time a panel is added using inputs_for as normal, even if it’s not visible. But yes, sorry if that isn’t what you’re talking about.

As for not being able to call put_assoc multiple times, it looks like you are trying to manipulating the same changeset after it’s been validated. The handle_event that is doing the updates should be constructing a brand new changeset from params. Can you share what you have there?

As for updating nested changeset, it’s just generally a pain. This library may be of interest:

Lastly, for a bit of nitpicky advice, your order changeset function would be a bit more robust if you didn’t use Map.get on params and instead do this:

  def changeset(order, attrs) do
    changeset = cast(order, attrs, @required)
    status = get_change(changeset, :status)

    changeset
    |> maybe_validate_required(status)
    |> ...

Sorry if this still isn’t super helpful, it is quite complicated and hard to tell without seeing everything!

1 Like
8

Sorry that was unclear. Yes, i meant that at this point from the user perspective in the UI there is no concept of group but the there is in the data structure.

This opened my eyes. I wasn’t working with the form params for the grouping operation. I was only working with the changeset ( source ) of the form. Updating the data with put_assoc. I changed my approach and now i’m only working with the form params. Accessing them via socket.assigns.form.params, updating them manually when i group/ungroup panels and not touching the changeset at all. This seems to work like a charm. The only problem i have is that, on the first mount, form.params is empty until a phx-change is triggered at least once. I worked around that by having a hidden input with a js hook that trigger a change on mount. A bit sketchy but seems to work just fine.

9

Haven’t read through the whole thread yet, but FWIW this is precisely what phoenix_ecto does in to_form: