Form inputs not mapping to the underlying entity

Phoenix Forum Questions / Help
, ,
1

Hello!

There is a fundamental question related to LiveView forms that I’ve been struggling with. In all the simple form tutorials, the form fields map one-to-one to the fields of the underlying data structure. But in my experience, that’s rarely the case.

Here’s a practical example I’m working on now. I am creating a database of books that also includes audiobooks. The length of a book is specified in the number of pages; the length of an audiobook is specified as hours and minutes, e.g. 12:37.

In the database, I store this as two fields: Integer that represents the number of pages or minutes, and an atom that specifies which unit is used (:pages or :minutes).

But on my form, I want to have a single text input where the user can type “256” or “12:37”.

This means that the structure backing my form must include a string field for the book length, but my DB data structure contains two fields: an integer and an atom.

In turn, this means that I must:

  • Translate structures loaded from the database to match what the form expects; and
  • Translate data entered by the user in the form to the form expected by the database.

Where should such translation take place? In the Context? In the form component?

More importantly, because forms are always backed by a struct, do I have to create a separate struct data type for my form (a struct that will contain length of type String)? Is this the expected solution? Or should I do something else, like create a custom type or a Phoenix component?

Thank you in advance,
Michal

2

One way to go is having changesets specifically for forms that then convert to the appropriate db schema, though that’s a bigger topic and something I never do personally.

A simpler way is to use a virtual field specifically for forms. In your case it could look roughly like this:

defmodule MyApp.Books.Book do
  use Ecto.Schema

  schema "books" do
    field :length, :integer
    field :length_type, Ecto.Enum, values: [:pages, :minutes]
    field :pages_or_minutes, :string, virtual: true

    # ...
  end

  def changeset(audiobook, attrs) do
    audiobook
    |> cast(attrs, [:pages_or_minutes, ...])
    |> change_length_attrs()
  end

  defp change_length_attrs(changeset) do
    pages_or_minutes = get_change(changeset, :page_or_minutes)

    if String.contains?(pages_or_minutes, ":") do
      changeset
      |> put_change(:length, convert_to_minutes(pages_or_minutes))
      |> put_change(:length_type, :minutes)
    else
      changeset
      |> put_change(:length, pages_or_minutes)
      |> put_change(:length_type, :pages)
    end
  end
end

And of course in your Books context, whenever you fetch or list books you will have to manually populate :pages_or_minutes to display in a form again. This is where Ash can really help, though it’s not so bad in plain ol’ Phoenix.

How you do it is up to you. The simpler way (at least to show on a forum) is to query then transform:

defp populate_virtual_fields(book) do
  case book.length_type do
    :minutes -> %{book | pages_or_minutes: convert_to_time(book.length)
    :pages -> %{book | pages_or_minutes: convert_to_pages(book.length)
  end
end

def get_book!(id) do
  Book
  |> Repo.get!(id)
  |> populate_virtual_fields()
end

def list_books do
  Book
  |> Repo.all()
  |> Enum.map(&populate_virtual_fields/1)
end

You could also do it in the database if you wanted to

Using the virtual field does tie the UI to the backend so YMMV there depending on the size of your project.

You can also do all these transformations by manipulating params but IMO this is rarely a good idea (though in a small app it could be ok). Doing it in changesets ensures that all the ways to manipulate (change) a resource are in one place. But forms don’t have to be backed by a struct. to_form(%{}) works and you could kinda do whatever you want in the web layer (I don’t have a lot of experience with this, though, which is why I didn’t start with that).

If you are interested having more separation of concerns, Saša Jurić’s Toward Maintainable Elixir: The Core and the Interface dives into it.

2 Likes
3

You are venturing down a dangerous path here. You should store this as three columns: :type (:audiobook and :book), :page_count, and :length_seconds. Then mix and match depending on the row.

If you wanted to denormalize further (context dependent) then you would break out into three tables (books, paper_books, and audiobooks) and use foreign keys to join them back. For a simple use case I would not bother doing this.

In your forms you can conditionally render inputs depending on the type of book. Simple case (or if) clauses in the template will suffice.

Overloading your columns with contextually separate data is a bad idea for many reasons. You will end up writing some pretty tortured queries down the line if you take that approach. I wouldn’t.

Apologies for not providing a specific answer to your question, but I think someone already has :slight_smile:

1 Like
4

FWIW, when it comes to your specific scenario I fully agree with @garrison, I was using your scenario to answer the more general question as I know you are learning!

1 Like
5

Hi guys,

This is all very interesting!

I have to say, though, that I am starting to get the feeling that LiveView is the kind of framework where my UI choices are going to be dictated by the (somewhat undocumented) capabilities of the framework instead of the framework enabling a variety of UI designs.

When the user creates a book, I want its type (audio or paper) to be specified by providing its length. I am trying to minimize the number of clicks and I want to avoid having a drop-down with “Audio” and “Paper” in it because it is redundant.

@sodapopcan - I love your idea of using the changeset for this. I will try it out!

Also, I know that I can make it work any which way - Phoenix is flexible. I already have a few forms in my project where I’m converting data back and forth etc. But I was wondering what the idiomatic way of handling such scenarios was because it seems that it would be extremely common that form fields don’t map 1x1 to the DB entity and that a translation layer is needed.

Thanks,
Michal

6

In general, when form fields don’t map exactly to the database entity, you can use an embedded schema that maps exactly to your UI for validation. You don’t persist the embedded schema but map it to your DB entity after validation and then persist that.

Here is a video showing this approach: Elixir Streams |> Phoenix forms backed by embedded schemas

And the previous tip shows how to do the same with a map instead of an embedded schema which is less explicit I guess: Elixir Streams |> Phoenix forms without changesets!

7

Just so we’re clear here, the things discussed on this thread are very documented. Make sure you spend some time with the Ecto guides and Changeset docs as well as they are very integrated with Phoenix’s form handling and you should have a good grasp of both for this sort of thing.

I think I see what you’re going for here. My intent was to warn you that the format you proposed to store your data, with a column overloaded to store semantically separate values, is a bad idea. This has little to do with Elixir, and more to do with how relational databases are designed. It’s also just kind-of confusing, as a programmer, to deal with.

However, how you store the data does not have to dictate the design of your form. There is no reason you can’t combine the two. You can use a virtual input to store the overloaded value and then parse it back out to the real database columns in your changeset.

Keep in mind that this is a simple example. I will leave parsing and validating the different inputs as an exercise for the reader :slight_smile:

defmodule Book do
  use Ecto.Schema
  import Ecto.Changeset
  schema "book" do
    field :type, Ecto.Enum, values: [:audio, :paper]
    field :pages, :integer
    field :length_seconds, :integer
    field :length_or_pages, :string, virtual: true
  end

  def changeset(book, params) do
    book
    |> cast(params, [:length_or_pages])
    |> apply_length_or_pages()
  end

  def apply_length_or_pages(changeset) do
    length_or_pages = get_change(changeset, :length_or_pages)
    if String.contains?(length_or_pages, ":") do
      changeset
      |> put_change(:type, :audio)
      |> put_change(:length_seconds, parse_hhmmss(length_or_pages))
    else
      changeset
      |> put_change(:type, :paper)
      |> put_change(:pages, String.to_integer(length_or_pages))
    end
  end
end
8

Funny, I hadn’t noticed this reply indeed also used a virtual field. So you see that I have in fact changed very little: all my example does, really, is split the length field into two.

9

Ha, it’s actually pretty validating seeing someone give essentially the same advice :sweat_smile:

1 Like
10

Thank you, guys! Virtual fields were the ticket.

For the record, I do have the book and audiobook lengths stored separately in the DB but I wanted to simplify the UI.

Regarding documentation, this is exactly what I could not find online: an in-depth presentation of how forms and Ecto go together. Most tutorials only show very simple cases. I’d like to understand how it all works behind the scenes. I should probably check how people build autocomplete inputs for things like tags - it could give me an idea.

Finally - but maybe I’ll create a separate discussion thread about it - I wonder what people do in cases like these: (1) forms that are not persisted in the database (e.g. search forms), or (2) situations when the data is stored in an API, not in a database.

Cheers,
Michal

11

Give this one a try: Data mapping and validation — Ecto v3.12.5