JSON-encode modified Ecto Schema

[svilen]

As far as I know, the recommended (and easiest) way to JSON-encode your Ecto Schema in Phoenix is using the Poison.Encoder like so:

defmodule MyApp.Web.User do
  use MyApp.Web, :model
  @derive {Poison.Encoder, only: [:first_name, :last_name, :email]}
  
  schema "users" do
    field :first_name, :string
    field :last_name, :string
    field :email, :string
    # ...
    timestamps()
  end
end

That works great as it allows me to simply return the User schema in a show.json Phoenix.View, for example, and then Poison would pick it up and serialize it.

However, I’m not sure what’s the best way to go if you want to do something extra, before returning the schema from the View.

Let’s say the user has a last_login field that I’d like to convert from UTC to a specific timezone. The actual timezone string identifier is not part of the User schema, so it needs to be fetched separately. Also I would like to add a field that’s not part of the original User schema. Right now, I’m constructing a new Map and returning it from the Phoenix.View, like so:

defmodule MyApp.Web.UserView do
  use MyApp.Web, :view

  def render("show.json", %{....}) do
    # ...
    tz = organization.timezone
    %{
      first_name: user.first_name,
      last_name: user.last_name,
      org_name: organization.name,
      last_login: Timex.Timezone.convert(user.last_login, tz)
    }
  end
end

That seems to work, but I’m not very keen on this approach as it is a lot of duplicated code; also if I want to return a list of 1000 users, this will construct 1000 maps just for encoding, which is not great.

Is there a better way to do this?

P.S. Forum moderators: I’m not sure if this topic belongs to the Phoenix, Ecto or a different category, so feel free to move it if it makes sense!

[wmnnd]

You could make use of Ecto’s virtual fields in order to add this key to your Schema struct and then have it encoded without creating a whole new Map.
Virtual fields on Ecto Schemas can be set and read but Ecto will not try to persist them to your database.

And since Ecto Schemas are really just normal Elixir structs, you can also simply update the value of your last_login field with the formatted string without it having any impact on your stored data.

[OvermindDL1]

Or if you really only want it in the json and not elsewhere then just make your own poison protocol to serialize out your schema. It is very simple to do and you can see how to do it via the docs or source.

[benwilson512]

@svilen

if I want to return a list of 1000 users, this will construct 1000 maps just for encoding, which is not great.

Why is this not great? Creating 1000 maps is still orders of magnitude faster than hitting the database to get 1000 users in the first place.

[svilen]

Thanks @wmnnd and @OvermindDL1, great advice! I totally forgot about virtual fields, that would be a clever way to make use of them; I wasn’t aware of Poison’s Encoder protocol at all. I found this article which has a short example that might be helpful to others:

JSON Serialisation of Ecto Models in Phoenix Channels (and views)

@benwilson512 I think I didn’t phrase this correctly. I’m always hitting the database, so what I meant was that instead of serializing the results coming from the DB, I was doing another round of operations to create the Maps and then serialize them. I’m not sure about the performance implications, but less operations == better, in my mind at least.

[benwilson512]

@svilen

I’m not sure about the performance implications, but less operations == better, in my mind at least.

Premature optimization comes with its own set of issues though. Suppose you go for the poison encoder route, to avoid creating some intermediate data. You’ve now tied your model to one very specific representation of JSON. What if you want an additional API endpoint where you serialize users differently? You’re out of luck, there can only be one implementation of a protocol per struct. There’s more on this in a nice blog post here: JSON Views in Phoenix | RokkinCat

Functional programming generally produces a lot of intermediate data, but since this is known, the BEAM (and other functional platforms) are built around handling this well. Immutable data means that very little actually needs to be copied, because you can point to a lot of the same data you were already pointing too since you know it won’t ever change. Garbage collectors are written to efficiently handle this kind of workload, and so on.

At the end of the day, you don’t want to make architectural decisions about how you relate your view layer to your modeling layer on the basis of what might gain you a couple of microseconds within a request that is already at a minimum gonna be several milliseconds.

[svilen]

@benwilson512

What if you want an additional API endpoint where you serialize users differently?

“What if” — sounds like swapping premature optimization for overengineering

I think you’ve raised some valid points, but everyone’s use case is different. I don’t envisage rendering models in different ways — I intend to keep the JSON representation of my models consistent — so implementing the Encoder protocol would be perfectly fine. If I do have an edge case where I need to render a “lightweight” version of a model, then I can always fall back to constructing and returning a map from the View.