Elegant way to return JSON from the raw/source JSONB Postgres record?

Questions / Help
1

The goal is to return the following format

{"b": {"f1":"foo", "gibberish":"1"}}

given these two Ecto schemas:

defmodule A do
  use Ecto.Schema
  schema "a" do
    embeds_one :b, B
  end
end

defmodule B do
  use Ecto.Schema
  @primary_key false
  embedded_schema do
    field :f1, :binary
    field :f2, Ecto.Enum,
      values: [human_readable1: "1", human_readable2: "2"],
      source: :gibberish,
      embed_as: :dumped
  end
end

In other words, how to keep the (more readable) field :f2 and its human_readable1 etc. values in the codebase, while also be able to return the underlying JSONB Postgres record? In the example above, it contains a column :gibberish, with raw values "1" etc.

Iā€™ve come up with 2 approaches.
Approach 1, by (partially) implementing Access behaviour:

# First add Access implementation to both `A` and `B` module:
def fetch(struct, key) do
  {:ok, Map.get(struct, key)}
end

# Then in API controller:
def show(conn, _) do
  a = Repo.get(A, some_id)
  |> Map.from_struct()
  |> Map.drop([:__meta__, :id]))

  b = Map.get(a, :b)
  |> Map.from_struct()
  |> Map.drop([:f2])
  |> Map.put(:gibberish, Ecto.Enum.mappings(B, :f2)[get_in(a, [:b, :f2])])

  json(conn, %{ b: b })
end

Approach 2, by implementing Jason.Encoder protocol:

# First add Jason.Encoder implementation to `B` module:
defimpl Jason.Encoder do
  def encode(struct, opts) do
    Jason.Encode.map(
      struct
      |> Map.from_struct()
      |> Map.drop([:f2])
      |> Map.put(:gibberish, Ecto.Enum.mappings(B, :f2)[Map.get(struct, :f2)]),
      opts
    )
  end
end

# Then in API controller:
def show(conn, _) do
  json(conn, Repo.get(A, some_id)
  |> Map.from_struct()
  |> Map.drop([:__meta__, :id]))

  json(conn, a)
end

Is there a more elegant approach #3? Or is this as good as it gets? Iā€™d of course go for approach #1, since there can only be one implementation of a protocol per struct.

2

Approach #3 might be approach #2 combined with two additional modules, in ā€œparallelā€ to A and B (above). These modules are called A2 and B2 in the example below: A2 refers to B2, and B2 drops the use of the :source option:

defmodule A2 do
  use Ecto.Schema
  @derive Jason.Encoder
  schema "a" do
    embeds_one :b, B2
  end
end

defmodule B2 do
  use Ecto.Schema
  @derive Jason.Encoder
  @primary_key false
  embedded_schema do
    field :f1, :binary
    field :gibberish
  end
end

# Then in API controller:
def show(conn, _) do
  json(conn, Repo.get(A2, some_id)
  |> Map.from_struct()
  |> Map.drop([:__meta__, :id]))

  json(conn, a)
end

Curious if you would also consider this to be ā€˜the bestā€™ approach so far: I can keep using B and its more-readable fields throughout the codebase, and use B2 solely for the JSON API. Let me know if there are better approaches still.

Also, I might seem to be over-complicating, but in my actual usecase there are many more fields (:f3, etc.) in the "b" schema. And thus many potentially needed Ecto.Enum.mappings/2 calls, which would need to be ā€œmanuallyā€, in an error-prone way, kept in sync with the schema.

3

This still uses Phoenix.View, but the same is just as much possible with the new phoenix 1.7 approach to views.

4

Thanks for the response. I think this is the approach #1 in my OP ā€“ though unlike the separate JSON view as in the docs, I inline the json/2 call inside the controller (as Iā€™m not using HTML views).

My struggle with the above is that JSONB records, to which above B and B2 schemas point to (and which the API should return as they are/verbatim) have tens of such :gibberish fields and thus the above show/2 function should be expanded with multiple Map.put calls:

def show(conn, _) do
#...
b = Map.get(a, :b)
  |> Map.from_struct()
  |> Map.drop([:f2])
  |> Map.put(:gibberish, Ecto.Enum.mappings(B, :f2)[get_in(a, [:b, :f2])])
  # ... 
  |> Map.put(:gibberish100, Ecto.Enum.mappings(B, :f100)[get_in(a, [:b, :f100])])

  json(conn, %{ b: b })
end

Would these Postgres JSONB records fall under the exception (below) that you mention in your article?

Auxiliary structs, which mostly represent complex values like coordinates or Decimal structs still benefit from global Jason.Encoder implementations given thereā€™s hardly any use in encoding just parts of the data they hold. Those structs are more akin to a single value represented by a struct of details and less a container of multiple distinct pieces of data.

5

Views are in no way related to HTML. You can use the very same approach to render data to a map or after format encoding json. Thatā€™s the point I try to make in the article.

Tbh I donā€™t see much issue with that. If the expectation is to encode the runtime data using the same transformation as used when dumping to the db than that would be the explicit way to do it. You could automate it a bit more by using ectos schema reflection Ecto.Schema ā€” Ecto v3.11.2 and maybe a reduce over a long list of manual Map.puts, but I still would leave the transformation explicit.

I donā€™t think so. A Decimal is rarely though of as their individual struct values and really represents a single value in most domains. I donā€™t get the feeling the same applies to the schema/struct you have here.

1 Like
6

Iā€™ll go with the transformations from approach #1 ā€“ thanks for allaying my doubts. :slight_smile:

Here I merely meant to point out that I am already using Phoenix Views in my approach #1, albeit with omitting an explicit *JSON module (and thus using json/2 directly in the controller, rather than render). With ā€œHTML viewsā€ I meant the other ā€œrendering formatā€, to use Phoenix docsā€™ terminology:

Phoenix typically uses one view per rendering format. When rendering HTML, we would use UrlHTML. Now that we are rendering JSON, we will find a UrlJSON view [ā€¦]

I stand corrected: strictly following the docs, I am not using views, which are either UrlJSON or UrlHTML in the example provided by the docs.