Jason encode when associations not loaded

Phoenix Forum Questions / Help
, ,
1

Hello guys! I’m developing an API only Phoenix project for pet donations, and at the moment i have the following schemas:

  • User schema:
  schema "users" do
    field :name, :string
    field :email, :string
    field :bio, :string
    field :avatar, :string
    field :website, :string
    field :phone, :string
    field :social_links, {:array, :string}
    field :roles, {:array, Ecto.Enum}, values: [:ADOPTER, :DONOR]
    has_many :pets, Announcements.Pet, foreign_key: :owner_id

    timestamps(type: :utc_datetime)
  end
  • Pet schema:
  schema "pets" do
    field :name, :string
    field :size, :decimal
    field :bio, :string
    field :photos, {:array, :string}
    field :age, :integer
    field :gender, Ecto.Enum, values: [:MALE, :FEMALE]
    field :species, Ecto.Enum, values: [:DOG, :CAT]
    field :vaccinated, :boolean, default: false
    field :dewormed, :boolean, default: false
    field :neutered, :boolean, default: false
    field :disability, :boolean, default: false
    field :pedigree, :boolean, default: false
    field :weight, :decimal
    belongs_to :user, Accounts.User, foreign_key: :owner_id
    belongs_to :breed, Announcements.Breed, foreign_key: :breed_id

    timestamps(type: :utc_datetime)
  end
  • and the Breed schema:
  schema "breeds" do
    field :name, :string
    field :temperaments, {:array, :string}
    has_many :pets, Announcements.Pet, foreign_key: :breed_id

    timestamps(type: :utc_datetime)
  end

As you can see, the relations above are pretty simple actually, users can own many pets announces and therefore each pet announcement belongs to a user, each pet belongs to a breed, and this specific breed can own a lot announcements, so another many to one relationship.

The problem is around pets and breeds, when i insert a pet, breed or user everything looks fine, but when i trigger the show action on pets_controller the Jason complains that it’s unable to encode because of NotLoaded Association. I know there's questions similars and i know i could preloadthe breeds association with pets, but the weird fact for me is that Jason does not complain if i don't preload the:userassociation, and when i triggershowaction ofuser_controller` Jason does the encoding pretty well even with not loaded pet associations, so what i want to understand is why this error is happening with only one association.

2

I think you’ll need to show the code that is loading stuff from the database.

3

This sounds like the same kind of issue I answered in another post recently. Ecto `change/2` preloading associations? - #3 by 03juan

Basically when you build a new struct, the change function replaces NotLoaded with the respective “empty” value, nil for belongs_to and has_one or [] for has_many, specifically for things like forms so you don’t have to do it manually. But when you’re loading an existing record from the db you have to preload the association to replace the NotLoaded struct before you access the data.

Since you mention Jason I assume you’re setting up your own api controller. If you want to send the base struct without having to preload, the you can either @derive {Jason.Encoder, except: [:user, :breed]} above the schema "pets" declaration, which will always strip out those keys when serialising, or manually define your own protocol implementation so you can send the nested user and breed when they’re preloaded or exclude those keys when their values are NotLoaded.

See the defimpl Jason.Encoder example and let us know how it goes.

4

Just a regular Repo.get! function is enough to trigger the error

5

I’ve tried the derive on pets schema but had not successful at all, Jason still complained about the Not Loaded Association

6

I’m sorry, my code is actually incomplete, you also have to exclude the :__meta__ key or it fails with ** (RuntimeError) cannot encode metadata from the :__meta__ field. However it should have removed the other keys and not complain about NotLoaded.

Please post the code for your derivation and the output from an iex session showing the failure, or how you’re using the struct in your controller.

Can’t help any more than this without it…

Does Repo.get!(Announcements.Pet, pet_id) error in iex?

7

You likely have this somewhere in your controller(s)?

|> json(...)
8

It doesn’t because the problem is not with Ecto, just related, the problem is when Jason tries to encode it, at the moment my code snippet of pet schema is just like that:

  @derive {Jason.Encoder, except: [:user, :breed, :__meta__]}
  schema "pets" do
    field :name, :string
    field :size, :decimal
    field :bio, :string
    field :photos, {:array, :string}
    ....

Even with the @derive clause, Jason seems just to ignore, don’t know why, the error remains the same:

(RuntimeError) cannot encode association :breed from PontoCao.Announcements.Pet to JSON because the association was not loaded.

9

Don’t mate, the show action of pet_controller looks just like the phoenix auto generate:

def show(conn, %{"id" => id}) do
    pet = Announcements.get_pet!(id)
    render(conn, :show, pet: pet)
end

And the Announcements.get_pet!/1 is just a Repo.get!/1 call

10

First of all, ideally you should have a render file where you decide what shape the return will have, it should look the following (this one was generated using mix phx.gen.json Accounts User users name:string age:integer):

defmodule PhoenixTestWeb.UserJSON do
  alias PhoenixTest.Accounts.User

  @doc """
  Renders a list of users.
  """
  def index(%{users: users}) do
    %{data: for(user <- users, do: data(user))}
  end

  @doc """
  Renders a single user.
  """
  def show(%{user: user}) do
    %{data: data(user)}
  end

  defp data(%User{} = user) do
    %{
      id: user.id,
      name: user.name,
      age: user.age
    }
  end
end

There you can dictate what you render, it also decouples your controller from your schema.

As for error you are getting, you cannot render associations that are not preloaded. There are 2 options of doing that:

  1. Using Repo.preload/3: Ecto.Repo — Ecto v3.11.2
  2. Using preload from query: Ecto.Query — Ecto v3.11.2

Otherwise if you don’t need them, don’t render them.

2 Likes
11

Yes it’s because there is a NotLoaded value that you haven’t removed or replaced before Jason tries to encode it.

But this is very weird. What does your heex code look like?

12

OMG i totally forgot it, quite obvious actually, the data/1 function on pets was like this:

%{
      id: pet.id,
      name: pet.name,
      bio: pet.bio,
      photos: pet.photos,
      age: pet.age,
      gender: pet.gender,
      breed: pet.breed,
      species: pet.species,
      vaccinated: pet.vaccinated,
      dewormed: pet.dewormed,
      neutered: pet.neutered,
      disability: pet.disability,
      pedigree: pet.pedigree,
      size: pet.size,
      weight: pet.weight
    }

Removing this breed key and pet.breed value solves the error, and the reasons why the error was not happening with the :users association is because on data is not trying to render users, thanks for this comment bro :handshake:

13

@D4no0 helped me to solve this, was just a silly detail on PetsJSON module, thanks for answer though mate! :handshake:

1 Like
14

Hey no worries. Always helps to include as much code as possible for context from the beginning when getting errors.

1 Like
15

[old curmudgeon voice] That’s why I think Phoenix generates too many files…

16

Having some magic under the hood for rendering your ecto schemas is not a solution either.

This is the default behavior of generators, you can always move the fields you are rendering either to your schema file, controller or any other abstraction you find fit.

17

In this case I meant the “view” files btw.

18

In this configuration, view files are not present, the generated files after running phx.gen.json are:

.
└── app_web/
    └── controllers/
        ├── user_controller.ex
        └── user_json.ex

I agree that those view files were extremely annoying and I am happy that we got rid of them at least for the default configuration.

19

Don’t the *_json.ex and *_html.ex kind of do the same as the view files? I haven’t kept up with Phoenix 1.7.