How to convert multi struct in a list to Json in phoenix controller

Hello, I have a struct that was made with preload, this output cannot be converted to json encode.
it should be noted, I can’t use @derive in my schema files, but I use it in my query file, but the task was broken , because I have multi struct in a list.

now how can I convert it to json, I’m using Jason lib,

my preload:

def show_support_with_code(code, user_id) do
      query = from u in SupportSchema,
          where: u.code == ^code,
          join: a in assoc(u, :users),
          # where: a.id == ^user_id,
          left_join: b in assoc(u, :blog_posts),
          left_join: c in assoc(u, :support_replies),
          left_join: d in assoc(c, :users),
          order_by: [desc: c.inserted_at],
          preload: [users: a, blog_posts: b, support_replies: {c, users: d}]
      
      case Repo.one(query) do
         nil -> {:error, :show_support_with_code}
         info -> {:ok, :show_support_with_code, info}
      end
    end

output preload:

%BankError.ClientApi.Support.SupportSchema{
   __meta__: #Ecto.Schema.Metadata<:loaded, "supports">,
   blog_posts: nil,
   code: "test47",
   description: "بررسی بهینه سازی در اگهی",
   id: "cd33f63f-44c5-46a7-848c-8926be21dd82",
   inserted_at: ~N[2019-08-25 17:43:49],
   post_id: nil,
   status: 3,
   support_replies: [
     %BankError.Support.SupportReplySchema{
       __meta__: #Ecto.Schema.Metadata<:loaded, "support_replies">,
       description: "برای تست اینجاییم",
       id: "e67dae2f-237c-439f-bee7-846f8f575cc9",
       inserted_at: ~N[2019-08-29 17:24:25],
       status: true,
       support_id: "cd33f63f-44c5-46a7-848c-8926be21dd82",
       supports: #Ecto.Association.NotLoaded<association :supports is not loaded>,
       updated_at: ~N[2019-08-29 17:24:26],
       user_id: "40c06a6b-15a0-4888-999b-132fbd326455",
       users: %BankError.Users.UserSchema{
         __meta__: #Ecto.Schema.Metadata<:loaded, "users">,
         id: "40c06a6b-15a0-4888-999b-132fbd326455",
         inserted_at: ~N[2019-05-26 08:12:14],
         last_name: "soam",
         mobile: "09213780329",
         name: "shiam",
         password: nil,
         password_hash: "$2b$12$N2c6fKCVez7ZB/5k7coXkuwNaVGESQboFNba8cmRcRqoaKufs4PAC",
         role: 1,
         status: 1,
         support_replies: #Ecto.Association.NotLoaded<association :support_replies is not loaded>,
         supports: #Ecto.Association.NotLoaded<association :supports is not loaded>,
         transactions: #Ecto.Association.NotLoaded<association :transactions is not loaded>,
         updated_at: ~N[2019-05-26 08:12:14]
       }
     }
   ],
   title: "ساختمان سازی یک تست دوم",
   type: "other",
   updated_at: ~N[2019-08-26 06:29:23],
   user_id: "d0c1c8c5-6747-47c7-ba21-89a15017be3c",
   users: %BankError.Users.UserSchema{
     __meta__: #Ecto.Schema.Metadata<:loaded, "users">,
     id: "d0c1c8c5-6747-47c7-ba21-89a15017be3c",
     inserted_at: ~N[2019-04-16 08:20:10],
     last_name: "tavakkoli",
     mobile: "09368094936",
     name: "shahryar",
     password: nil,
     password_hash: "$2b$12$k/Ql7o9VyUbeeHi3Dr1BuOub2Wdg6NZ0I8bIfM8SmaqNop8o60H.G",
     role: 3,
     status: 2,
     support_replies: #Ecto.Association.NotLoaded<association :support_replies is not loaded>,
     supports: #Ecto.Association.NotLoaded<association :supports is not loaded>,
     transactions: #Ecto.Association.NotLoaded<association :transactions is not loaded>,
     updated_at: ~N[2019-08-26 05:38:18]
   }
 }

I tested it

   def test1() do
      [
         %BankError.Support.SupportReplySchema{
           __meta__: "s",
           description: "برای تست اینجاییم",
           id: "e67dae2f-237c-439f-bee7-846f8f575cc9",
           inserted_at: ~N[2019-08-29 17:24:25],
           status: true,
           support_id: "cd33f63f-44c5-46a7-848c-8926be21dd82",
           supports: "#Ecto.Association.NotLoaded<association :supports is not loaded>",
           updated_at: ~N[2019-08-29 17:24:26],
           user_id: "40c06a6b-15a0-4888-999b-132fbd326455",
           users: %BankError.Users.UserSchema{
             __meta__: "",
             id: "40c06a6b-15a0-4888-999b-132fbd326455",
             inserted_at: ~N[2019-05-26 08:12:14],
             last_name: "soam",
             mobile: "09213780329",
             name: "shiam",
             password: nil,
             password_hash: "$2b$12$N2c6fKCVez7ZB/5k7coXkuwNaVGESQboFNba8cmRcRqoaKufs4PAC",
             role: 1,
             status: 1,
             support_replies: "#Ecto.Association.NotLoaded<association :support_replies is not loaded>",
             supports: "#Ecto.Association.NotLoaded<association :supports is not loaded>",
             transactions: "#Ecto.Association.NotLoaded<association :transactions is not loaded>",
             updated_at: ~N[2019-05-26 08:12:14]
           }
         }
       ]
       |> test
   end
   def test(struct) do
      struct
      |> Enum.map(fn %{users: users} = x -> 
         x
         |> Map.delete(:__struct__) # change the struct to a Map
         |> Map.drop([:__meta__])
         |> Enum.filter(fn {c, v} -> v end)
         |> Enum.into(%{})
      end)   
   end

but the output is:

I can’t be able to delete %BankError.Users.UserSchema and some field like __meta__ and etc in users map

help me please

@peerreynders @OvermindDL1 @NobbZ

1 Like

Have you tried a simple

Map.from_struct

https://hexdocs.pm/elixir/Map.html#from_struct/1

2 Likes

I couldn’t use it, I cant delete all the field I need like (password_hash , support_replies, __meta__, and etc.)
I mean a loop should be created and find these keys and delete, but it just works in a single tuple like %{name: 1, last_name: 2} not a list like my first topic

I can’t use @derive in my schema files

Why?

1 Like

it just works in a single tuple like %{name: 1, last_name: 2} not a list like my first topic

Sorry, I totally missed the part where you had a list in the struct :slight_smile:

At first, the struct created was used in many files and then I didint want to change my files I hadn’t written test, but I test @derive in my schema file, and it didnt work for me I dont know why ?!

like:

defmodule BankError.ClientApi.Support.SupportSchema do
   use Ecto.Schema

   import Ecto.Changeset
   @primary_key {:id, :binary_id, autogenerate: true}
   @foreign_key_type :binary_id

   @derive {Jason.Encoder, only: [:title, :description, :type, :status, :code]}
   
   schema "supports" do

      field :title, :string, size: 200, null: false
      field :description, :string, null: false
      field :type, :string, size: 100, null: false
      field :status, :integer, null: false
      field :code, :string, size: 100, null: false
  
      belongs_to :users, BankError.Users.UserSchema, foreign_key: :user_id, type: :binary_id
      belongs_to :blog_posts, BankError.Blog.BlogSchema, foreign_key: :post_id, type: :binary_id
      has_many :support_replies, BankError.Support.SupportReplySchema, foreign_key: :support_id, on_delete: :nothing

      timestamps()
    end

   @all_fields ~w(title description type status code user_id post_id)a
   @all_required ~w(title description type status code user_id)a

   def changeset(struct, params \\ %{}) do
      struct
      |> cast(params, @all_fields)
      |> validate_required(@all_required, message: "فیلد مذکور نمی تواند خالی باشد.")
      |> validate_length(:title, max: 200, message: "حداکثر ۲۰۰ کاراکتر")
      |> validate_length(:type, max: 100, message: "حداکثر ۱۰۰ کاراکتر")
      |> validate_length(:code, max: 100, message: "حداکثر ۱۰۰ کاراکتر")
      |> validate_inclusion(:status, 1..4)
      |> foreign_key_constraint(:post_id)
      |> foreign_key_constraint(:user_id)
      |> unique_constraint(:code, name: :unique_index_on_support_code, message: "کد پیگیری تکراری می باشد.")
   end
end

I added @derive in other schema files like ( support_replies, users)

def show_support_with_code(code, user_id) do
      query = from u in SupportSchema,
          where: u.code == ^code,
          join: a in assoc(u, :users),
          # where: a.id == ^user_id,
          left_join: b in assoc(u, :blog_posts),
          left_join: c in assoc(u, :support_replies),
          left_join: d in assoc(c, :users),
          order_by: [desc: c.inserted_at],
          preload: [users: a, blog_posts: b, support_replies: {c, users: d}]
      
      case Repo.one(query) do
         nil -> {:error, :show_support_with_code}
         info -> {:ok, :show_support_with_code, info}
      end
    end

But it didn’t change

Maybe a dumb suggestion, but I would make sure your project really is using Jason and not Poison.

From the Jason docs you can also place the derive outside any module.

Finally, if you don’t own the struct you want to encode to JSON, you may use Protocol.derive/3 placed outside of any module:

Protocol.derive(Jason.Encoder, NameOfTheStruct, only: [...])

I didn’t install Poison in my project I think phoenix 1.4 uses Jason default.

I saw this Protocol.derive(Jason.Encoder, NameOfTheStruct, only: [...]) , but don’t know where I can use it, because I never have used Protocol before, but am reading the documents

Thank you

Hope that helps! Also, make sure when you add the @derive that your project recompiles.

1 Like

I used it in my controller.

require Protocol
Protocol.derive(Jason.Encoder, BankError.ClientApi.Support.SupportSchema, only: [:title, :description, :type, :status, :code, :support_replies])

and I have this error:

[error] #PID<0.1061.0> running BankErrorWeb.Endpoint (connection #PID<0.1060.0>, stream id 1) terminated
Server: localhost:4000 (http)
Request: POST /api/v1/support
** (exit) an exception was raised:
    ** (Protocol.UndefinedError) protocol Jason.Encoder not implemented for %BankError.Support.SupportReplySchema{__meta__: #Ecto.Schema.Metadata<:loaded, "support_replies">, description: "برای تست اینجاییم", id: "e67dae2f-237c-439f-bee7-846f8f575cc9", inserted_at: ~N[2019-08-29 17:24:25], status: true, support_id: "cd33f63f-44c5-46a7-848c-8926be21dd82", supports: #Ecto.Association.NotLoaded<association :supports is not loaded>, updated_at: ~N[2019-08-29 17:24:26], user_id: "40c06a6b-15a0-4888-999b-132fbd326455", users: %BankError.Users.UserSchema{__meta__: #Ecto.Schema.Metadata<:loaded, "users">, id: "40c06a6b-15a0-4888-999b-132fbd326455", inserted_at: ~N[2019-05-26 08:12:14], last_name: "soam", mobile: "09213780329", name: "shiam", password: nil, password_hash: "$2b$12$N2c6fKCVez7ZB/5k7coXkuwNaVGESQboFNba8cmRcRqoaKufs4PAC", role: 1, status: 1, subscribers: #Ecto.Association.NotLoaded<association :subscribers is not loaded>, support_replies: #Ecto.Association.NotLoaded<association :support_replies is not loaded>, supports: #Ecto.Association.NotLoaded<association :supports is not loaded>, transactions: #Ecto.Association.NotLoaded<association :transactions is not loaded>, updated_at: ~N[2019-05-26 08:12:14]}}, Jason.Encoder protocol must always be explicitly implemented.

If you own the struct, you can derive the implementation specifying which fields should be encoded to JSON:

    @derive {Jason.Encoder, only: [....]}
    defstruct ...

It is also possible to encode all fields, although this should be used carefully to avoid accidentally leaking private information when new fields are added:

    @derive Jason.Encoder
    defstruct ...

Finally, if you don't own the struct you want to encode to JSON, you may use Protocol.derive/3 placed outside of any module:

    Protocol.derive(Jason.Encoder, NameOfTheStruct, only: [...])
    Protocol.derive(Jason.Encoder, NameOfTheStruct)
. This protocol is implemented for: BankError.ClientApi.Support.SupportSchema, Ecto.Association.NotLoaded, Ecto.Schema.Metadata, Date, BitString, Jason.Fragment, Any, Map, NaiveDateTime, List, Integer, Time, DateTime, Decimal, Atom, Float
        (jason) lib/jason.ex:199: Jason.encode_to_iodata!/2
        (phoenix) lib/phoenix/controller.ex:279: Phoenix.Controller.json/2
        (bank_error) lib/bank_error_web/controllers/api_support_controller.ex:1: BankErrorWeb.ApiSupportController.action/2
        (bank_error) lib/bank_error_web/controllers/api_support_controller.ex:1: BankErrorWeb.ApiSupportController.phoenix_controller_pipeline/2
        (bank_error) lib/bank_error_web/endpoint.ex:1: BankErrorWeb.Endpoint.instrument/4
        (phoenix) lib/phoenix/router.ex:275: Phoenix.Router.__call__/1
        (bank_error) lib/bank_error_web/endpoint.ex:1: BankErrorWeb.Endpoint.plug_builder_call/2
        (bank_error) lib/plug/debugger.ex:122: BankErrorWeb.Endpoint."call (overridable 3)"/2
        (bank_error) lib/bank_error_web/endpoint.ex:1: BankErrorWeb.Endpoint.call/2
        (phoenix) lib/phoenix/endpoint/cowboy2_handler.ex:33: Phoenix.Endpoint.Cowboy2Handler.init/2
        (cowboy) /Applications/MAMP/htdocs/elixir-ex-source/bank_error/deps/cowboy/src/cowboy_handler.erl:41: :cowboy_handler.execute/2
        (cowboy) /Applications/MAMP/htdocs/elixir-ex-source/bank_error/deps/cowboy/src/cowboy_stream_h.erl:296: :cowboy_stream_h.execute/3
        (cowboy) /Applications/MAMP/htdocs/elixir-ex-source/bank_error/deps/cowboy/src/cowboy_stream_h.erl:274: :cowboy_stream_h.request_process/3
        (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

but if I change it to :

require Protocol
Protocol.derive(Jason.Encoder, BankError.ClientApi.Support.SupportSchema, only: [:title, :description, :type, :status, :code])

I mean delete support_replies
it works but it doesn’t load other my tables that are called like

join: a in assoc(u, :users),
          where: a.id == ^user_id,
          left_join: b in assoc(u, :blog_posts),
          left_join: c in assoc(u, :support_replies),
          left_join: d in assoc(c, :users),

in my Preload query

my schema:

defmodule BankError.ClientApi.Support.SupportSchema do
   use Ecto.Schema

   import Ecto.Changeset
   @primary_key {:id, :binary_id, autogenerate: true}
   @foreign_key_type :binary_id

   @derive {Jason.Encoder, only: [:title, :description, :type, :status, :code]}
   
   schema "supports" do

      field :title, :string, size: 200, null: false
      field :description, :string, null: false
      field :type, :string, size: 100, null: false
      field :status, :integer, null: false
      field :code, :string, size: 100, null: false
  
      belongs_to :users, BankError.Users.UserSchema, foreign_key: :user_id, type: :binary_id
      belongs_to :blog_posts, BankError.Blog.BlogSchema, foreign_key: :post_id, type: :binary_id
      has_many :support_replies, BankError.Support.SupportReplySchema, foreign_key: :support_id, on_delete: :nothing

      timestamps()
    end

   @all_fields ~w(title description type status code user_id post_id)a
   @all_required ~w(title description type status code user_id)a

   def changeset(struct, params \\ %{}) do
      struct
      |> cast(params, @all_fields)
      |> validate_required(@all_required, message: "فیلد مذکور نمی تواند خالی باشد.")
      |> validate_length(:title, max: 200, message: "حداکثر ۲۰۰ کاراکتر")
      |> validate_length(:type, max: 100, message: "حداکثر ۱۰۰ کاراکتر")
      |> validate_length(:code, max: 100, message: "حداکثر ۱۰۰ کاراکتر")
      |> validate_inclusion(:status, 1..4)
      |> foreign_key_constraint(:post_id)
      |> foreign_key_constraint(:user_id)
      |> unique_constraint(:code, name: :unique_index_on_support_code, message: "کد پیگیری تکراری می باشد.")
   end
end

Hello again, I made some progress

I added this line in my controller (BankErrorWeb.ApiSupportController):

require Protocol
Protocol.derive(Jason.Encoder, BankError.ClientApi.Support.SupportSchema, only: [:title, :description, :type, :status, :code, :users, :blog_posts])

after adding top line code in my controller I added @derive {Jason.Encoder, only: [..]} into my schema files that were connected like, (users, blog_posts, support_replies)

it works but it has a problem, because I don’t input support_replies: {c, users: d} in my preload, I mean

Protocol.derive(Jason.Encoder, BankError.ClientApi.Support.SupportSchema, only: [:title, :description, :type, :status, :code, :users, :blog_posts, :support_replies])

if support_replies added !!! :frowning_face: it shows me this error:

** (exit) an exception was raised:
    ** (Protocol.UndefinedError) protocol Jason.Encoder not implemented for %BankError.Support.SupportReplySchema{__meta__: #Ecto.Schema.Metadata<:loaded, "support_replies">
   def show_support_with_code(code, user_id) do
     query = from u in SupportSchema,
         where: u.code == ^code,
         join: a in assoc(u, :users),
         where: a.id == ^user_id,
         left_join: b in assoc(u, :blog_posts),
         left_join: c in assoc(u, :support_replies),
         left_join: d in assoc(c, :users),
         order_by: [desc: c.inserted_at],
         preload: [users: a, blog_posts: b, support_replies: {c, users: d}]
     
     case Repo.one(query) do
        nil -> {:error, :show_support_with_code}
        info -> {:ok, :show_support_with_code, info}
     end
   end

update

it is fixed, I had a bad schema relation

Thanks

1 Like