Fl4m3Ph03n1x

Fl4m3Ph03n1x

Background

I am trying to encode a structure into json format using the Jason library. However, this is not working as expected.

Code

Let’s assume I have this struct:

defmodule Test do
   defstruct [:foo, :bar, :baz]
end

And that when using Jason.enconde(%Test{foo: 1, bar: 2, baz:3 }) I want this json to be created:

%{"foo" => 1, "banana" => 5}

Error

It is my understanding that to achieve this I need to implement the Jason.Enconder protocol in my struct:

defmodule Test do
   defstruct [:foo, :bar, :baz]
   
   defimpl Jason.Encoder do
      @impl Jason.Encoder 
      def encode(value, opts) do
         Jason.Encode.map(%{foo: Map.get(value, :foo), banana: Map.get(value, :bar, 0) + Map.get(value, :baz, 0)}, opts)
      end
   end
end

However, this will not work:

Jason.encode(%Test{foo: 1, bar: 2, baz: 3})
{:error,
 %Protocol.UndefinedError{
   description: "Jason.Encoder protocol must always be explicitly implemented.\n\nIf you own the struct, you can derive the implementation specifying which fields should be encoded to JSON:\n\n    @derive {Jason.Encoder, only: [....]}\n    defstruct ...\n\nIt is also possible to encode all fields, although this should be used carefully to avoid accidentally leaking private information when new fields are added:\n\n    @derive Jason.Encoder\n    defstruct ...\n\nFinally, if you don't own the struct you want to encode to JSON, you may use Protocol.derive/3 placed outside of any module:\n\n    Protocol.derive(Jason.Encoder, NameOfTheStruct, only: [...])\n    Protocol.derive(Jason.Encoder, NameOfTheStruct)\n",
   protocol: Jason.Encoder,
   value: %Test{bar: 2, baz: 3, foo: 1}
 }}

From what I understand, it looks like I can only select/exclude keys to serialize, I cannot transform/add new keys.
Since I own the structure in question, using Protocol.derive is not necessary.

However I fail to understand how I can leverage the Jason.Encoder protocol to achieve what I want.

Questions

  1. Is my objective possible using the Jason library, or is this a limitation?
  2. Am I miss understanding the documentation and doing something incorrect?

Showing Posts 14 to 5

belgoros

belgoros

Pff, my bad, I named my module in a wrong way, sorry for confusing. It had to be:

defmodule ElixirDraft.Json.JsonReader do
	alias ElixirDraft.Json.User
	
	def to_json(%User{} = user) do
		Jason.encode!(user)
	end	
	
	def to_model(json) do
		Jason.decode!(json)
	end
end

No warnings, it worked as expected :).

belgoros

belgoros

I have the same issue. If I get it right from the Consolidation docs, adding this to the mix.exs file would have fixed the warning:

def project do
  ...
  elixirc_paths: elixirc_paths(Mix.env())
  ...
end

defp elixirc_paths(:test), do: ["lib", "test/support"]
defp elixirc_paths(_), do: ["lib"]

But I still have it.
Here is how I defined my module and marked the fields to encode:

defmodule Draft.Json.User do
	
	@derive {Jason.Encoder, only: [:name, :email, :age]}
	defstruct [:name, :email, :age]
end

And here is a custom encoder module:

defmodule Draft.Json.JsonReader do
	alias Draft.Json.User
	
	def to_json(%User{} = user) do
		Jason.encode!(user)
	end	
	
	def to_model(json) do
		Jason.decode!(json)
	end
end

When starting an IEx session, the warning pooped up as before:

Erlang/OTP 27 [erts-15.0] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit]

    warning: the Jason.Encoder protocol has already been consolidated, an implementation for Draft.Json.User has no effect. If you want to implement protocols after compilation or during tests, check the "Consolidation" section in the Protocol module documentation
    │
  4 │ 	defstruct [:name, :email, :age]
    │         ~~~~~~~~~~~~~~~~~~~~~~~~
    │
    └─ lib/json/user.ex:4: Draft.Json.User (module)

What am I missing?

Fl4m3Ph03n1x

Fl4m3Ph03n1x OP

Thank you!

kartheek

kartheek

Docs state advantage of protocol consolidation as below:

Consolidation directly links protocols to their implementations in a way that invoking a function from a consolidated protocol is equivalent to invoking two remote functions.

For solving the problem in original post - you have to create the module and encoder in a file say test.ex in lib folder and it will work.

Fl4m3Ph03n1x

Fl4m3Ph03n1x OP

@kartheek great answer, and I very much enjoyed the link to the book.

One last thing though, just to confirm, the only reason for not doing option 1, is because test performance can be affected, correct?

As for the proposed solution, how would I go about it?

def project do
  ...
  elixirc_paths: elixirc_paths(Mix.env())
  ...
end

defp elixirc_paths(:test), do: ["lib", "test/support"]
defp elixirc_paths(_), do: ["lib"]

I understand I need to have a "test/support/something.ex" file, but it is not clear to me what the contents of this file would be.

kartheek

kartheek

You can read about Protocol Consolidation.

Protocol consolidation happens at compile time. You can disable or enable from mix config as mentioned in docs.

When it is enabled (by default) - it will link protocols with their implementations at compile time to optimise invocations. So when you are trying it in iex - it has no effect as protocol consolidation already happened.

Two solutions:

  • disable protocol consolidation using mix config (not recommended) - Section 16.6 don’t go by title read the content in linked section in this book - https://www.elixircryptobot.com/.
  • define all these encoders in files and compile the project (recommended)

No problem with the code or library or project - its working as it is supposed to be.

Fl4m3Ph03n1x

Fl4m3Ph03n1x OP

@kartheek and @thomasbrus
I am trying this directly in iex, yes. I read the comment, but I don’t quite understand what it means.
I also don’t quite understand how this affects the code I am writing. If someone could explain, I would be thankful.

thomasbrus

thomasbrus

Did you see this warning by any chance?

warning: the Jason.Encoder protocol has already been consolidated, an implementation for Test has no effect. If you want to implement protocols after compilation or during tests, check the “Consolidation” section in the Protocol module documentation
dummy.exs:12: Test (module)

(meant to reply to @Fl4m3Ph03n1x)

sneako

sneako

I stand corrected, thanks @LostKobrakai !

LostKobrakai

LostKobrakai

That’s not required if the implementation is nested within the module the implementation is for.

Where Next? Top

Trending in Questions Top

RSP87
I’m working on a project that simulates the bumbl example in the programming phoenix book. It acts almost like an email client. We have a...
New
nseaSeb
Hello, I know there is an approach for handling lists that allows for optimized traversal, but I can’t recall the specific method (somet...
New
RemyXRenard
I’m seeing that a list inside a Kino.DataTable will be interpreted as a charlist, even if the Kino.configure() is set to charlists: :as_l...
New
velrest
So my question is quite simple and i have found no conclusive answer on forum, google or AI. Should we use :erlang.float for Integer to ...
New
brecabral
Documentation While reading the Scoped Routes section, I noticed that the documentation currently refers to a problem without explainin...
New
samoloth
Hi, I’ve just set up an application with ash_authentication. There is only magic link strategy for now, so there is no confirmation add o...
New
FlyingNoodle
If a change or preparation module uses Ash.Changeset.get_argument/2 or Ash.Query.get_argument/2 (or any of the other get_argument functio...
New

Other Trending Topics Top

mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
marciok
Hi there! We created Gust: A task orchestrator inspired by Airflow. For those who have never heard about Aiflow, it’s a Python-based wor...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Dmk
Xamal is a deployment tool for Elixir apps that deploys native releases to bare metal servers over SSH. It’s a port of GitHub - basecamp/...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
webofbits
With AI doing more of the implementation work, I’ve been wondering how much coding I should deliberately keep doing myself. My main conc...
#ai
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews