How to have custom enconding for struct using Jason?

Questions / Help
, , ,
1

Background

I am trying to encode a structure into json format using the Jason library. However, this is not working as expected.

Code

Let’s assume I have this struct:

defmodule Test do
   defstruct [:foo, :bar, :baz]
end

And that when using Jason.enconde(%Test{foo: 1, bar: 2, baz:3 }) I want this json to be created:

%{"foo" => 1, "banana" => 5}

Error

It is my understanding that to achieve this I need to implement the Jason.Enconder protocol in my struct:
https://hexdocs.pm/jason/Jason.Encoder.html

defmodule Test do
   defstruct [:foo, :bar, :baz]
   
   defimpl Jason.Encoder do
      @impl Jason.Encoder 
      def encode(value, opts) do
         Jason.Encode.map(%{foo: Map.get(value, :foo), banana: Map.get(value, :bar, 0) + Map.get(value, :baz, 0)}, opts)
      end
   end
end

However, this will not work:

Jason.encode(%Test{foo: 1, bar: 2, baz: 3})
{:error,
 %Protocol.UndefinedError{
   description: "Jason.Encoder protocol must always be explicitly implemented.\n\nIf you own the struct, you can derive the implementation specifying which fields should be encoded to JSON:\n\n    @derive {Jason.Encoder, only: [....]}\n    defstruct ...\n\nIt is also possible to encode all fields, although this should be used carefully to avoid accidentally leaking private information when new fields are added:\n\n    @derive Jason.Encoder\n    defstruct ...\n\nFinally, if you don't own the struct you want to encode to JSON, you may use Protocol.derive/3 placed outside of any module:\n\n    Protocol.derive(Jason.Encoder, NameOfTheStruct, only: [...])\n    Protocol.derive(Jason.Encoder, NameOfTheStruct)\n",
   protocol: Jason.Encoder,
   value: %Test{bar: 2, baz: 3, foo: 1}
 }}

From what I understand, it looks like I can only select/exclude keys to serialize, I cannot transform/add new keys.
Since I own the structure in question, using Protocol.derive is not necessary.

However I fail to understand how I can leverage the Jason.Encoder protocol to achieve what I want.

Questions

  1. Is my objective possible using the Jason library, or is this a limitation?
  2. Am I miss understanding the documentation and doing something incorrect?
1 Like
2

I don’t see anything wrong with what you do to cause this error, but noticed that you have @impl Jason.Encoder, which you don’t need. @impl is related to behaviours not protocols.

1 Like
3

Even without @impl the error is the same and the behavior is the same.
I am just as confused as you :smiley:

4

Check if you got any warning like this during compilation:

warning: the Jason.Encoder protocol has already been consolidated, an implementation for Test has no 
effect. If you want to implement protocols after compilation or during tests, check the "Consolidation" 
section in the Protocol module documentation

If you are trying out these directly in iex - it will not work due to protocol consolidation.

iex(1)> t = %Test{foo: 1, bar: 2, baz: 3}
%Test{bar: 2, baz: 3, foo: 1}
iex(2)> Jason.encode!(t)
"{\"banana\":5,\"foo\":1}"
1 Like
5

You need to say which module this Jason.Encoder implementation is for. Note the for: Test

defimpl Jason.Encoder, for: Test do
  def encode(value, opts) do
    Jason.Encode.map(%{foo: Map.get(value, :foo), banana: Map.get(value, :bar, 0) + Map.get(value, :baz, 0)}, opts)
  end
end
1 Like
6

That’s not required if the implementation is nested within the module the implementation is for.

3 Likes
7

I stand corrected, thanks @LostKobrakai !

2 Likes
8

Did you see this warning by any chance?

warning: the Jason.Encoder protocol has already been consolidated, an implementation for Test has no effect. If you want to implement protocols after compilation or during tests, check the “Consolidation” section in the Protocol module documentation
dummy.exs:12: Test (module)

(meant to reply to @Fl4m3Ph03n1x)

1 Like
9

@kartheek and @thomasbrus
I am trying this directly in iex, yes. I read the comment, but I don’t quite understand what it means.
I also don’t quite understand how this affects the code I am writing. If someone could explain, I would be thankful.

10

You can read about Protocol Consolidation.

Protocol consolidation happens at compile time. You can disable or enable from mix config as mentioned in docs.

When it is enabled (by default) - it will link protocols with their implementations at compile time to optimise invocations. So when you are trying it in iex - it has no effect as protocol consolidation already happened.

Two solutions:

  • disable protocol consolidation using mix config (not recommended) - Section 16.6 don’t go by title read the content in linked section in this book - https://www.elixircryptobot.com/.
  • define all these encoders in files and compile the project (recommended)

No problem with the code or library or project - its working as it is supposed to be.

3 Likes
11

@kartheek great answer, and I very much enjoyed the link to the book.

One last thing though, just to confirm, the only reason for not doing option 1, is because test performance can be affected, correct?

As for the proposed solution, how would I go about it?

def project do
  ...
  elixirc_paths: elixirc_paths(Mix.env())
  ...
end

defp elixirc_paths(:test), do: ["lib", "test/support"]
defp elixirc_paths(_), do: ["lib"]

I understand I need to have a "test/support/something.ex" file, but it is not clear to me what the contents of this file would be.

1 Like
12

Docs state advantage of protocol consolidation as below:

Consolidation directly links protocols to their implementations in a way that invoking a function from a consolidated protocol is equivalent to invoking two remote functions.

For solving the problem in original post - you have to create the module and encoder in a file say test.ex in lib folder and it will work.

1 Like
13

Thank you!

2 Likes