Tuple keys to json

Questions / Help
, ,
1

In Jason/Poison it is possibe to encode tuple values using

defimpl Encoder, for: Tuple do
  def encode(data, options) when is_tuple(data) do
    data
    |> Tuple.to_list()
    |> Encoder.List.encode(options)
  end
end

But it seems like there is no way no encode maps with tuple keys … right?

EDIT: As possible for values I’m looking for a way to explicitly transform the tuples to a valid json key (a string). For example sth like

defimpl EncoderThatAlsoHandlesKeys, for: Tuple do
  def encode(data, options) when is_tuple(data) do
    inspect(data)
   end
end

which hypothetically would transform:

%{{1,2} => "test"}

to

{"{1,2}": "test"}
2

JSON only supports objects with string keys.

Jason therefore only support maps whose keys are strings (or can be turned into strings using the String.Chars protocol).
(I expect Poison to do the same, but I have not tested it myself.)

JSON is quite a peculiar format because it is unversioned but also receives updates from time to time (from two different standards-maintaining organizations; these standards conflict in some details.), as well as allowing parsers to support arbitrary extensions (with no guidance on how to indicate that these are used).
As a result, what different JSON parsing libraries do by default and support in general varies quite significantly.

Some other ‘fun’ pitfalls:

  • Encoding number values outside of the range -2**35+1..2**53-1 is non portable.
  • Before the updated IETF standard of 2014, only objects and arrays were permitted as ‘top level’ JSON values. Many parsers will still reject JSON that tries to encode a number, string, boolean or null directly, so this also cannot be done portably in practice.
  • String encoding rules differ between what is supported in JSON and what is supported in JavaScript.
  • NaN and +-Infinity are not supported by the standard but created by many encoders/decoders as ‘extension’.

See Parsing JSON is a Minefield for more details on some the edge cases one might consider.

3

Sorry for the bad question.
I want to encode the tuple to a String, so that I get valid JSON key.

4

What is the goal? Do you want to be able to read the resulting data from another language, or from another Elixir program? Do you need to be able to decode the key-strings back to their tuple values or not?
What is contained inside the tuple elements?

5

The way that I’d do this is with an actually explicit transformation. As in, instead of map_with_tuple_keys |> Jason.encode! do map_with_tuple_keys |> keys_to_strings |> Jason.encode! where keys_to_strings/1 goes through and explicitly turns all of your tuple keys into a string of your choosing.

2 Likes
6

Actually I just wanted to have a look at my data and tried to create JSON which I’m used to.

Now I discovered, that

inspect(data, pretty: true, limit: infinity)

Is a better way to do that, but still I’d just like to know if there is a way to explicitly encode the keys (like we can values).

7

I like the approach @benwilson512 shared.
If you want to explicitly encode your map keys, call a function on your map to turn them into strings before calling Jason.encode! or another encoding function.

You could encode those tuples using inspect().

If you want to be able to decode datatypes later, then inspect cannot help you however: Turning them back into Elixir requires calling Code.eval_string (which is a function of last resort), and will not be able to deal with opaque datatypes like MapSet, Stream, etc. that return an inspect format which starts with a # (as this is read as a comment).

Another approach is using :erlang.term_to_binary (and :erlang.binary_to_term to decode). This supports (with very few exceptions) all datatypes of Elixir/Erlang, and encoding/decoding is quite performant.
Libraries for some other languages besides the BEAM exist as well.
However, this is a binary format and thus it is not humanly readable.

1 Like
8

That’s a more long form explanation to that approach.

3 Likes
9

Thanks for the insights guys. conclusion:

  • its not possible to hook into encoding of the keys
  • if you really want to, do it manually
  • or: flatten the maps, eg %{{1, 2} => %{...} becomes [%{key_: [1, 2], ...}, ...]
  • for the task I originally had (looking at data) use inspect(data, pretty: true, limit: infinity)
  • store data to reuse later in erlang world: use :erlang.term_to_binary and :erlang.binary_to_term
  • in general: JSON - there is some troube under the hood
2 Likes