Serialize and unserialize structs, tuples, keyword lists, atoms?

Questions / Help
1

I know I can implement some protocols to make JSON encoding work with structs and tuples.
However, decoding JSON would never produce the same input unless you included some meta data that specified which terms were atoms or structs etc.

I am seeing a few packages that do serialization/deserialization that can serialize structs, tuples, keyword lists, and atoms AND then unserialize them. The closest I think I found was php_serializer | Hex – it’s close, but the unserialize does not come back the way it went in.

I understand some things cannot be serialized (like captured functions I think?), but I think the structs and keyword lists can call be brought back. Does anyone know of a library that can unserialize this way?

So that:

input = %MyStruct{atom: :test, keyword: [a: "yes", b: "no"], tuple: {:ok, "yes"}}

result = input |> serialize() |> unserialize()

input == result

Thank you for any thoughts!

2

There you go:

result = input |> :erlang.term_to_binary() |> :erlang.binary_to_term()
3 Likes
3

Poison has limited support (can’t do what you want) and Jason intentionally does not support decoding into data structures at all, see: Support for decoding into data structures · Issue #124 · michalmuskala/jason · GitHub. Jiffy also can’t do it.

If possible use term_to_binary/binary_to_term or do it yourself.

I’d like to see a JSON decoder like python’s

json.load(fp, *, cls=None, object_hook=None ... object_pairs_hook=None ...)

object_hook is an optional function that will be called with the result of any object literal decoded (a dict). The return value of object_hook will be used instead of the dict. This feature can be used to implement custom decoders (e.g. JSON-RPC class hinting).

object_pairs_hook is an optional function that will be called with the result of any object literal decoded with an ordered list of pairs. The return value of object_pairs_hook will be used instead of the dict. This feature can be used to implement custom decoders. If object_hook is also defined, the object_pairs_hook takes priority.

4

I did not know this!

5

I recommend you take a look at the documentation for this function, there are some cool options to it, for example, you can compress the binary output using the compressed option like this:

:erlang.term_to_binary(data, compressed: 6)
6

Just beware, that it is not safe to use :erlang.binary_to_term/1 on untrusted data.

2 Likes
7

And as soon as things are stored beyond the lifecycle of a single application version one needs to still handle struct (and likely records) with care. Changing their definition might be troublesome.

1 Like