Anyone using Erlang External Term Format (ETF) instead of e.g. JSON?

We currently use BERT between Erlang/Java in one subsystem and JSON between Erlang/Java in another. I have also use BERT in other projects.

We found using JSON to be a PITA to change and update in comparison to BERT. This is partly because, at the very least, the Java side we can rely on (minimal) compiler typechecking.

Although you could use a JSON validator the Erlang validator was feature weak (4 years ago). YMMV.

I prefer BERT because it is a wire protocol and compact. I would even consider protobuf over JSON/XML any day.

Hex.pm API responds with JSON or ETF. The latter is very convenient as we don’t need a JSON parser to understand it:

curl --silent -H "accept: application/vnd.hex+erlang" https://hex.pm/api | elixir -e "IO.read(:stdio, :all) |> :erlang.binary_to_term() |> IO.inspect()"
%{
  "documentation_url" => "http://docs.hexpm.apiary.io",
  "key_url" => "https://hex.pm/api/keys/{name}",
  "keys_url" => "https://hex.pm/api/keys",
  "package_owners_url" => "https://hex.pm/api/packages/{name}/owners",
  "package_release_url" => "https://hex.pm/api/packages/{name}/releases/{version}",
  "package_url" => "https://hex.pm/api/packages/{name}",
  "packages_url" => "https://hex.pm/api/packages"
}

@Qqwy I’m using BERT as often as possible. JSON is used only if it’s requried. I love to use such data format, because I do not need to add any external library to write full work result with minimal number of lines + all what you have already said. For me main use case is for Elixir only (my private projects), temporary with JavaScript (client) and again Elixir only (scenic client + Elixir to WebAssembly client) in future.

Have you tried using :erlang.binary_to_term/2 which has a safe option which is designed to help with this. binary_to_term/2

Wow! Great! I was not aware of this feature, and I will immediately start using it!

I wish there was another one called data_only or something similar that would forbid funs. If that was the case, it probably would cover all the needs automatically. Right now, in most cases, I need to traverse the decoded data to check if there were any funs in there.

Are you using BERT as defined in http://bert-rpc.org/ or :erlang.term_to_binary directly? They are two different things. In particular BERT itself doesn’t do maps and has a separate, much more verbose encoding.

Oh, sorry I though that :erlang.term_to_binary is BERT implementation. Is there any other implementation written in Erlang? If so then which one is better for Elixir ↔ other language and Elixir ↔ Elixir?

Any downside to using Plug.Crypto.safe_binary_to_term to filter out any funs? At least that’s what I’ve been doing… and it looks like they have recently extracted plug_crypto to a separate library in case you’re not already using plug itself.

Since options are passed to erlang I guess you could use Plug.Crypto.safe_binary_to_term(bin, [:safe]) to protect against atom exhaustion also?

No downside, just that it’s slower than a native option could be since you have to traverse the data after it’s decoded. For now that’s the best option, though.

Yesterday I wrote a post on this topic, so I would like to share my opinion here https://blog.ispirata.com/how-to-destroy-your-application-using-erlang-binary-to-term-1-575ff7d05333 (I was going to talk about this on the Bert.js topic). Let me know if you don’t agree