Bert.js is a javascript library that allows (de)serialization of Erlang terms between javascript and Erlang/Elixir.
Here it is on GitHub and on NPM
Bert.js is a javascript library that allows (de)serialization of Erlang terms between javascript and Erlang/Elixir.
Here it is on GitHub and on NPM
Looks useful though you should detail which of the ETF types you donāt support (like closures and so forth) and of the ones that you do support then how they map to javascript, for documentation reasons.
Thanks! Iām still in the process of writing up documentation, but Iāll update this thread once I have that done
I was just looking at this the other day wondering if there were any libraries doing active development on this. I canāt wait to see how this works with my library thatās diffāing AST (HTML parsed into AST) and pushing patches over the wire. I imagine berts will be quite the optimization
Let me know how that turns out! Iām curious as well
@dgmcguire Benchmark it. ETF serialization is not as fast as youād think, even JSON can be faster at times (though less expressive).
Yup, Iāve learned to trust nothing but benchmarks and to not trust them either
Bert.js with your AST diffing (is that iolistt?), they could work with Phoenix.LiveView. That would be interesting.
Iām not really sure this is a good idea: using :erlang.binary_to_term() may lead to atoms exhaustion.
A malicious user might craft a payload that contains a huge number of new atoms, with just few malicious payloads this might happen: āAtoms are not garbage-collected. Once an atom is created, it is never removed. The emulator terminates if the limit for the number of atoms (1,048,576 by default) is reached.ā. In my opinion this sounds like a denial of service.
I think that JSON might be enough for your case. Also, like I said it before, donāt diff AST. Diff a special internediate representation.
I think bert-encoded-html over websocket has been already done by n2o project a few years ago. I donāt remember it using any kind of diffing back then, though.
@alexdovzhanyn you might be interested in reading through the sources of some other bert encoders/decoders in js like https://github.com/synrc/n2o/blob/master/priv/protocols/bert.js and GitHub - discord/erlpack: High Performance Erlang Term Format Packer, in case you havenāt already.
:erlang.binary_to_term/2 has an option to prevent DoS via the atoms table
I still need to go back and reread all of your advice, but Iām wondering if you have a different idea of what I mean by AST. Iām not talking about diffing elixir AST. Iām diffing html AST
(and really itās not even a tree diff right now, like I said about using floki at runtime - right now Iām focusing on a usable library before an optimized library, but I am getting very close to moving to optimizations)
when you say intermediate representation do you mean the flattened out nodes list?
We are talking about the same thing then. Sorry.
I think decoding functions would also create atoms.
The safe version of the decoder also prevents that.
You can pass an option to :erlang.binary_to_term/2
to tell it not to create new atoms (will error if someone tries). (EDIT: as @dgmcguire says further below that post)
Thanks, good point
I made a pull request to Bert.js, https://github.com/ElixiumNetwork/bert-elixir/pull/1 to make it clear
I think we should always keep Elixir comunity security aware
There is a discussion on a related topic here: Anyone using Erlang External Term Format (ETF) instead of e.g. JSON?, so I posted there the link I was going to post here.
Very cool!
Question time: When is it useful to use Bert.js instead of JSON? (Because while BERT/ETF is Elixirās ānativeā serialization, JSON is the browserās native serialization, so Iām not sure about the advantages of using BERT here.) How does its JS (de)serialization compare in sense of execution speed/memory usage vs JSON?