You don’t need to learn binary to save to a log file, you just use :erlang.term_to_binary and then pass it into IO.binwrite(file_descriptor, binary_data) to save into the file and when reading from the file you jsut use :erlang.binary_to_term to get your data decoded:
defmodule FileIO do
@moduledoc """
iex(3)> fd = FileIO.open! "test.bin"
#PID<0.200.0>
iex(4)> FileIO.append! fd, %{key: "value"}
%{end_position: 22, file_descriptor: #PID<0.200.0>, size: 22, start_position: 0}
iex(5)> FileIO.read! "test.bin"
%{key: "value"}
"""
@write_mode [:append, :binary]
# @write_mode [:append, :binary, :delayed_write]
# @write_mode [:append, :binary, {:delayed_write, 1, 1}]
def open!(path, write_mode \\ @write_mode) do
File.open!(path, write_mode)
end
def close(file_descriptor) do
File.close(file_descriptor)
end
def append!(file_descriptor, data) do
{:ok, start_position} = :file.position(file_descriptor, :cur)
data = :erlang.term_to_binary(data)
:ok = IO.binwrite(file_descriptor, data)
{:ok, end_position} = :file.position(file_descriptor, :cur)
%{
file_descriptor: file_descriptor,
start_position: start_position,
end_position: end_position,
size: byte_size(data)
}
end
def read!(path) do
{:ok, data} = :file.read_file(path)
data |> :erlang.binary_to_term()
end
end
This example is a very simplified form of what this EventLog library does:
EventStoreDB is a custom-built event store which provides very high performance. There’s an Elixir library named extreme which can be used as a client. This is different from the Postgres-based Elixir EventStore library that I wrote (which it was inspired by). Both event stores are also compatible with Commanded and you can choose to use either store. Commanded provides a CQRS framework for event sourcing.
There is now a cloud hosted version named Event Store Cloud which allows you to deploy managed EventStoreDB clusters in the cloud through AWS, GCP, and Azure. I’d recommend looking at this if you want the best possible performance. The only caveat is that EventStoreDB is migrating to using gRPC as its client interface which is not well supported in Elixir. The existing extreme library uses the older TCP interface.
Returns true on iex, I got confused thinking with integers they will be converted into strings and subsequently concatenated. It’s clear that in this case the integers will simply be stored in successive bytes unlike it string counterparts which are concatenated.
b = << 1::size(2), 1::size(3) >> == <<9::size(5)>>
Does the above implies when you indicate a size either than 8, it forces the numbers to be stored together hence generating a single value?
EventStoreDB does not use Postgres, it uses its own internal storage format and writes directly to disk. The Elixir EventStore library (confusingly I named it similarly) does use Postgres.
So it is not limited to the “joining”. In general <<>> can encode 2 things in Erlang and Elixir:
bit strings which is set of bits that is stored as sequence of bytes, with last entry being possibly non 8-bit long
binaries which are bit-strings with last byte always full aka bit strings with bit length that is multiply of 8
<<"a", "b">> is special syntax that is inherited from Erlang, where binaries that contain only Unicode can be written ass <<"a">> which is the same as <<$a>> (Elixir syntax for that is <<?a>>).
Binary by itself has nothing to do with unicode. <<3>> is the binary number 3 in a byte, which is also decimal 3.
If you want the unicode code points you have to do something like <<"☺"/utf8>> in erlang or “” in elixir, which is <<226,152,186>> deep down.
A binary is a sequence of arbitrary bytes. An elixir string is a binary representing the utf8 encoded string.
All strings are binaries, not all binaries are strings.
If you want to use a string literal, do not write it in binary form like <<…>> because it will confuse the hell out of readers. Use a double quoted string.
If you want to use a binary literal that is not a legal utf8 string, write it in binary form with numbers 0~255 to avoid confusions like: <<354>> == <<98>> == "b"
” in elixir, which is 
