Hello,
I’m having a list of elixir structs and I need to store in into file for later use and I don’t want to serialise it.
So I used :erlang.term_to_binary on the list and that worked fine, but sometimes it happens that there is too much data and system kills the process because it’s out of memory.
So I was thinking about converting it one by one using streams, I have something like this
values_list
|> Stream.map(fn item -> :erlang.term_to_binary(item) end)
|> Stream.into(File.stream!(dir <> "/data.dat"))
|> Stream.run()
However I’m not sure if this is actually somehow better then just simply :erlang.term_to_binary(values_list), but I assume it could be better as it’s one by one, I’m I wrong?
Another problem is that I don’t know how to actually read the file properly. I guess I should add some binary mark, where one item ends, right?
Sorry for probably novice question, I’m still getting to know elixir.
Thank you in advance.
