How to store lots of data in memory?

[yurko]

P.S. data stored in memory is a parsed YML file. It occupies 10 KB in filesystem but 10 MB when read and parsed - maybe there’s a way to optimize storing it in memory as well?

There should be, it’s 1000 times more in memory then on disk, doesn’t sound right How did you measure this difference? If you dump it into file, is it 10kb again?

Using ETS doesn’t seem to solve the problem.

If you store it there “as is” as one entry it will not bring much, but if you put the parts of that (structured) data under different keys it will help with memory consumption.

See related discussion:

Elixir processes and no shared heap memory

If a process wants to share a data structure with another process, how could that be possible? There are cases when sharing data between processes is exactly what you need to do, you can use ETS http://erlang.org/doc/man/ets.html to achieve that.