I am trying to dynamically create Elxiir modules which have binary file contents embedded in their functions.
It boils down to this:
defmodule EmbeddedFile do
def contents do
unquote(File.read!("some_file.jpg"))
end
end
When compiling this, I observe that compilation takes very long and runs out of memory often. It seems as though the memory usage is exponential to the size of the binary. Even a file of 5MB triggers this, I have 8 GB of RAM.
I wanted to run a webserver as escript similar to what erlang PL does, so it needs to embed all “files” inside the escript. I’ve now looked how they do it and do it differently, embedding the files in the escript .zip.
However my question still stands, the memory usage is quite surprising. @josevalim mentions something similar in this post however there it’s only about slowness of compilation, not the memory usage.
Fair point! I don’t know off hand, reading very large files in the module body itself is fine, but it seems that trying to place that value within the function causes issues.
