There’s a fairly large data structure represented as an XML file. I don’t feel like parsing said file every time I need a piece of data out of it represented in Elixir terms. Since the data for current intent and purpose can be treated as static I thought of “compiling it in” so that it could be accessed quickly and without re–parsing the XML. But before I start dusting off my sed-fu to convert text files, maybe the Elixir experts community have something better/ready for such occasions? How do you typically pull such external data into Elixir applications?
1 Like
If things are static enough I usually parse the external file at compile time and pull only the parts I need in a module attribute.
2 Likes
Could you be so kind as to elaborate a bit on that?
1 Like
The @external_resource module attribute is a good place to start:
https://hexdocs.pm/elixir/Module.html#module-external_resource
You could point that at your XML file, then parse it into a module attribute:
defmodule BagOfXml do
@external_resource "path/to/some.xml"
@parsed_data SomeOtherModule.parse("path/to/some.xml")
def lookup_thing(id) do
# use @parsed_data and id to do stuff
end
end
This will call SomeOtherModule.parse at compile-time (it cannot be a function in the same file) and store the resulting data in the compiled BEAM file.
@external_resource will ensure that if the XML file changes the module will be recompiled.
3 Likes
Essentially that, but also if only parts of the data are needed at runtime, discarding everything not needed. Nobody wants unreasonable large beam files.
2 Likes
I suspected there must be something better! This looks exactly right for the purpose. The fact that it will recompile once the data file changes (rarely but still can’t be fully excluded) is an icing on the cake. Thanks guys.
