How to place *.json alongside source code and ensure in production/runtime they were accessable?

Phoenix Forum Questions / Help
,
1

I want to place some data in json files, like lib/my_app/json_store/data.json, then access them in Phoenix/liveview controller; (these json files were created, modified by other apps/tools )
these json files do not need accessable in url as public assets , only read by our project .ex source code;
but after release in production env , they were not found in “_/build/…” .

so how to solve this , package these data files into that self-contained directory ? Thanks for any help

2

I think you would want to put them in your project’s /priv directory.

1 Like
3

As @arcanemachine suggests, put them in the priv directory. Then access them with something like:

Path.join(:code.priv_dir(:my_app_name), "json/my_json.json")
2 Likes
4

Thank you guys, first hear the :code.priv_dir it does worked.

5

If the files do not change and you always read them in full, why not convert them into Elixir maps/lists and have them compiled? You can even automate that process and have them recompiled when corresponding JSON files are modified. They need to be outside of priv directory

2 Likes
6

Those data json files may be large or small; in most cases, my controller/logic just picks some key-values of them, so it always have to read them full first, then find items in it (when user visit our liveview pages); so from the server memory consumption/performance perspective, I wonder store in json or map/list as you said, which way is better ? of course those data were created/modified only in develpment, before we compile project and release to production.

7

Having them in memory as maps or lists will be faster (no disc access or decoding), but will increase your memory footprint.

That tradeoff is yours to decide on. :wink:

1 Like
8

Memory footprint will be larger of course, but in many cases it is a tradeoff you wish to make. You eliminate disk access, decoding and some garbage collection.

Of course, as with everything, you need to do some analysis. Can you afford holding all the data in RAM? What is anticipated request rate and requests concurrency? And so on

If there is much data and individual files are large, you could even consider processing the files and putting them into a database

2 Likes
9

This sounds like a job either for a dedicated database table or an ETS cache, initialized at app startup.

10

You’re right, but I choose Cachex instead, load full json data into cache/memory with a short expired lifetime, then persistent those mostly accessed key-value items; I think this way may be more balance :grinning_face:

1 Like
11

Good choice. Cachex is great.

1 Like