How to place *.json alongside source code and ensure in production/runtime they were accessable?

longnightElixir

I want to place some data in json files, like lib/my_app/json_store/data.json, then access them in Phoenix/liveview controller; (these json files were created, modified by other apps/tools )
these json files do not need accessable in url as public assets , only read by our project .ex source code;
but after release in production env , they were not found in “_/build/…” .

so how to solve this , package these data files into that self-contained directory ? Thanks for any help

10 comments

/phoenix #json

0 502 10

2025-04-25 09:38:28 UTC

Showing Posts 1 to 10

Show Best Posts
Show All (oldest first)
Show All (newest first)

arcanemachine

I think you would want to put them in your project’s /priv directory.

Post #1

kip

ex_cldr Core Team

As @arcanemachine suggests, put them in the priv directory. Then access them with something like:

Path.join(:code.priv_dir(:my_app_name), "json/my_json.json")

Post #2

longnightElixir

Thank you guys, first hear the :code.priv_dir it does worked.

Post #3

ademenev

If the files do not change and you always read them in full, why not convert them into Elixir maps/lists and have them compiled? You can even automate that process and have them recompiled when corresponding JSON files are modified. They need to be outside of priv directory

Post #4

longnightElixir

Those data json files may be large or small; in most cases, my controller/logic just picks some key-values of them, so it always have to read them full first, then find items in it (when user visit our liveview pages); so from the server memory consumption/performance perspective, I wonder store in json or map/list as you said, which way is better ? of course those data were created/modified only in develpment, before we compile project and release to production.

Post #5

smon

Having them in memory as maps or lists will be faster (no disc access or decoding), but will increase your memory footprint.

That tradeoff is yours to decide on.

Post #6

ademenev

Memory footprint will be larger of course, but in many cases it is a tradeoff you wish to make. You eliminate disk access, decoding and some garbage collection.

Of course, as with everything, you need to do some analysis. Can you afford holding all the data in RAM? What is anticipated request rate and requests concurrency? And so on

If there is much data and individual files are large, you could even consider processing the files and putting them into a database

Post #7

dimitarvp

This sounds like a job either for a dedicated database table or an ETS cache, initialized at app startup.

Post #8

longnightElixir

You’re right, but I choose Cachex instead, load full json data into cache/memory with a short expired lifetime, then persistent those mostly accessed key-value items; I think this way may be more balance