Caching large session data - is this a viable approach?

Phoenix Forum Questions / Help
, , ,
1

Hey guys.

I’m quietly reading the forum for a longer time now and really like it. Now I finally came to my first project using elixir and phoenix. So I’m quite new to elixir and phoenix.

In the project I’m working on a user can upload a CSV File which is generated by a third party mobile application. I’m trying to find a way to persist the data for the current user session - so that the user does not need to upload the file again when switching pages. I don’t want to use ecto and don’t want to persist the data to any database because the data is short lived and will definitely change on the next upload.

My first intention was to store the data in the session - however the CSV can get quite large (up to 10k rows) which will hit the 4KB session cookie limit. Now I’m thinking about the following approach and wonder if this is a good way to accomplish this:

When a user uploads the CSV, I generate a random ID and store this is in the session. Then I use some sort of cache on the server (ETS or a GenServer?) and store the uploaded data in the cache under the generated ID key. So I can receive the data by using the ID which has been saved in the session.

Is this a viable approach? How would I manage the deletion of data when the session expires? It’s a small app and it will run on only one node.

The project is open source - If you’re interested, you can find it here: https://github.com/benlime/dashwallet

Thanks!

2

If not much will be stored ‘at the same time’ then just store it in a GenServer and ask for it again via a unique key (like a ref) and you store that unique key in the session. Just keep a time of ‘last requested’ on it and clean it out with a janitor timeout process on occasion to clear ones that have not been touched in however-much-time.

Or just use something like Cachex, which handles all of that for you already, janitor sweeping and all. :slight_smile:

3

Thanks for your reply. I think I’ll use Cachex for now to get started quickly. Anyway, for learning purposes it would be better to implement a simple cache by myself. :stuck_out_tongue: