So I have this bunch of data I’m caching on ETS that refers to the content (videos) I want to be shown to all my users when they log in into my system. The content is the same for everybody, except they all have an association field that keeps track of each user’s progress percentage on each video.
I’ve been trying to come up with solutions to conciliate the cached information with users individual progress on the videos they started watching, like querying all progress data available and then going through each cached video and merging the data but I feel like very inefficient solution… I was wondering how you guys would go about handling this?
I think what I’m having issues modeling in my mind is how to go about querying the user dependent data based on what is on cache.
For instance, picture a Netflix that always shows the same movies in the main page to every user. How to query for progress data for every movie reference that in cache?
I ask this because in my naive solution I’m querying all the movies in cache against the user id to get back which ones they started watching and their progress so far on each and then stitching the data together. But that got me thinking if this is not more inefficient (or as efficient) as not having a cache and querying the database for the entire set of movies + user progress each time…
Where user_id is the cache key. Once they start watching a video - progress data gets added. If a movie_id does not exist - they have not started watching. (progress = 0)
Might be worth putting this kinda data in a DB though as you have realized…
Yeah, keep all the user progress in one cache entry for reading, this way you need to access only one entry with a key that you always know (user id). But you should also write the data to the database when it is changed, so that you are sure it is always consistent (depending on the cache backend it may be ephemeral, etc.).
