alexgleason
Pre-generating user feeds in a Twitter-like microblogging platform
Hey guys, I’m developing Pleroma and preparing it to scale. Of course the bottleneck is not Elixir, but the Postgres database.
It runs an SQL query to generate each user’s home feed, and this query is expensive and slow.
In the Ruby world, Mastodon solves this problem with Redis. A background worker pre-generates feeds for each user and stores them in memory, ready to be served at any time.
I’m wondering what your opinions are on options in Elixir. Is it reasonable to use ETS to solve this problem?
First Post!
dimitarvp
It definitely is but it really depends on the shape of the data and how quickly does it expire and it’s especially dependent on how well can you segment this data – I mean in terms of unique caching keys.
Have you looked at Cachex?
If you are thinking about having multi-node caches I feel you’d be going against the wind here and maybe should just use Redis indeed.
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
We build reliable cloud platforms for business-critical systems.
Categories:
Sub Categories:
Forums
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Develop your skills with books, videos, and courses.
Enabling companies to succeed by building software people love.
The deployment platform built for Elixir: PaaS ease, VPS control.
Courses that'll move you from confusion to "Aha, now I get it!"