alexgleason
Hey guys, I’m developing Pleroma and preparing it to scale. Of course the bottleneck is not Elixir, but the Postgres database.
It runs an SQL query to generate each user’s home feed, and this query is expensive and slow.
In the Ruby world, Mastodon solves this problem with Redis. A background worker pre-generates feeds for each user and stores them in memory, ready to be served at any time.
I’m wondering what your opinions are on options in Elixir. Is it reasonable to use ETS to solve this problem?
Where Next?
Trending in Questions
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
Elixir, Erlang & messaging conferences from Code Sync. Next up: ElixirConf US (10–11 Sep) & Code BEAM Europe (21–22 Oct), Haarlem + online.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixirconf-us
- #ai
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #hex
- #security
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Master modern tools, write cleaner code, and tackle real-world challenges.
Producing high quality Elixir screencasts since 2017.
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 2- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
dimitarvp
It definitely is but it really depends on the shape of the data and how quickly does it expire and it’s especially dependent on how well can you segment this data – I mean in terms of unique caching keys.
Have you looked at
Cachex?If you are thinking about having multi-node caches I feel you’d be going against the wind here and maybe should just use Redis indeed.
gpopides
When you say that the query is expensive/slow, to me it seems that you have identified your problem and it’s in the database. Perhaps improving the query it self could solve the problem? Maybe using database views? Trying to optimize the application layer with caches before looking for a solution for the problem itself feels like you will encounter more problems with invalidating caches and so on.