Oban oban_jobs index size

Special Sections Oban
, ,
1

I have observed some possible performance issues with oban reflected in RDS performance insights. We are making heavy use of Oban, and so any minor increase in execution time (only 20ms latency on average, but with 17 calls/s) on oban_jobs seems to have an impact on the rest of our system.

One thing I noticed is that our data size for the table consistency stays at just about 4MB, we only retain data in that table for 5 minutes since we’re running about 7000 jobs/minute at the moment. Our index size, when I just checked was at 25GB! I issued a REINDEX and the index size was reduced down to 500KB. I’m wondering if we have something setup wrong, or if there is a best practice we’re missing, or do we just need to set something up to REINDEX that table every day?

Anybody else encountered something like this? How did you solve the issue?

2

This is a common issue and there’s an official plugin to help reindex on a schedule: Oban.Plugins.Reindexer — Oban v2.18.2

4 Likes
3

Have you checked Oban.Plugins.Reindexer — Oban v2.18.2?

We have it configured like this

 {Oban.Plugins.Reindexer, schedule: "0 4 * * *", timeout: :timer.minutes(5)},
4 Likes
4

Thanks, I had missed this, appreciate the help.