This is a great analysis, but I’ll nitpick on a couple of points.

Persisting the queue to disk directly from Erlang is straightforward using DETS or disk-based Mnesia table. I used the latter approach in my first system and had no problems with it.

If Redis solves it by not being distributed, so can Erlang A 3rd party non-distributed approach is not the simplest solution to a distributed problem. An in-tech (in this case pure Erlang or Elixir) solution could be simpler

I didn’t use Redis for many years, but IIRC it actually has distributed support. However, that didn’t fare well in tests made by aphyr in his Jepsen series (disclaimer: they were taken a few years ago, so maybe Redis improved since then). My takeaway from reading those was that Redis offers no strong guarantees in distributed setting. I can easily build my own in-Erlang/Elixir distributed solution that offers no real guarantees

Therefore, I don’t see compelling use case for Redis and wouldn’t recommend using it. However, some other 3rd party solution might be fine. Assuming it actually delivers on its promise, it can certainly be compelling, since unfortunately I’m not aware of currently available high-level-easy-to-use-partition-tolerant distributed abstractions in Erlang. While there are some libraries, most notably Riak Core and Riak Ensemble, the usage is not straightforward, and you need to use some forks to make them work with the most recent Erlang.

I hope this will improve with time. Phoenix Presence is a great example of a distributed abstraction built using well understood algorithms, and at the same time it seems easy to use and well documented. I hope we’ll see more of such libraries in the future.

I think you came around to the answer at the end there. IMHO it takes pretty specific requirements to make me even consider going outside beam for concurrent processing.

That said I’ve heard of verk and IIRC it also supports sidekiq format. That situation is really only useful if you have a ruby app creating background jobs to be processed in elixir.