Elixir alternative to AWS SQS?

Questions / Help
1

How would an Elixir implementation be organized that provides the main features of SQS?

  • The queue is persistent if the system shuts down
  • Can accept new items quickly
  • Makes it easy to check or be notified when a new item is ready to pop.

I’m about to use SQS (and maybe Broadway to read from it, and connect to the next stage), but thinking there might be some Elixir / OTP components that could provide this functionality.

3 Likes
2

You could build something similar with Mnesia and regular messages, or with custom storage. See RabbitMQ for example. However it will be more troublesome and maintenance-needed solution than just using SQS. However if you for example already use PostgreSQL and you will not have large volume of messages then you can check out PostgreSQL NOTIFY mechanism, maybe it will be enough for you.

3

I would advise against using Mnesia because in fact Mnesia doesn’t give the ACID guarantees it advertises in the docs, as its not durable and you can loose data with netsplits, due to unexpected failures(it has a delay to write to disk) or when you start it(my case).

Some posts:

http://erlang.2086793.n4.nabble.com/mnesia-sync-transactions-not-fsynced-td4673313.html

I am ditching Mnesia in favor of Mnevis, aka Mnesia with the RAFT consensus:

1 Like
4

While I don’t have a direct alternative to suggest for you in Elixir I would recommend you to read:

5

No personal experience, but Oban was discussed recently on the Thinking Elixir podcast.

1 Like
6

EventStore may be suitable for your use case. It uses PostgreSQL for storage.

3 Likes