Elixir alternative to AWS SQS?

[Exadra37]

@dogweather

I’m about to use SQS (and maybe Broadway to read from it, and connect to the next stage), but thinking there might be some Elixir / OTP components that could provide this functionality.

While I don’t have a direct alternative to suggest for you in Elixir I would recommend you to read:

Can we beat Kafka if we build it in Elixir?

I am going through the kafka architecture. All the features what the kafka is providing are already in Erlang. I would like hear your opinion on the Kafka type implementation with Erlang. Kafka® is used for building real-time data pipelines and streaming apps. It is horizontally scalable, fault-tolerant, wicked fast, and runs in production in thousands of companies.

What would it take to create Kafka like solution in Elixir?

Just went through this great discussion thread on Big Data with Elixir and looking at this diagram I couldn’t help but ask can Broadway be a replacement for Kafka ? But on further research it turns out Broadway is a way of creating efficient data processing pipelines. And as such it sits more downstream on the consumer side of the equation. Though one can see a consumer further producing something which then is picked up by another data processing pipelines. There have been attempts like ErlBus,…

How would you compare Kafka to BEAM and OTP?

Apache Kafka also promise about similar benefits as BEAM and OTP offer like Distribution, Fault Tolerance etc. Edit: Apache Cassandra which is AMAIK a database, also promise things like Linear Scalability, High Availability and Fault Tolernace. (Because it’s database/data-storage, we might better compare it to ETS or Mnesia). End-Edit How are these different/better/worse than BEAM and OTP ?