henrikc

henrikc

Use Broadway with ElasticMQ in docker

This is my first attempt at using Broadway, SQS and I am very new to Elixir, so there is hopefully a simple answer. I want to use broadway to consume from a queue in ElasticMQ so that I do not have to use AWS during development. My attempt at the configuration for broadway is,

name: __MODULE__,
producer: [
  module:
    {BroadwaySQS.Producer,
      queue_url: "http://localhost:9324/queue/service-queue",
      config: [
        access_key_id: "x",
        secret_access_key: "x",
        region: "elasticmq"
      ]}
],
processors: [
  default: []
],
batchers: [
  default: [
    batch_size: 10,
    batch_timeout: 2000
  ]
]

The docker-compose to set-up for ElasticMQ looks like,

 sqs:
  image: "softwaremill/elasticmq"
  ports:
   - "9324:9324"
   - "9325:9325"
  volumes:
    - ./elasticmq.conf:/opt/elasticmq.conf

and the elastic.conf file is,

include classpath("application.conf")

queues {
   service-queue {
     defaultVisibilityTimeout = 10 seconds
     delay = 5 seconds
     receiveMessageWait = 0 seconds
   }
}

The problem is that no events are received and after a few seconds I get the error message [error] Unable to fetch events from AWS. Reason: :nxdomain. How should I configure Broadway to make it work with ElasticMQ? Or is that not possible?

I should perhaps note that I tried out the url(since the error indicates a non-existing domain) to the queue using Python and boto3, as described on github, and that works for both receiving and sending messages. I suspect that there is something I missed in the configuration for the producer, any input would be awesome.

Most Liked

henrikc

henrikc

Turns out It was a problem with the producer config. For the next person that might run in to the same error this was the correct config given the settings of elasticmq above:

BroadwaySQS.Producer,
queue_url: "http://localhost:9324/queue/service-queue",
config: [
   access_key_id: "x",
   secret_access_key: "x",
   host: "localhost",
   port: "9324",
   scheme: "http://",
   region: "elasticmq"
]

You need to specify port,scheme and host in the config as well. It was the same as this issue for connecting to localstack with ex_aws.

Where Next?

Popular in Questions Top

sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
hariharasudhan94
lets say i have a sample like a = 20; b = 10; if (a > b) do {:ok, "a"} end if (a < b) do {:ok, b} end if (a == b) do {:ok, "equa...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: The documentation above suggests that while ...
New
dotdotdotPaul
Okay, I’m having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I’m sure I’...
New
svb
Hi! Currently I want to submit a form by pressing the Enter key. However, since my input field is of type “textarea” this is just adds a...
New

Other popular topics Top

hariharasudhan94
lets say i have a sample like a = 20; b = 10; if (a > b) do {:ok, "a"} end if (a < b) do {:ok, b} end if (a == b) do {:ok, "equa...
New
AngeloChecked
What learn first? Rust or Elixir Hi Elixir community! I’m here because i want learn a new language. I’m a junior developer and mainly i ...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
gausby
I asked this very same question on twitter and got some interesting feedback, but I thought it would be a good question to ask here as we...
1207 39297 209
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
marick
I had some trouble figuring out how to make many-to-many associations work. Once I got it working, I wrote a blog post. Because I’m a nov...
New
PeterCarter
There are pre-rolled solutions for other frameworks that do work. However, Phoenix does not seem to have these. Have people had good expe...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

We're in Beta

About us Mission Statement