henrikc
Use Broadway with ElasticMQ in docker
This is my first attempt at using Broadway, SQS and I am very new to Elixir, so there is hopefully a simple answer. I want to use broadway to consume from a queue in ElasticMQ so that I do not have to use AWS during development. My attempt at the configuration for broadway is,
name: __MODULE__,
producer: [
module:
{BroadwaySQS.Producer,
queue_url: "http://localhost:9324/queue/service-queue",
config: [
access_key_id: "x",
secret_access_key: "x",
region: "elasticmq"
]}
],
processors: [
default: []
],
batchers: [
default: [
batch_size: 10,
batch_timeout: 2000
]
]
The docker-compose to set-up for ElasticMQ looks like,
sqs:
image: "softwaremill/elasticmq"
ports:
- "9324:9324"
- "9325:9325"
volumes:
- ./elasticmq.conf:/opt/elasticmq.conf
and the elastic.conf file is,
include classpath("application.conf")
queues {
service-queue {
defaultVisibilityTimeout = 10 seconds
delay = 5 seconds
receiveMessageWait = 0 seconds
}
}
The problem is that no events are received and after a few seconds I get the error message [error] Unable to fetch events from AWS. Reason: :nxdomain. How should I configure Broadway to make it work with ElasticMQ? Or is that not possible?
I should perhaps note that I tried out the url(since the error indicates a non-existing domain) to the queue using Python and boto3, as described on github, and that works for both receiving and sending messages. I suspect that there is something I missed in the configuration for the producer, any input would be awesome.
Most Liked
henrikc
Turns out It was a problem with the producer config. For the next person that might run in to the same error this was the correct config given the settings of elasticmq above:
BroadwaySQS.Producer,
queue_url: "http://localhost:9324/queue/service-queue",
config: [
access_key_id: "x",
secret_access_key: "x",
host: "localhost",
port: "9324",
scheme: "http://",
region: "elasticmq"
]
You need to specify port,scheme and host in the config as well. It was the same as this issue for connecting to localstack with ex_aws.
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance









