Using an alternate-exchange in Broadway RabbitMQ; should it receive rejected/failed messages from the primary exchange?

fireproofsocks

I’ve set up a headers exchange in RabbitMQ with an alternate exchange (of type fanout). Partly I just want to see how this works, but partly I think this is a bit more flexible than a dead-letter exchange/queue (for example, if I want to have multiple queues in the alternate-exchange with different producers so the same message can be handled in different ways).

The setup is like this:

{:ok, connection} = AMQP.Connection.open([]),
{:ok, channel} = AMQP.Channel.open(connection)
:ok = AMQP.Exchange.declare(channel, "fallback_exchange", :fanout, durable: true),
:ok = AMQP.Exchange.declare(channel, "headers_exchange", :headers, durable: true, arguments: [{"alternate-exchange", :longstr, "fallback_exchange"}])

# The regular queue(s)...
AMQP.Queue.declare(channel, "q1", durable: true)
AMQP.Queue.bind(channel, "q1", "headers_exchange", arguments: [{"h1", :longstr, "abc"}, {"x-match", :longstr, "all"}])
# ... etc...

# The dead-letter queue(s)...
AMQP.Queue.declare(channel, "misfits", durable: true)
AMQP.Queue.bind(channel, "misfits", "fallback_exchange")

I have a Broadway producer bound to "q1", and it is handling regular messages fine. But when it fails a message, that message is not sent onward to the "fallback_exchange"… it just seems to be acked and discarded.

I can see that the "misfits" queue receives messages when a message is published to the headers exchange but there were no matching bindings.

Do I have to manually publish messages back to the "fallback_exchange" in the handle_failed/2 callback? I think this will work, but I would prefer that the Broadway producer not know or care about that stuff. Is this possible? Did I configure this incorrectly?

Thanks for any guidance!

1 comment

#amqp #broadway_rabbitmq

0 389 1

2024-10-08 11:31:00 UTC

Showing Posts 1 to 1

Show Best Posts
Show All (oldest first)
Show All (newest first)

fireproofsocks

I can now answer my own question:

The alternate exchange is a feature of an exchange and it kicks in when the exchange could not route a message to a queue. E.g. you send a message to a headers exchange, but the message headers do not meet any of the requirements specified by the bindings, so the first exchange hot-potato’s the message over to the alternate queue.

If you want to republish failed messages to a different exchange, that’s the job of the dead-letter exchange feature. The x-dead-leader-exchange argument is a feature of a queue, so you must define it when you define the queue (NOT the binding). If you use the RabbitMQ dashboard, you can see queues with the DLX feature. If your Broadway pipeline is processing messages from a queue with a dead-letter exchange specified, then when that pipeline fails a message (or when an exception is raised), RabbitMQ will republish that message to the dead-letter exchange.

TL;DR: alternate exchanges does not cover all the use-cases covered by the dead-letter exchange. They are related, but different.

Post #1

— All posts loaded —