geofflane

geofflane

Getting Message and Errors from Broadway

Broadway catches exceptions and handles other errors by logging the error and “failing” the message. Is there a way to get a callback called to do something with that error as well? I would need the original message and the error (message, exception, etc).

My use case:

We’re collecting a bunch of data from a bunch of people over time. As the data gets collected it gets put on a queue and consumed by another app to coalesce that data into a bunch of “facts” (for lack of a better term) that we can query and report on. Any errors get sent back to the collection application by publishing a message to another queue with the original payload and the error reason. The sending application listens for these messages and collects them into a report for “quality control” to rectify.

Rabbit dead-letters don’t work because a nack doesn’t allow you change the message as far as I know. So we need to actually push another message. We built all of this ourselves before Broadway, but not quite as sophisticated as Broadway (e.g. termination process). I was looking to convert to Broadway and couldn’t see a way to do that. Any pointers?

Thanks.

Marked As Solved

msaraiva

msaraiva

Broadway Core Team

Hi @geofflane!

We’ve been discussing the possibility of adding a callback (or something similar) for that purpose. For now, one thing you could try would be wrapping the code that might fail inside handle_message/3 or handle_batch/4 in a try/catch block, do your stuff and then re-raise the exception (or set the message as failed using Message.failed/2).

Not optimal, I confess. But it might do the trick until we have a built-in solution.

Cheers.

-marlus

Last Post!

dnsbty

dnsbty

In case anyone else stumbles across this in the future, since Broadway 0.5.0, the handle_failed/2 callback has been added to handle this: Broadway — Broadway v1.3.0

Where Next?

Popular in Questions Top

rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list. ...
New
Lily
In templates/appointment/index.html.eex: <%= for appointment <- @appointments do %> <tr> <td><%= appoi...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New

Other popular topics Top

rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list. ...
New
nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
274 42576 114
New

We're in Beta

About us Mission Statement