geofflane

geofflane

Getting Message and Errors from Broadway

Broadway catches exceptions and handles other errors by logging the error and “failing” the message. Is there a way to get a callback called to do something with that error as well? I would need the original message and the error (message, exception, etc).

My use case:

We’re collecting a bunch of data from a bunch of people over time. As the data gets collected it gets put on a queue and consumed by another app to coalesce that data into a bunch of “facts” (for lack of a better term) that we can query and report on. Any errors get sent back to the collection application by publishing a message to another queue with the original payload and the error reason. The sending application listens for these messages and collects them into a report for “quality control” to rectify.

Rabbit dead-letters don’t work because a nack doesn’t allow you change the message as far as I know. So we need to actually push another message. We built all of this ourselves before Broadway, but not quite as sophisticated as Broadway (e.g. termination process). I was looking to convert to Broadway and couldn’t see a way to do that. Any pointers?

Thanks.

Marked As Solved

msaraiva

msaraiva

Broadway Core Team

Hi @geofflane!

We’ve been discussing the possibility of adding a callback (or something similar) for that purpose. For now, one thing you could try would be wrapping the code that might fail inside handle_message/3 or handle_batch/4 in a try/catch block, do your stuff and then re-raise the exception (or set the message as failed using Message.failed/2).

Not optimal, I confess. But it might do the trick until we have a built-in solution.

Cheers.

-marlus

Where Next?

Popular in Questions Top

marius95
Hello everyone, I try to use an Javascript Event Handler in my root.html.leex file. Therefore I created a function in the app.js file: ...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
JulienCorb
I am trying to implement my new.html.eex file to create new posts on my website. new.html.eex: <h1>Create Post</h1> <...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
RisingFromAshes
I've read in another post that it may be possible with a router helper - but I couldn't find an appropriate one, and tbh, I'm still just ...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New

Other popular topics Top

vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
AstonJ
We’ve put together this wiki for Phoenix LiveView - please feel free to add any info you feel is worth including. What is Phoenix LiveV...
New
hariharasudhan94
Lets say i have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => "XX...
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New

We're in Beta

About us Mission Statement