Thanks for the great project @sorentwo ! I’m working on using it to handle a long running port process in a separate app. However, I have a requirement to report back percentage progress while it is running, so I’m using a GenServer to manage feedback from the port. What’s a good way to prevent the job from being marked complete until my GenServer has finished its work?
arton
sorentwo
Oban Core Team
Anything that blocks the perform/1 function will prevent the job from being marked complete. There isn’t a limit on how long a job can execute (since it isn’t executed within a transaction). Without knowing more about your use case or seeing the code I can only provide some rough suggestions.
- Make a
callto your GenServer process (we need a call so that we get thefromtuple). - Within your
handle_callfunction grab thefromfor the caller (which will be your job process) and start a Task to track progress. - Respond from
handle_callimmediately without something like{:reply, :ok, newstate}, don’tawaitthe task. - Using a loop within the task start sending progress messages to the caller. You can loop with a delay by using
Process.sleep/1. The original caller will need to need use areceiveloop to block and wait for progress messages. - When the task is complete send the caller a message to let it know work is finished.
- Be sure to set an
aftertimeout in your receive loop to prevent blocking forever and leaving a zombie job.
The worker portion would look roughly like this (warning, untested code):
defmodule MyApp.WaitingWorker do
use Oban.Worker
@impl true
def perform(%{id: id}) do
:ok = MyApp.Business.work(id)
receive_loop(id)
end
defp receive_loop(id) do
receive do
{:progress, percent} ->
report_progress(id, percent)
receive_loop(id)
:complete ->
:ok
after
60_000 ->
raise RuntimeError, "no progress for #{id} after 60s"
end
end
end
Hope that helps and doesn’t just cause more confusion 
6
sorentwo
Oban Core Team
Oban v0.5.0 has been released. This includes some minor features, a number of bug fixes and some documentation improvements.
There are some changes that may cause unexpected retries and different telemetry payload. See the CHANGELOG excerpt and the linked docs below for more details.
Added
- [Oban.Pruner] Added
prune_limitoption to constrain the number of rows deleted on each prune iteration. This prevents locking the database when there are a large number of jobs to delete.
Changed
- [Oban.Worker] Treat
{:error, reason}tuples returned fromperform/1as a failure. The:kindvalue reported in telemetry events is now differentiated, where a rescued exception has the kind:exception, and an error tuple has the kind:error. Thanks @lucasmazza
Fixed
-
[Oban.Testing] Only check
availableandscheduledjobs with theassert|refute_enqueuedtesting helpers. Thanks @jc00ke -
[Oban.Queue.Watchman] Catch process exits when attempting graceful shutdown. Any exits triggered within
terminate/2are caught and ignored. This safely handles situations where the producer exits before the watchman does. Thanks @axelson -
[Oban.Queue.Producer] Use circuit breaker protection for gossip events and call queries directly from the producer process. This prevents pushing all queries through the
Notifier, and ensures more granular control over gossip errors. Thanks @axelson
7
arton
Looks like a great update, keep up the good work @sorentwo !
I have another question. I need to alert the end user if the entire job fails after all retries. Do you have a recommendation? I see the :telemetry.attach("oban-errors", [:oban, :failure], &ErrorReporter.handle_event/4, nil) example, which could be a possibility, but thought I’d check with you.
1
sorentwo
Oban Core Team
That is exactly what I would recommend doing. The meta map passed to your handler will have both max_attempts and the attempt field. You can easily check if retries have been exhausted and send your alert to the user.
The table at the top of Oban.Telemetry docs has a breakdown of all the fields passed to each event. (Though now that I liked to it I see a typo, it is max_attempts plural, not max_attempt)
3
luizpvasc
Thanks for the great work @sorentwo, Oban looks amazing. I’m currently playing with it in a project and everything is running smoothly 
I have a question about pruning, if you don’t mind. In the Readme, the first example suggests prune: {:maxlen, 100_000}. Looking at the code, it seems prune is used to delete truncated jobs and outdated jobs, but I’m not sure what those are. Can you explain how a job gets truncated or outdated?
Thanks!
1
sorentwo
Oban Core Team
Glad to hear Oban is working well for you!
Any job that won’t be executed (either now or in the future) can be pruned. This may be explained better with some examples:
Never Pruned
- a new job, which has the state
available - a scheduled job, which has the state
availableand is scheduled to execute some time in the future - a job that failed execution and will be retried in the future
Possibly Pruned
- any job that ran successfully, it is in the
completedstate - any job that has reached the maximum number of retries or has been manually killed, these are in the
discardedstate
It is designed so that you’ll never lose jobs that still have work to do, but you can prevent the table from growing indefinitely.
3