Persistent background jobs?

I need to broadcast a message to a client, and try to broadcast again within 10 seconds unless I get a confirmation message from client that the message has arrived.

What’s the recommend approach for such a problem?

I thought of registering a Task and let it manage re-broadcasting the message to the client until getting a confirmation,

I register Task like this:

Agent.start_link(fn -> state  end, name: {:global, process_name}) do

then, to stop it when I get a confirmation:

Agent.stop({:global, process_name});

But if the phoenix server got restarted? or if I uploaded a new release, what will happen to those registered Tasks? how can I make sure that they will not be removed when restart or upload a new release to the phoenix server?

On a multi-node system you could just ensure that those tasks don’t exist on the same node as the Phoenix server. But I sense that you are referring to a single node system - in which case you are going to have to persist the active tasks outside of the node.

If you are already using a database you could store them there but it could be something as simple as a file that is read on startup - but that means that you are going to have to manage that persistence, adding tasks that have gone active and removing those that have completed (and launching those that exist on startup).

Also Agent is not a good fit for this job - it is simply passive state container. Using a GenServer makes more sense - it can use Process.send_after/4 to “remind itself” to contact the client again, and call to remove itself off the “resurrection list” (managed by yet another process) before it initiates [termination]
(https://hexdocs.pm/elixir/GenServer.html#c:terminate/2).

The other issue is whether you actually need a process per (message,client) pair or whether a single process could simply handle all of them as the occur (i.e. no need to stop the process while the system is running) - as it may not be that difficult to handle “concurrent pairs”.

1 Like

All the other implementation details aside, a dets table would probably be the simplest Erlang/Elixir approach.

Other than that you’re looking at an external persistence of some kind, probably Redis if you want a lower overhead. RabbitMQ has all of your requirements built in, but is a bit more of a learning curve.

2 Likes