pedromvieira

pedromvieira

How do you ensure that only one “copy” of a worker will be active in a multi node otp application?

How to ensure that only one “copy” of worker will be active in a multi node otp application?
We created some tasks to update KPIs that will trigger on a set timed interval (ex: 60 seconds).
That’s why to avoid duplication and extra resources usage, only one per worker type need to be up running accross all nodes.

defmodule MyApp.Application do
  @moduledoc """
  Application Settings.
  """

  use Application

  def start(_type, _args) do
    import Supervisor.Spec

    children = [
      supervisor(MyApp.Repo, []),
      supervisor(MyApp.Endpoint, []),
      worker(Guardian.DB.Token.SweeperServer, []),
      worker(MyApp.Services.UserAgents.Server, []),
      worker(MyApp.Services.IPs.Server, [])
    ]

    opts = [strategy: :one_for_one, name: MyApp.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

Most Liked

keathley

keathley

This really depends on what you mean when you say, “ensure”. How important is it that you only ever have one copy of a worker? There are loads of libraries out there to help solve this problem: swarm, syn, gproc, or even :global. Each of these libraries have different tradeoffs and guarantees. They handle things like network partitions and node failures differently. It’ll really come down to the kinds of guarantees you need. If you have a small and relatively stable set of nodes and can tolerate occasionally having “duplicate” workers then I’d look at swarm. If you need somewhat stricter guarantees then you might look at gproc. If you need even stricter guarantees you might want to use raft or better yet use an external data store like redis. My intuition (which should be taken with a massive grain of salt since I don’t know your exact use case) is that you’re probably best off using something like swarm. You can use “at least once” messaging guarantees over whatever transport you’re using and work to make your downstream service idempotent.

peerreynders

peerreynders

Would Raft be applicable to your use case?

Where Next?

Popular in Questions Top

chokchit
** (DBConnection.ConnectionError) connection not available and request was dropped from queue after 2733ms. You can configure how long re...
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
mgjohns61585
Could someone help me? I’m making my first elixir program, number guessing game. I can’t figure out how to convert the user’s guess from ...
New
johnnyicon
Hi all, I’ve just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I’m trying to use Postgres...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New

Other popular topics Top

danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 29603 241
New
sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
985 43487 311
New
marius95
Hello everyone, I try to use an Javascript Event Handler in my root.html.leex file. Therefore I created a function in the app.js file: ...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? Ecto.Repo — Ecto v3.14.0 has exampl...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
RisingFromAshes
I’ve read in another post that it may be possible with a router helper - but I couldn’t find an appropriate one, and tbh, I’m still just ...
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 54120 245
New

Latest on Elixir Forum

Elixir Forum

We're in Beta

About us Mission Statement