How to make sure a function only runs once each time it's intimated by Quantum in a cluster.

Questions / Help
, , ,
1

Running a function everyday on an app that is deployed to Kubernetes.

config :myapp, MyApp.Scheduler,
jobs: [
  {"0 11 * * *", fn -> MyApp.MyModule.my_function() end}
]

The function is making a few api calls that takes about 10s in total. What happens in production is that it runs multiple times in parallel, only the first process is showing up in the k8s pod logs but from the db logs and api service logs it’s clear that it ran multiple times.
All the solutions I’ve found so far uses a GenServer which is overkill since I only need to prevent a simple function to run more than once each day when called by Quantum.
One GenServer solution I found looks like this snippet source

def start_link() do
  case GenServer.start_link(__MODULE__, nil, name: {:global, __MODULE__}) do
    {:ok, pid} ->
      {:ok, pid}
    {:error, {:already_started, pid}} ->
      Process.link(pid)
      {:ok, pid}
    :ignore ->
      :ignore
  end
end

Is there a simple solution to this that I have missed or can I use a Supervisor to prevent multiple instances? Is it possible to name a Task and get the same functionality as above?
Need help getting pointed in the right direction.
Thanks.

2

Maybe quantum task is running on different k8s pods :thinking:. Each pod will run its own quantum task. So, if you have 2 pods - you will run task twice

1 Like
3

Tested with one pod, still runs several times

4

A GenServer is not really overkill here, it’s what it is for.

You can always set a flag in an ETS table to synchronize on though.

You can use the Parent.Periodic module in the Parent library to replace your Quantum code (it is a newer Quantum replacement and it has the ability to prevent overlapping runs build in).

5

We encounter the same behaviour in one project. Some Quantum defined methods run several times, even after restart or full deploy. Did you find any solution to this?