aviraj
Running quantum job from single node
We have elixir application deployed on two servers. The application has been configured with some quantum jobs which execute from both servers at the same time. We want to limit execution of the quantum jobs to a single server. As per documentation, we could see the following options,
- Add
global?: trueto the job configuration run_strategy: {Quantum.RunStrategy.Random, :cluster}
We tried option (1), it didnt work. The job still executed from both servers. Do both (1) and (2) need to be set at the same time in a job? or there something else missing in the configuration
config :titan, Titan.Scheduler,
timeout: 20_000,
global?: true,
jobs: [
job_schedule: [
schedule: "20 6 * * *",
task: {Titan.ContentSchedule, :schedule_updates, []}
],
]
How does this clustering work? Even without giving the list of nodes, how do the two servers communicate among themselves to co-ordinate the job execution?
Most Liked
connorlay
Hey! I recently solved this problem of running globally unique jobs in a clustered Elixir application. The project was written using Quantum 2, which had built-in support for clustering via the global: true mode. In our experience the implementation of global jobs was unreliable and we found that many jobs would stop executing entirely.
After doing some research, we ended up moving from Quantum to periodic for the job scheduler, combined with highlander to ensure uniqueness in the cluster.
I have a proof-of-concept here that shows the basic functionality.
poops
I had this same issue where I had to move off Oban because it locked a table and took our entire application down. Moved to quantum 3 and used highlander to have quantum run on only one process:
https://github.com/quantum-elixir/quantum-core/issues/411#issuecomment-652733990
connorlay
Fixed! Thanks for pointing that out ![]()
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance









