Dome
Hi everybody, this is my first time here so sorry if I make some mistakes.
I’m working on a little project for my thesis. During my study on Elixir I found the swarm library and it suites perfectly on what I need to do. The problem is that I have some troubles using it:
- Process state handoff: I have a GenServer with a very simple struct and I need to distribute this worker around my cluster. When the node on which the worker is running goes down, Swarm resume the process on a new node but I cannot manage to resume the state. I’ve followed the worker example provided by the original author on github but it seems to not work. Here’s my code:
defmodule MyApp.Worker do
use GenServer
defstruct elements: %{}
def start_link(opts \\ []) do
GenServer.start_link(__MODULE__, opts, name: __MODULE__)
end
def init(opts) when opts == [] do
state = %EventStore.Worker{}
{:ok, state}
end
def init(opts) do
state = opts
{:ok, state}
end
def handle_call({:swarm, :begin_handoff}, _from, state) do
{:reply, {:resume, state}, state}
end
def handle_call({:swarm, :end_handoff, state}, _from, state) do
{:noreply, state}
end
def handle_cast({:swarm, :end_handoff, state}, state) do
{:noreply, state}
end
def handle_cast({:swarm, :resolve_conflict, _delay}, state) do
{:noreply, state}
end
def handle_info({:swarm, :die}, state) do
{:stop, :shutdown, state}
end
end
More details. I’ve used Swarm.register_name(workername, workername, :start_link, [opts]) for starting my GenServer and I’ve tried to kill the node via double CTRL+C and via :observer.start too. I really don’t know what is the problem since I’ve also looked at the Swarm tracker.ex file and saw that there’s a call with :begin_endoff which I (think) correctly handle in my GenServer. Indeed if I do the same call manually via shell, my worker handle it.
- Same worker on more nodes: if I have a cluster of 3 or more nodes and I would like to run the same process on more nodes at the same time, how can I do? Because after the first registration using the same Swarm function mentioned above, Swarm warns me that there’s already that registered process.
Trending in Questions
Hey guys,
I’ve got a huge CSV ( around 10 GB ) that needs to be processed hourly
Do you guys have any suggestions what is the best prac...
New
Hello!
Could someone please give me a help/sample code, how to delete a file from s3 using waffle/waffle_ecto from Phoenix app.
I creat...
New
I have what I’ve heard referred to as a “lookup table” in my database. This is a way of assigning codes to common values. One common lo...
New
What approach to take when sending live updates to “random” users Hi! I have a question, I have a little chat app, and when I create a DM...
New
I’m seeing that a list inside a Kino.DataTable will be interpreted as a charlist, even if the Kino.configure() is set to charlists: :as_l...
New
Anyone here using Honeybadger?
My Honeybadger account is being overwhelmed with noise from some bots. Seeing a lot of
Bandit.HTTPError...
New
Hi, I’ve just set up an application with ash_authentication. There is only magic link strategy for now, so there is no confirmation add o...
New
Other Trending Topics
I am happy to introduce the very α version of the new programming language compiled to BEAM.
Welcome Cure.
It has literally three kille...
New
Hobbes is a low-level distributed database for the Elixir programming language.
Hobbes provides a simple, safe, and scalable storage lay...
New
ExRatatui lets you cook up rich terminal UIs in Elixir, powered by Rust’s ratatui via Rustler NIFs. Build interactive terminal applicatio...
New
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve.
They are GUI (Emerge) and State management (S...
New
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
There are three potential reasons for members of this forum to have a look at https://vutuv.de
You are tired or annoyed of LinkedIn.
Yo...
New
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #blog-post
- #elixir-ls
- #ai
- #elixirconf-us
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #hex
- #security
- #metaprogramming










Showing Posts 1 to 7- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
SophisticaSean
I’m not super well versed with Swarm but I think that handoff only happens if the node that is going down is allowed to gracefully exit. It sounds like you were force killing the node. If you force kill the node it has no time to do this handoff. You can try
:init.stop()as described here: Graceful shutdown on SIGTERM? - #3 by cjbottaro to see if that graceful shutdown triggers the handoff.Dome
Thank you, I will look at that!
But now my doubt is: at this point it’s not a bit useless in a real case scenario? I mean, a node could go down for many cases without have the proper time to save tha state (unless implementing a sort of periodic backup).
SophisticaSean
I’ve been running Elixir in prod for 3 years now and it is exceedingly rare for a healthy node to explode. The 99.99 percent most likely cause for a node shutdown is a deploy. During a deploy you can give your old nodes some time to gracefully shutdown and utilize the Swarm handoff use case.
Elixir/erlang is very resilient when it comes to crashing and unhandled errors. That being said, I am a firm believer of writing code that can rebuild it’s state in the worse-case scenario. So Swarm is a nice to have but not the only line of defense when it comes to rebuilding whatever state we had stuffed in Swarm.
mpope
Erlang is resilient, we’re all at the mercy of the Linux OOM kill and machines being recycled by a cloud provider. I think it is worth investing in a ‘phone home’ for workers who are unable to recover their state, if it isn’t being distributed by mnesia or by other means.
mpope
For number 2, you’re going to have to add some randomization. Something like
works well for naming things.
tristan
Exactly. If you want to protect against losing state when losing nodes you need to either replicate it or simply store it in external storage. In Erleans (based on Microsoft Orleans) the latter is the default, GitHub - erleans/erleans: Erlang Orleans · GitHub. The state for a grain is saved to a storage provider and will be reloaded when reactivated.
Erleans also has stateless grains which may be a solution to your second question about the same process on multiple nodes, but not positive.
mpope
My favorite usage of lasp