autodidaddict

autodidaddict

Author of Real World Event Sourcing

I’ve reviewed the other topics here and done a bunch of googling and haven’t been able to find out the right way to deal with this.

I have DynamicSupervisors , about 4-5 of them. Each of those spawns GenServers as children via DynamicServer.start_child.

What I then need to be able to do is expose a function in the supervisor that will terminate a child by key, so it needs to: a) look up the child (this works fine), b) terminate the child.

The problem is I can’t figure out how to “cleanly” and “idiomatically” terminate the child process. In my terminate_child_by_key(foo) function, what’s the accepted way of shutting that pid down? Not only do I need to shut that pid down, but I need to be able to publish a message on my broker so that I can emit the “child died” event.

Since I’m going to do this over and over again, I want to do it right. I’ve tried using Process.flag(..) in the child as a way of getting advance notice that the child is going to die but that handle_info never gets called. When I use process flag in the supervisor, the supervisor never gets called during child death.

Additionally, for this scenario, I can’t use transient children (?) because they come back immediately after I kill them… the behavior I want is when I choose to kill the child, it’ll stay dead, but when it dies due to exception failure, it’ll restart.

Any advice would be greatly appreciated as I’ve been doing this all in a very ugly fashion and I don’t want to continue repeating that same ugliness all over my code base.

Showing Posts 1 to 8

axelson

axelson

Scenic Core Team

Have you tried using DynamicSupervisor.terminate_child/2?

If you’re stopping a transient child cleanly then it should not be restarted.

ityonemo

ityonemo

I think Genservers already have that process flag set and are designed to trap exit messages and handle them with the terminate/2 callback.

ityonemo

ityonemo

also I would say the idiomatic thing to do in elixir is to spin up a Registry and send a command to the GenServer for them to just self-destruct (something like def handle_call(:stop, _, state), do: {:stop, :normal, :ok, state}). Trawling through a supervisor’s child list seems like a very ungraceful thing to do.

As an aside, note that the reason for termination matters. IIRC you should supervise :transient but note that the “reason” for shutdown matters; :normal or :shutdown will not trigger restart, everything else (exceptions, kills, brutal_kills, custom reasons) will.

al2o3cr

al2o3cr

If the process is actively handling messages (versus being blocked in a call or similar), consider adding an explicit “hey could you please shut down” message to the GenServer’s public API. The handle_call head for that can return {:stop, :shutdown, :ok, state} and the GenServer will exit with reason :shutdown.


Otherwise, signaling from outside is done via Process.exit/2 and functions built using it. It’s useful to understand exactly what “signaling a process to exit means”:

  • If any Erlang process gets an exit signal with a reason of :kill, it will exit immediately with the reason :kill.

  • If a process that isn’t trapping exits gets an exit signal, it will exit immediately with the same reason.

  • If a GenServer is trapping exits, the built-in handler from gen_server will invoke the terminate/2 callback with the reason. (more info)

(the above is summarized from The many and varied ways to kill an OTP Process | The log of Paul Wilson )

The idiomatic sequence implemented by terminate_child in DynamicSupervisor and Supervisor is:

  • Process.exit(pid, :shutdown)
  • wait for the :DOWN message
  • if it doesn’t arrive (default in 5s), Process.exit(pid, :kill)

Prefer using those functions over building custom Process.exit setups unless you have a real good reason.


Re: reanimating processes - restart: :transient will restart the process if it exits with a reason other than :normal, :shutdown, or {:shutdown, term()} - for instance, if the process doesn’t respond to the :shutdown signal and gets killed. If that’s undesirable, consider restart: :temporary instead.


One final note: pay close attention to the gotchas listed in the terminate callback’s documentation. If you need a 100% reliable “GenServer went away” hook, consider using Process.monitor and handling the {:DOWN, ...} message.

autodidaddict

autodidaddict OP

Author of Real World Event Sourcing

Thanks for the tip. I’ll try and combine the use of DynamicSupervisor.terminate_child and a registry and see if that gives me the kind of cleanliness of code I’m looking for.

ityonemo

ityonemo

Ah, just fyi you don’t have to combine them. If you use DynamicSupervisor.terminate_child, you don’t need to use Registry, and vice versa. I think there are two major differences:

  1. Difference 1: naming
  • For DynamicSupervisor.terminate_child strategy, the process that is responsible for starting the process must be able to assign its id.
  • For Registry, you can lazily “figure out what my id is” at launch time.
  1. Difference 2: performance.
  • Registry is going to be faster, because there isn’t a lookup that fetches information that has to be copied from another process (it uses ETS which is blazing fast, the only block is an internal mutex in C).

In either case, be mindful of the {:normal/:shutdown}/:kill/everything-else semantics with respect to restart logic.

autodidaddict

autodidaddict OP

Author of Real World Event Sourcing

Thanks for all the information here! I ended up deciding not to use Process.exit. Instead I followed @ityonemo 's suggestion of sending a “soft halt” by doing a GenServer.call(pid, :halt_and_cleanup) , this gives me the handle_call for this explicit type of termination, which in turn lets me publish the “child died” message on my broker and then safely return {:stop, :normal, :ok, state}.

I also adopted the use of the Registry and I’m now storing the child pids there along with their keys.

ityonemo

ityonemo

Nice! I feel like this is the idiomatic Elixir solution.

— All posts loaded —

Where Next? Top

Trending in Discussions Top

AstonJ
As the title says, please share what you’ve been up to with Elixir. Whether that’s been learning it, looking into it, making stuff with i...
2977 94592 917
New
cblavier
Hey there, It’s been more than a year since we started using LiveView as our main UI library and building a whole library of UI componen...
New
mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
heathen
Quite interesting article Google brought me. Didn’t find any mentions about it here. What do you think in general? Would you use togethe...
New
mhanberg
Hi everyone! The first release candidate for the Expert language server project is now available! We’ve published a press release detai...
New
AstonJ
Since we have deprecated our Erlang sections (as we have dedicated Erlang Forums now) let’s add this thread for those who’d like to post ...
New
maennchen
:warning: Security advisory: Decimal DoS vulnerability A vulnerability has been published for decimal where very large exponents can cau...
New

Other Trending Topics Top

marciok
Hi there! We created Gust: A task orchestrator inspired by Airflow. For those who have never heard about Aiflow, it’s a Python-based wor...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Dmk
Xamal is a deployment tool for Elixir apps that deploys native releases to bare metal servers over SSH. It’s a port of GitHub - basecamp/...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
webofbits
With AI doing more of the implementation work, I’ve been wondering how much coding I should deliberately keep doing myself. My main conc...
#ai
New
webofbits
Aludel - LLM Evaluation Workbench Aludel is an embeddable Phoenix LiveView dashboard for evaluating and comparing LLM prompts across mult...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews