Hi, I’m particularly hoping to get the attention of OTP experts like @sasajuric here

I’ve found myself starting to write significant amounts of code where basically I start a genserver, which in turn starts a couple more genservers in the init phase. I then store the pids of these genservers in my genserver and … hope for the best

Its gradually dawning on me that this may not be optimal and I’m having trouble getting my application to stop, which I suspect is due to dangling processes

I guess, 2 questions:

1. How should I best rearrange my app to avoid this situation?!
2. I presume that it’s sometimes valid to implement as I have done. However, what do I need to do to make it “safe” and idiomatic? Particularly I think I have corner cases at present if the top genserver is shutdown with :normal, possibly also during an immediate/kill shutdown? And I guess potentially other corner cases?

I will try and present a more concrete example.

I was writing a genserver which monitors a serial port. So I made use of the Nerves.UART library, something like as follows:

defmodule UARTMonitor do
use GenServer

  def start_link(args \\ []) do
    ..
    GenServer.start_link(__MODULE__, args, opts)
  end

  def init(args) do
    {:ok, pid} = UART.start_link()
    :ok = UART.open(pid, args[:uart], some_other_args)
 
    state = %{
      uart_pid: pid,
      other_state: blah
     }
  end

  ...
end

Now, having reviewed some posts by Sasa, and I’m skimming through the Elixir language example and also Elixir in Action ch9 (I confess I read it a long while back and it’s only suddenly starting to make sense now…). These make me think that I should be conforming with a general handwaving guide to always start things through a supervisor, hence my code above feels “wrong”?

Should I have created a DynamicSupervisor and had that call my UART.start_link() ?

However, such a change potentially increases boiler plate quite a bit and leaves me unsure how to always create the semantics that I want?

a) If I want the supervisor to restart my UART process if it dies, I guess then I need a process registry to give me some kind of consistent PID naming (so I can keep calling functions in that process)? However, in the event of spawning multiple processes, it seems sometimes difficult to invent unique dynamic names?

b) I’m not sure how to create the semantics that: if the main process dies, it should kill the UART process also? I guess I would start the UART under (say) a DynamicSupervisor, then just do a “link()” to my own process?

c) What if I needed to guarantee some cleanup that must be done if the UART process stops (not just crashes)? Do I need a third process linked to the UART or can I reliably catch exits from the top process (as it dies) and cleanup the UART process?

d) How does shutdown happen? I can’t get my head around whether it’s enough to start the (say) DynamicSupervisor for the UART after the top level process? Thinking about ensuring hypothetical cleanup runs correctly (imagine it was important to squirt some “bye” message down the UART before closing it)

Going in the other direction, what do I need to do to make the current situation “safe”? Do I need to catch exits? What conditions need to be handled (assuming that if the main process is going down it needs to stop the UART process as well?) Is anything else needed?

There must be some good articles on “how to do this stuff”? It seems like I’m missing some real 101 getting started? It does feel as though things could sometimes be simplified if there was a form of start_link() which would also shutdown dependent processes in the case of a :normal shutdown?

I think I’m on the right track to proceed with (in general) “start everything through a supervisor”. So my next thought is about how to construct libraries and whether there are ways that the library can wrap some or all of these concerns? In general should a library provide some of the supervisor pieces, and if yes, how to offer those in a way that can be inserted into the applications supervisor tree?

I would appreciate some thoughts on how to best structure libraries which need a resource checkout to create some kind of usage handle? So this is your database library and the like. So we need a coordinator which will do some work to acquire a handle, we will create a process to store this handle and do the actual work with the handle. Another process will request the resource.

My thought is that this is:
ResourceAllocator module - does the work to figure out a handle, starts a process through:
ResourceDynamicSupervisor - holds the resource processes

A process needing a handle will call into the ResourceAllocator, which will both start_link the new process into the DynamicSupervisor, but also monitor/link with the calling process (as appropriate).

Is this a good pattern to be using?

If yes, why don’t more libraries include the supervisor part in their implementation? Why didn’t the Nerves.UART library choose to include a supervisor to track these processes? Nothing is black and white, but would it generally be a good strategy to include some kind of dynamicsupervisor in the library?

There is good documentation on Supervisors and Genservers, but I feel we could benefit from more description on good patterns for using these building blocks, especially over how to allocate and cleanup resources at runtime and shutdown. Is it a common pattern to build a new supervisor module which wraps and provides utility functions to start processes (I’m thinking of TaskSupervisor), or is it more normal to keep a separate module for managing this?

If I look at a lot of libraries in the wild it seems like it’s most common to provide just a start_link() function, and leave the library user to figure out how to add to a supervisor tree. However, if I set out to design something which would immediately be stuffed into a DynamicSupervisor, then I would probably structure the implementation quite differently? Possibly even offering a custom supervisor module in the style of TaskSupervisor?

Why is the TaskSupervisor style of implementation not the defacto interface for many libraries? Why are we more commonly offering an interface to start processes and not an implementation to start and supervisor a process in one go? I realise that many cases it will be useful to design a custom supervision strategy, but I sense that a significant number would not?

Anyone else have any good articles on how to build solid apps? How to structure apps which need to start multiple instances of things, how to structure the layers of supervisors and what technique used to manage/wrap the starting of the processes?