DynamicSupervisor starting child with more than one argument

Questions / Help
1

Hey there guys,

I’m trying to use a DynamicSupervisor to start multiple children, however I need to register them with a specific name. However from the documentation there seems to be no way of doing this, unless I pass a map to the start_child/2 function:

DynamicSupervisor.start_child(pid, {Pipeline, job_module})

What would be the best way of starting and registering this child with a name?

2

The second element of the tuple can be a keyword list, where you would pass multiple options:

{Pipeline, job_module: job_module, name: ...}

In case the Pipeline module does not support the new child specs, then you need to build a child specification by hand:

%{
  id: Pipeline,
  start: {Pipeline, :start_link, [job_module, [name: ...]]
}
3 Likes
3

Hi José,

Thanks for replying!

I was trying to keep the Pipeline.start_link/2 function standard with arg + options arguments. With your suggestion I would have to change the start_link function, right?

Here’s my Pipeline module if that helps: https://gist.github.com/05419f6bbfd9b5a6e037f809eb716a9f

4

So I’ve fixed this as suggested by @josevalim by changing my Pipeline.start_link/2 signature from:

Pipeline.start_link(job_module, options \\ []), do: GenServer.start_link(job_module, options)

which mimics the GenServer (which it is) start_link/2 function to:

Pipeline.start_link(job_module: job_module, options: options), do: GenServer.start_link(job_module, options)

which is fine and I am now able to start it using the DynamicSupervisor, however I do find it weird that I need to go away from the standard start_link function signature.

Is this intended or am I doing something wrong?

5

I interpreted his advice to go with the fully featured map-based version of the spec:

%{
  id: Pipeline,
  start: {Pipeline, :start_link, [job_module, [name: ...]]
}

rather than the tuple short form, not necessarily modifying start_link.

GenServer (which it is) start_link/2

FYI: actually it’s start_link/3

start_link(module, args, options \\ [])

Ultimately it’s this line

  start: {Pipeline, :start_link, [job_module, [name: ...]]

that determines how the child process is started. Pipeline is simply the module that owns the startin function, :start_link identifies the function to “start” the process (it could be set :fred - and then you better provide a fred function) and the list that follows are the arguments to be used when calling start_link (or fred).

So there really is no default signature - the tuple spec assumes a name of start_link but that is a mere convention but there is no fixed arity.

Ultimately the function is run with something like Kernel.apply/3 - so the number of elements in the args list ties into the function’s arity - keeping in mind that the last element can be keyword list to mimic variadic arguments (I personally find it confusing when the enclosing brackets are left out “for convenience”).

Just to crudely demonstrate:

defmodule Pipeline do
  use Supervisor

  def init(args) do
    {:ok, args}
  end

  defp queue_name(job_module),
    do: String.to_atom("#{job_module}-queue")

  def spec_map(job_module),
    do: %{
      id: Pipeline,
      start: {
        Pipeline,
        :fake_start_link,
        [job_module, [name: queue_name(job_module)]]
      }
    }

  def fake_start_link(job_module, opts) do
    IO.puts("job_module: #{inspect job_module}");
    IO.puts("opts: #{inspect opts}")
    // GenServer.start_link(job_module, [], opts)
  end

end

job_module = Queue;
spec = Supervisor.child_spec(Pipeline, Pipeline.spec_map(job_module))
IO.inspect spec
{m,f,a} = spec.start
Kernel.apply(m,f,a)

$ elixir demo.exs
%{
  id: Pipeline,
  start: {Pipeline, :fake_start_link, [Queue, [name: :"Elixir.Queue-queue"]]},
  type: :supervisor
}
job_module: Queue
opts: [name: :"Elixir.Queue-queue"]
2 Likes
6

Yes, exactly. The preference should be to use start_link/1. This requires you to pass everything as an option. Use Keyword.fetch!/2 for required fields. This way you can use the “tuple child specifications”.

If for some reason that is not possible and really can’t be made possible, then use the map version.

The faster everyone standardizes on start_link/1, the sooner the problem will disappear. :slight_smile:

7

Thanks guys, that’s very useful.

I wonder if things like GenServer and friends will eventually move to a start_link/1 function as well?

8

The way I see it start_link/1 is primarily intended for the general case where modules implement the GenServer behaviour themselves. In that case the module can provide its own child_spec and the module’s “start” function can specify it’s own start options via GenServer.start_link/3. The single argument is a keyword list for init(opts) (the opts name hints at it being a keyword list - not to be confused with options from start_link/3 which contains the “start options”).

GenServer.start_link/3 on the other hand is in a league of it’s own. Apart needing to be supplied with the starting module when it is used inside module based GenServers - Replacing GenEvent by a Supervisor + GenServer shows in interesting indirect use:

EventManager uses a generic child spec

%{
  id: GenServer,
  start: {GenServer, :start_link, []},
  type: :worker,
  restart: :temporary
}

which gets instantiated as

%{
  id: GenServer,
  start: {GenServer, :start_link, [handler, opts]},
  type: :worker,
  restart: :temporary
}

which translates to

GenServer.start_link(handler, opts, []) # opts are handler arguments as a keyword list - not starting options

which initializes a new process through handler.init(opts) - the handler module doesn’t even have to use GenServer it just has to implement all the right GenServer callbacks.

So in your case you may have been able to use a child spec like:

%{
  id: GenServer,
  start: {GenServer, :start_link, [job_module, opts, [name: queue_name(job_module)]]},
}

instead of having a separate Pipeline.start_link function.

9

Thanks for the in-depth example @peerreynders.

I understand that, however, I do appreciate the separation of concerns you get with a start_link/*+1. Where runtime options are the last argument preceded optionally by any arguments you might need for your business case. This is seen across most of the standard lib & OTP: GenServer, Agent, Registry, etc.

But this is probably just my mind trying to stick with what is familiar, I would however like to see some standardization so that we could mostly use Tuple specs everywhere. This could happen either by making the standard lib implement a start_link/1 on every module (i.e. GenServer could implement something along the lines of GenServer.start_link(mod: MyGen, name: MyGen, arg1: 1)), or eventually allow a tuple of dynamic size so that we use it to start modules that implement start_link/n functions where n is 2 or more, such as: {GenServer, MyGen, 1, name: MyGen}.

10

I don’t think that will happen because you always need to implement the GenServer callbacks and that’s when you will wrap its start_link/3 by your own start_link/1. I would never expect somebody to start a {GenServer, …} directly in a supervision tree.

2 Likes