Two supervisors starting the same worker

Questions / Help
1

Hi,

I have a service with a couple of independent applications running together on the same machine. But two of these both want to start the same kind worker to access the same database. This gives me the following error:

** (Mix) Could not start application my_app: MyApp.Application.start(:normal, ) returned an error: shutdown: failed to start child: Bolt.Sips
** (EXIT) already started: #PID<1.2.3>

It all works fine if I just uncomment the worker in one of the applications when I run them together but that would be to (quite heavily) violate their independece. I also would prefer not to put them in some kind of umbrella due to the same, or atleast a similar, reason.

Is there some way I can give them different names, only start a second if one is already running or something of that sort?

In case it is relevant I use {:bolt_sips, “~> 0.4.12”} to connect to a neo4j database

Started this way:

  def start(_type, _args) do
    children = [
      worker(Bolt.Sips, [Application.get_env(:bolt_sips, Bolt)])
    ]
    opts = [strategy: :one_for_one, name: MyApp.Supervisor]
    Supervisor.start_link(children, opts)
  end

With config similar to this:

config :bolt_sips, Bolt,
  hostname: 'localhost',
  basic_auth: [username: "xxx", password: "yyy"],
  port: 7431,
  pool_size: 5,
  max_overflow: 1

(Where MyApp represents the two different application names)

*and yes, I know having two applications sharing the same database seems a bit dodgy when it comes to independence but because of reasons it turned out this way as they actually are two separate concerns.

2

I’m making a guess that Bolt.Sips is a GenServer ? In Bolt.Sips you should have code that looks like:

def start_link(state) do
    GenServer.start_link(__MODULE__, state, name: __MODULE__)
  end

Note: the code above is the default and implicit if it isn’t listed in your GenServer module.

The fragment to focus on is name: __MODULE__ which names the process the same name as its module. Hence the error regarding Bolt.Sips already started since both applications start a GenServer with the same name.

You can pass in a different name for each application and change the above to:

def start_link(state) do
    name = get_name_from_state(state)
    GenServer.start_link(__MODULE__, state, name: name)
  end
3 Likes
3

You could move the part that starts up the Bolt.Sips supervisor to a third app and have both apps declare an umbrella dependency on it.

4

It looks like you will need to submit a PR to bolt_sips to allow you to customize the module name because right now it is hard-coded:

But at least it should be an easy and backwards compatible change.

1 Like
5

I’ve seen lots of examples of libraries that also accept an arbitrary set of genserver opts as well. That way someone can add multiple workers to their supervision tree and pass in name as well as other standard options.

def start_link(state, opts) do
  GenServer.start_link(__MODULE__, state, opts)
end

It’s not always easy to backport this to an existing library though, because the library will have to worry about how to find those genservers rather than always finding them via the hardcoded name.

3 Likes
6

Thank you for the thorough answer.

Being relatively new to elixir I wonder which is the preferred way to do this. Is it a /2 method such as

def start_link(opts, name) do
    Supervisor.start_link(__MODULE__, opts, name:  name)
end

or to include it in the opts like

def start_link(opts) do
    Supervisor.start_link(__MODULE__, opts, name:  opts.name)
end

(or something else)

1 Like
7

If name is optional then it should go in opts.

If name is not optional then it should go before opts.

Though honestly I quite often put non-optional arguments that are potentially ambiguous in opts as well and throw if they are missing, though this call is simple enough that it is not an issue here.

8

I would say that this form would be a good starting place:

def start_link(opts, name \\ __MODULE__) do
    Supervisor.start_link(__MODULE__, opts, name:  name)
end

Although assuming that opts is a keyword list you might do it like this as well (or possibly using Keyword.pop/3):

def start_link(opts) do
  name = Keyword.get(opts, :name, __MODULE__)
  Supervisor.start_link(__MODULE__, opts, name: name)
end
2 Likes
9

Eh, putting it after wouldn’t let it be used in the elixir pattern as easily though. If you want it in opts then I’d say:

def start_link(opts \\ [name: __MODULE__]) do
    Supervisor.start_link(__MODULE__, opts, name:  opts[:name])
end

Although if you are defaulting the name anyway then you could leave it out regardless unless you need to access it by name instead of PID.

10

I’d prefer the second form I showed earlier if you wanted the name in the opts (which is reasonable), although I guess I’d amend it to have a default value for opts of []. I don’t favor a default of [name: __MODULE__] because if you specify your own options for another key, you lose the default for :name.

i.e. start_link([num_workers: 5]) will result in opts[:name] == nil. Whereas if you use Keyword.get(opts, :name, __MODULE__) then you will have name == __MODULE__.

11

You can always use opts[:name] || __MODULE__ as well, pretty easy to specify a default that way. I like to put defaults in the head though because then it gets added to the documentation more explicitly. I keep meaning to write a library to simplify these patterns because I use named non-option and so forth arguments a lot and there is a lot of overhead in code to parse them out…

1 Like
12

Seems legit.

@axelson @OvermindDL1 Thank you both.

2 Likes
13

Yeah if there aren’t any options it would be nice for the options to appear in the head, although I don’t see that being worth losing the default value for a key if you pass in some options (or alternatively having to specify the default multiple times). I don’t see much difference between opts[:name] || __MODULE__ and Keyword.get(opts, :name, __MODULE__) (although if for some reason you wanted :name to have the value of false or nil you’d have to use the second form).

What I’d really prefer is to have the default show up in the functions @doc, which is perhaps what you’re alluding to with your library idea. From a quick search on hex.pm the only similar library that accomplishes this (and more!) I see is https://hex.pm/packages/optimal

1 Like
14

Yep exactly. ^.^

Oooo, I’ve not ran across that one yet! Thanks for the link!!

1 Like