lessless

lessless

How to start dynamic supervisor workers on application startup?

Hello,

In our app, we have to start dynamic amount of workers and we can’t just define them as a part of supervision tree in the application’s start callback, because of using :simple_one_for_one strategy

use Application

 def start(_type, _args) do
    children = [
      supervisor(Registry, [:unique, Postman.AccountRegistry])
    ]
    account_supervisors = Enum.map(accounts(), &account_supervisor/1)

    Supervisor.start_link(children ++ account_supervisors, [strategy: :one_for_one, name: Postman.Supervisor])
  end

  defp account_supervisors(account),
    do: Enum.map(accounts, &(supervisor(Postman.Processor.AccountSupervisor, [&1.name], id: &1.name)))


defmodule Postman.Processor.AccountSupervisor do
  use Supervisor

  def start_link(account_name) do
    Supervisor.start_link(__MODULE__, nil, name: via_tuple(account_name))
  end

  def start_child(account_name, max_demand),
    do: via_tuple(account_name) |> Supervisor.start_child([account_name, max_demand])

  def init(_) do
    supervise(
      [worker(Postman.Processor.Stage.Account, [])],
      strategy: :simple_one_for_one
    )
  end

  defp via_tuple(account_name),
    do: {:via, Registry, {Postman.AccountRegistry, "#{account_name}-supervisor"}}

end

That means to spawn new workers with AccountSupervisor.start_child/2 we have to wait until supervisor finishes its initialization.

The only way around that I can think of is to add GenStage that will start after Supervisors and which solely job will be to spawn children. But is spawning order guaranteed, i.e. is it synchronous?

Maybe there is a whole better way get around that problem?

Most Liked

josevalim

josevalim

Creator of Elixir

Yup, this is the way to go. You don’t need a GenServer though, a Task would suffice. And you want to pass worker(Test.Starter, [], restart: :transient) so it is not restarted right after it terminates.

idi527

idi527

I don’t quite understand your problem, but why can’t you just put a genserver at the end of the children list, which will start all your accounts and terminate? Is there really a need for genstage?

application.ex

defmodule Test.Application do
  use Application

  def start(_type, _args) do
    import Supervisor.Spec, warn: false

    children = [
      supervisor(Registry, [:unique, Test.Registry]),
      supervisor(Test.Account.Supervisor, []),
      worker(Test.Account.Starter, [], restart: :transient)
    ]

    opts = [strategy: :one_for_one, name: Test.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

account/supervisor.ex

defmodule Test.Account.Supervisor do
  use Supervisor

  def start_link do
    Supervisor.start_link(__MODULE__, nil, name: __MODULE__)
  end

  def start_child(account) do
    Supervisor.start_child(__MODULE__, [account])
  end

  def init(_) do
    children = [
      worker(Test.Account, [])
    ]

    supervise(children, strategy: :simple_one_for_one)
  end
end

account/starter.ex

defmodule Test.Starter do
  use GenServer

  def start_link do
    GenServer.start_link(__MODULE__, nil)
  end

  def init(_) do
    send(self(), :start_children)
    {:ok, nil}
  end

  def handle_info(:start_children, _) do
    accounts = [:a, :b, :c]
    Enum.each(accounts, fn account ->
      Test.Account.Supervisor.start_child(account)
    end)
    {:stop, "iz kil"}
  end
end

Anyway, it’s just a suggestion. I also think there is a simpler way.

lessless

lessless

@idi527 you’re right - it should be GenServer there.

Thanks a lot, folks!

Where Next?

Popular in Questions Top

JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
Kurisu
For example for a current url like http://localhost:4000/cosmetic/products?_utf8=✓&query=perfume&page=2, I would like to get: ...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
hariharasudhan94
Lets say i have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => "XX...
New

Other popular topics Top

sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
985 42842 311
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
chrismccord
This release brings a number of exciting features, including integration with the new Phoenix LiveDashboard and Phoenix LiveView. There h...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New
KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 35953 110
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
sergio
Kind of like when jquery came out, it was super necessary. Existing drag and drop libraries have a bunch of baggage to support old browse...
New

We're in Beta

About us Mission Statement