For exercise’s sake, I am trying to make a home-grown pool-like parallel job runner. Imagine if I start 2 GenServers under a Supervisor and then want to enqueue 11 tasks to them; the way I’d do it is to use chunking and put 6 tasks on the call queue of the first server and then 5 on the call queue of the second server. In my mind this ensures always only 2 tasks will run due to the serial nature of GenServer message processing.
Note that I want to enqueue work from multiple processes; if it was only from one process then Task.async_stream or Task.Supervisor.async_stream[_no_link] would have been enough. Sadly their max_concurrency option does not offer VM-wide limitation of maximum amount of concurrently running jobs.
(Or maybe I am XY-problem-ing this, do tell me if you feel like I do – it’s just the first idea that popped in my mind and I decided to pursue it for the time being. Maybe I missed a readily available builtin API for exactly my purpose?)
Probably because I am coding at 6:00 AM shortly before bed but I can’t seem to be able to use a GenServer.call with a name that seems to me is explicitly given in GenServer.start_link.
Application
defmodule Zyx.Application do
@moduledoc false
use Application
@impl true
def start(_type, _args) do
children = [
Zyx.Iterator
]
opts = [strategy: :one_for_one, name: Zyx.Supervisor]
Supervisor.start_link(children, opts)
end
end
Supervisor
defmodule Zyx.Iterator do
use Supervisor
def start_link(init_arg) do
Supervisor.start_link(__MODULE__, init_arg, name: __MODULE__)
end
@impl true
def init(_init_arg) do
children = [
%{
id: :zyx_worker_0,
start: {Zyx.Worker, :start_link, [:zyx_worker_0]},
type: :worker
},
%{
id: :zyx_worker_1,
start: {Zyx.Worker, :start_link, [:zyx_worker_1]},
type: :worker
}
]
Supervisor.init(children, strategy: :one_for_one)
end
end
GenServer
defmodule Zyx.Worker do
use GenServer, shutdown: 30_000
# Server (callbacks)
@impl true
def init(name) do
{:ok, name}
end
@impl true
def handle_call({:run, fun}, from, state) when is_function(fun, 0) do
IO.puts(
inspect(DateTime.utc_now()) <>
": worker " <> inspect(state) <> " called from " <> inspect(from)
)
Process.sleep(1000)
{:reply, nil, state}
end
# Client
def start_link(name) do
GenServer.start_link(__MODULE__, name: name)
end
end
For a first iteration I simply want to be able to hop in iex and do this:
GenServer.call(:zyx_worker_0, {:run, fn -> IO.puts("0") end})
But this fails with:
** (exit) exited in: GenServer.call(:zyx_worker_0, {:run, #Function<45.65746770/0 in :erl_eval.expr/5>}, 5000)
** (EXIT) no process: the process is not alive or there's no process currently associated with the given name, possibly because its application isn't started
(elixir 1.13.2) lib/gen_server.ex:1019: GenServer.call/3
Note that if I fetch the two worker IDs with Supervisor.which_children(Zyx.Iterator) and then use :erlang.list_to_pid to construct the PIDs manually, and then use GenServer.call(pid, ...), this works without a hitch.
What doesn’t work is calling the GenServers by name. And I am pretty sure I am missing something mega-obvious. 
Will you help a guy resolve his brain fart?
Additional ideas on how to approach a home-grown parallel job runner with VM-wide maximum concurrency are also welcome (but I admit I got a bit triggered by not being able to code this in 30 minutes so I’d still want to see it done exactly my way first, just to prove a point 
).
