The process - deep dive into Elixir processes

ityonemo · February 14, 2021, 9:00pm

I’m doing a 7-part screencast that deep-dives into how Elixir (yup, that’s elixir, not erlang) does processes.

Prerequisites are understanding Elixir syntax, content ranges from beginner to “ready to become and advanced Elixir programmer”.

eksperimental · February 15, 2021, 3:30am

Amazing introduction.
I really like the way you explain things.
keep up the good work, looking forward to the next video.

ityonemo · February 15, 2021, 4:16am

Part two: Tasks

(The last five chapters will be coming in… A bit less rapidly )

ityonemo · February 28, 2021, 8:52am

Part 3a: Conundrum with Concurrency.

Strategies for sharding global shared state

soyjeansoy · February 28, 2021, 1:55pm

great content btw, really helped me deep dive into processes

McTheels · February 28, 2021, 8:44pm

victorferreira · February 28, 2021, 10:07pm

Amazing series! Can’t wait for the following episodes.

Aetherus · March 1, 2021, 11:37am

I love it! And I did learn something from the series. Thank you.

RudManusachi · March 3, 2021, 5:13am

This is awesome! Thank you!

However, I noticed that it’s recommended to run tasks under Task.Supervisor… But in that case looks like it loses $callers and Mox fails to define an expectation in the caller’s process and I have to :set_mox_global and forget about running tests with async: true

Any suggestion?

ityonemo · March 3, 2021, 6:11am

Haha you got ahead of me I will address that my solution is that Elixir Std Lib should support that out of the box (and it would also be nice to have GenServers forward callers via an option too). If you absolutely must have a solution, I wrote this library: Multiverses.DynamicSupervisor — multiverses v0.7.0

Note that a Task.Supervisor is actually a DynamicSupervisor

ityonemo · March 3, 2021, 9:05am

Hm. I took a second look at it and Task.Supervisor.start_child should be setting callers correctly:

github.com

elixir-lang/elixir/blob/d30c5c0185607f08797441ab8af12636ad8dbd7e/lib/elixir/lib/task/supervisor.ex#L423



Similar to `start_child/2` except the task is specified
by the given `module`, `fun` and `args`.
"""
@spec start_child(Supervisor.supervisor(), module, atom, [term], keyword) ::
        DynamicSupervisor.on_start_child()
def start_child(supervisor, module, fun, args, options \\ [])
    when is_atom(fun) and is_list(args) do
  restart = options[:restart]
  shutdown = options[:shutdown]
  args = [get_owner(self()), get_callers(self()), {module, fun, args}]
  start_child_with_spec(supervisor, args, restart, shutdown)
end

defp start_child_with_spec(supervisor, args, restart, shutdown) do
  # TODO: This only exists because we need to support reading restart/shutdown
  # from two different places. Remove this, the init function and the associated
  # clause in DynamicSupervisor on Elixir v2.0
  # TODO: Once we do this, we can also make it so the task arguments are never
  # sent to the supervisor if the restart is temporary
  GenServer.call(supervisor, {:start_task, args, restart, shutdown}, :infinity)

Exadra37 · March 3, 2021, 10:16am

Not wanting to hijack the post, but if I am not in mistake @josevalim said once that most probably we wouldn’t have an Elixir V2.0, but after all it seems to be planned one???

LostKobrakai · March 3, 2021, 10:30am

There are a bunch of those comments in elixir core, but I guess not because it’s planed, but because it’s useful if it eventually becomes an option.

RudManusachi · March 4, 2021, 8:12am

Huh, right! Thank you for clarifying! Tried on simple example it worked as expected!

In the project I’m working on turned out the issue was in a different module where async processing is implemented via GenServer handle_info… I misread the error message

Thank you for bringing it up here!
I just replaced stateless GenServers with supervised tasks and tested with Mox with async: true (yay!)

ityonemo · March 4, 2021, 8:21am

Each time someone changes a linear-flow GenServer to a Task, an angel gets their wings lol.

Aetherus · March 4, 2021, 9:10am

I heard that some part of the Phoenix framework is some processes that are not GenServers but still can be supervised by a supervisor. I wonder what’s the minimal requirement to make a process that can be supervised.

LostKobrakai · March 4, 2021, 9:19am

There are many non GenServer processes, which can be supervised: Task, :gen_statem and some other :gen_* behaviours. Basically there are plain processes, which are just barebone, then there are processes spawned by :proc_lib, which also adhere to OTP design principles and then there are higher level abstractions and behaviours, which often use :proc_lib internally. Those last ones are the tasks, genservers, statemachines, ….

Aetherus · March 4, 2021, 9:27am

Then how many messages should a process be able to handle so that it can be supervised? None?

Also, what messages should a supervised process be able to spit out? On what condition?

LostKobrakai · March 4, 2021, 9:29am

Those things are afaik documented in the docs for :proc_lib/:sys, like here:

Erlang -- sys and proc_lib

Not sure if there’s more, I never did a deep dive into the topic.

Aetherus · March 4, 2021, 9:31am

Thanks a lot for the documentation. That’s fascinating. I’ll give it a shot (in Elixir, of course) this weekend.