Creating process from GenServer and passing state

venomnert

Context:

I am pretty new to OTP. In trying to learn it I’m creating an application that does the following (high level overview):

users submit X number tasks to a GenServer.
The GenServer iterates over the queue of tasks and creates a process for each task.
Once the process is completed it will notify the GenServer it’s done and return its value.
Once the GenServer get’s notified it will update the state with the process’s return value, and notify the client with the updated state.

The diagram demonstrates what I’m trying to achieve.

I am able to spawn a process from the GenServer; however, I’m struggling to connect between the GenServer and process, so the process can call the GenServer; in return, the GenServer can handle_call/3.

Questions:

Am I taking the right approach of calling a process from a GenServer? Is there better way to create process from GenServer and notify the process about the state?
How do I call GenServer from a process and passing the state to server?

6 comments

#processes #state #genserver

1 3200 6

2019-10-29 01:39:05 UTC

Showing Posts 1 to 6

Show Best Posts
Show All (oldest first)
Show All (newest first)

al2o3cr

This sounds a lot like the Task functionality in the Elixir standard library. Here’s some relevant discussion from 2018, complete with a code example:

The tricky part with spawning additional processes is making sure everything works sensibly when things go wrong - what happens when those processes shut down, when the parent shuts down, etc. The Task library wraps the relevant parts (linking, supervising and so on).

Post #1

venomnert

I believe this is what I needed, thank you @al2o3cr! I’m going to give @joaquinalcerro implementation a shot.

Post #2

joaquinalcerro

@venomnert,

Even though this code might work for your current use case you have to consider what happens if one of the tasks crashes.

With this code, if one task crashes, your GenServer will also crash with your state because the Task is still sending the GenServer a message that is currently not handled. You can handle them as follows:

   def handle_info({:EXIT, _task, _reason}, state) do
     IO.puts("Crashed!!!!!")
     {:noreply, state}
   end

  def handle_info({:DOWN, _ref, :process, _pid, reason}, state) do
    IO.inspect(reason)
    {:noreply, state}
  end

The first message will have the tuple {:EXIT, task, reason}. The other one is {:DOWN, _ref, :process, _pid, reason}. You already have a tuple similar to this one but handles the happy path when the task exits normally like this {:DOWN, _ref, :process, _pid, :normal}… note the :normal atom at the end will be pattern matched.

With this in place, your GenServer will not crash and will be available to handle other tasks.

So this might work but: I just wanted to share with you some recommendations @whatyouhide gave us during this weeks ElixirConfLA in Medellin.

He recommended us:

Limit the amount of tasks that can be spawned to avoid exhausting the resources
Use the proper strategy for your use case:
a. one_to_one
b. one_for_all
c. rest_for_one

So what happens if one of the task fails? is it worth continue processing the other tasks? how will your state be affected if one task fails?

Check this link for details of each one: Supervisor — Elixir v1.20.2
Nest you supervision tree with other supervisors with the proper strategy. For example, under your main application supervisor, create a worker supervisor with a one_for_one strategy.
All processes should be supervised
Name your supervisors to access them by name which is easier.
Test you supervision tree

I will be monitoring when the talks are published to link it to this thread.

Best regards,

Post #3

joaquinalcerro

You can also check this thread which has a complete example and explanation:

Best regards,

Post #4

venomnert

Note

I just wanted to say thank you for taking the time to providing me with extra detailed information.

My understanding

I notice the main difference between the handle_info in the other thread and the one you have mention on this thread is the following:

Here we are accepting any reasons for :EXIT and :DOWN.
In the other post we are accepting a specific case of :EXIT and :DOWN.

Questions

However, I’m still unclear as to the reason why we are not pattern matching :normal?
In my case is it better to have a single task that handles all of the user submitted tasks or is it better to have a task per user submitted tasks?

Limit the amount of tasks that can be spawned to avoid exhausting the resources

Also can you point me towards a resource on testing tasks and supervisor tree?

Test you supervision tree

Post #5

joaquinalcerro

@venomnert,

So we are pattern matching the tuples with :EXIT and :DOWN one with the :normal atom and the other one with any reason (you need to pattern match all the cases). The normal case will match when the task finishes successfully without crashing and the other one when it crashes for any reason.
As always, depends. If you are having lots of users request to a single GenServer, it might become a bottleneck and you might be better of spawning a GenServer per user request. If that is not the case, you might be already set. Another question: what happens to the state if two users send tasks to the GenServer at the same time? should the state be affected by both user’s tasks? or do you need to have different state for each user request… if this is the case, you might need to spawn a GenServer per user request which will also spawn tasks async.
Well Andrea recommended this site: GitHub - ferd/sups: PropEr model helper library to validate implementations of supervisor trees · GitHub … but he also said it was a bit hard to implement. Having said that, you could use the Observer (in iex session, call it by " :observer.start() ") to see your supervision tree and kill the process to see how is it working.

Hope this helps.

Post #6

— All posts loaded —