How to gracefully handle GenServer terminating error in the caller

In the Phoenix application I am using Hound to scrape web page. Hound
is used within GenServer like

defmodule MyApp.Scraper do
    use Hound.Helpers
    use GenServer

    def start_link(_) do
      GenServer.start_lin(__MODULE__, [], name: __MODULE__)
    end

    def init(_) do
      {:ok, []}
    end

    def handle_cast({:scrape, msg}, state) do
      scrape(msg)
    end

    defp scrape(target) do
      Hound.start_session
      .....
    end
  end

Occasionaly Hound throws an error on Hound.start_session call

[error] GenServer Hound.SessionServer terminating
** (RuntimeError) could not create a new session: econnrefused, check webdriver is running
    (hound 1.1.1) lib/hound/session_server.ex:101: Hound.SessionServer.create_session/2
    (hound 1.1.1) lib/hound/session_server.ex:78: Hound.SessionServer.handle_call/3
    (stdlib 3.14) gen_server.erl:715: :gen_server.try_handle_call/4
    (stdlib 3.14) gen_server.erl:744: :gen_server.handle_msg/6
    (stdlib 3.14) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
Last message (from MyApp.Scraper): {:change_session, #PID<0.1627.0>, :default, []}
State: %{}
Client MyApp.Scraper is alive

    (stdlib 3.14) gen.erl:208: :gen.do_call/4
    (elixir 1.11.3) lib/gen_server.ex:1024: GenServer.call/3
    lib/my_app/hound_runner.ex:11: MyApp.HoundRunner.scrape/1
    (my_app 0.1.0) lib/my_app/scraper.ex:24: anonymous fn/2 in MyApp.Scraper.handle_cast/2
    (elixir 1.11.3) lib/enum.ex:2193: Enum."-reduce/3-lists^foldl/2-0-"/3
    (my_app 0.1.0) lib/my_app/scraper.ex:20: MyApp.Scraper.handle_cast/2
    (stdlib 3.14) gen_server.erl:689: :gen_server.try_dispatch/4
    (stdlib 3.14) gen_server.erl:765: :gen_server.handle_msg/6
    (stdlib 3.14) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
[error] GenServer MyApp.Scraper terminating
** (stop) exited in: GenServer.call(Hound.SessionServer, {:change_session, #PID<0.1627.0>, :default, []}, 60000)
    ** (EXIT) an exception was raised:
        ** (RuntimeError) could not create a new session: econnrefused, check webdriver is running
            (hound 1.1.1) lib/hound/session_server.ex:101: Hound.SessionServer.create_session/2
            (hound 1.1.1) lib/hound/session_server.ex:78: Hound.SessionServer.handle_call/3
            (stdlib 3.14) gen_server.erl:715: :gen_server.try_handle_call/4
            (stdlib 3.14) gen_server.erl:744: :gen_server.handle_msg/6
            (stdlib 3.14) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
    (elixir 1.11.3) lib/gen_server.ex:1027: GenServer.call/3
    lib/my_app/hound_runner.ex:11: MyApp.HoundRunner.scrape/1
    (my_app 0.1.0) lib/my_app/scraper.ex:24: anonymous fn/2 in MyApp.Scraper.handle_cast/2
    (elixir 1.11.3) lib/enum.ex:2193: Enum."-reduce/3-lists^foldl/2-0-"/3
    (my_app 0.1.0) lib/my_app/scraper.ex:20: MyApp.Scraper.handle_cast/2
    (stdlib 3.14) gen_server.erl:689: :gen_server.try_dispatch/4
    (stdlib 3.14) gen_server.erl:765: :gen_server.handle_msg/6
    (stdlib 3.14) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
Last message: {:"$gen_cast", {:scrape, %{target: "xyz"}}}
State: %{last_scrape_time: ~U[2021-05-21 16:18:43.688680Z], retry: []}

Which, obviously, causes my GenServer to restart. I know I can keep
the state separately and retry everything on the next restart but is
there some way to handle this Hound crash? In general, is caller always
supposed to crash when calling GenServer.call raises error or there is
a way I can handle it if such error is expected in cases when calee is
not explicitelly started and linked with caller.

2 Likes

The way OTP processes are linked, yes, it’s expected for the caller to crash.

As for your error, it looks like you’re having a “connection refused” error. IMO it’s completely fine for that process to die and get restarted since in a future call the connection will succeed. You might however want to tune the parameters of how many times it’s OK for a child process to die before the parent supervisor is killed as well.

Connection fails because, for some reason, Selenium needs to be restarted after few scrapes. But my GenServer is neither parent supervisor not linked process. It is just caller of Hound function.

That’s why I wanted to catch Hound crash without crashing my GenServer, restart Docker container and restart scrape. But seems like I will just have to handle that in terminate callback and reinitiate scrape message.

You are able to handle exits if you wish?

However, I think in this case it’s just a runtime error? So you can catch it with try do ?

Maybe a slightly more resilient setup for you would be to have a single scrape worker (or a pool of them) and your functions can just send it/them messages? That way if a worker crashes it will just be restarted and your calling process won’t get killed. Takes a little setup but it works fine when you achieve it.

No, try…rescue does not work in this case and probably the best solution is, as @dimitarvp wrote, to have scrape worker and send messages to it.