Unexpected :DOWN message received by GenServer using Process.send_after

Rodeoclash

I have an odd problem with a GenServer receiving a :DOWN message when I’m not expecting it. The GenServer itself is responsible for polling an external service every n number of minutes and making sure that I have a matching monitored process in my Elixir app. The file itself can be viewed here:

gitlab.com/Rodeoclash/StageHand

app/lib/app/kubernetes_v2/monitor.ex

master

defmodule App.KubernetesV2.Monitor do
  alias App.{KubernetesV2, Previews}
  require Logger
  use GenServer

  @requeue_time Application.get_env(:stagehand, :kubernetes_resource_poll_interval)
  @kubernetes_v2 Application.get_env(:stagehand, :kubernetes_v2)

  def start_link do
    GenServer.start_link(__MODULE__, nil)
  end

  def init(state) do
    work()
    queue()
    {:ok, state}
  end

  def handle_info(:work, state) do
    work()

This file has been truncated. show original

It is started in my application.ex using worker(App.KubernetesV2.Monitor, []).

Every now and again it will receive a message similar to: {:DOWN, #Reference<0.3921202181.3305373697.88131>, :process, #PID<0.7999.9>, :normal}. From my reading, this message should be sent when a process monitored has crashed or otherwise gone down. My question is, why is this server receiving the message? As far as I can tell, it shouldn’t be monitoring anything.

4 comments

/phoenix #ecto #troubleshooting

0 1846 4

2019-11-11 20:26:07 UTC

Showing Posts 1 to 4

Show Best Posts
Show All (oldest first)
Show All (newest first)

david_ex

That depends on what you mean by “otherwise gone done”: a :DOWN message is sent when the monitored/supervised process exits. It’s important to realize that the exit message will also be sent when a process exits normally (i.e. NOT having crashed), which is what is happening in your example: the :normal reason indicates that the process has exited “successfully”.

Basically, for what you’re attempting to do, you need to match the :DOWN messages in the handle_info: only report the exit reasons that aren’t :normal (if that’s indeed what you’re aiming for).

Post #1

Rodeoclash

Sorry, that was a silly typo. It should be “gone down”.

I’m more curious about why I might be getting the message at all, rather than trying to match on it and handle it. Another bug I’ve been working through might be related to this. If an HTTPoison (or other process) launched from this GenServer crashes, could I expect the :DOWN message to originate from that?

Post #2

david_ex

Let’s quickly recap what :DOWN messages are and how they work, just to make sure there’s no confusion there.

A given process will typically receive a :DOWN message because:

a process it was monitoring has exited
a linked process has exited and the current process is trapping exits

Notice that the condition for getting a :DOWN message is simply “process exited” and NOT “process crashed”.

Now that that’s out of the way, let’s go over some points in your post that may help you debug.

why I might be getting the message at all

You were either monitoring the process, or were linked to it while trapping exits. As far as I can tell, you’re not trapping exits in the GenServer, therefore the :DOWN message is likely due to a monitored process exiting.

rather than trying to match on it and handle it

Usually, if you’re interested n :DOWN messages, you’ll match on them, because in most cases you want to run different logic if the downed process exited normally or if it crashed (e.g. you typically don’t want to report successful exits to crash reporting services). Since receiving a :DOWN message only tells you that a process has exited, you need to match on the reason value (i.e. :normal in the message you posted) to determine whether the exit was due to a crash or not.

If an HTTPoison (or other process) launched from this GenServer crashes, could I expect the :DOWN message to originate from that?

No: it would send a :DOWN message, but not the one you posted. The posted one has a :normal exit reason, therefore it wasn’t due to a process crashing.

Hope this helps!

Post #3

Rodeoclash

Ok, well the recap certainly helps and has given me something to think about, thanks @david_ex. I’m still unsure how I could be monitoring or linking a process in that GenServer code but it’s given me a starting point!

Post #4

— All posts loaded —