Running code when a script exits

I have a script executed with mix run that has an infinite loop that collects some data in an agent. I’d like to print that data when I interrupt the script.

Tried System.at_exit/1 without success, maybe because the only way I know to interrupt the execution is C-c C-c.

Is there a way to accomplish that?

Ctrl+C, Ctrl+C kills the Virtual Machine. Ctrl+G, Q should exit in a way that the exit handler gets called, alternatively you can try kill $BEAM_PID from another terminal.

I am trying

System.at_exit(fn _status ->
  IO.puts("Bye!")
end)

IO.puts("Interrupt me!")
Process.sleep(:infinity)

C-g + q does not seem to interrupt the script here, and kill does, but does not trigger System.at_exit/1.

That is weird. What output do you get when you do Ctrl-G? It should be something like

User switch command
 --> 

Try doing ?

Btw you did press Enter after the q? It is not stated anywhere but otherwise nothing happens.

When I type Ctrl-G nothing happens apparently, I have recorded a GIF.

CleanShot 2020-06-14 at 18.54.59

Is that strange? This is macOS and elixir -v returns

Erlang/OTP 23 [erts-11.0.2] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] [hipe]

Elixir 1.10.3 (compiled with Erlang/OTP 23)

They two are installed with asdf.

Just throwing out a random idea, but is it different if you execute the script with mix run?

With mix run happens the same, indeed I was trying elixir to remove Mix from the equation.

I have revised shortcuts in iTerm2 in case C-g is intercepted, but don’t find anything. Terminal.app makes no difference either.

C-g q shows “User switch command” in IEx and erl, which to me suggests the character is actually not intercepted by my terminals or my OS. However, System.at_exit/1 does not seem to be invoked:

iex(1)> System.at_exit(fn _ -> IO.puts("Bye!") end)
:ok
iex(2)>
User switch command
 --> q
~/tmp/x %

Also, I have replaced the sleep call with a dummy receive block, just in case sleeping meant whatever code needs to be triggered by C-g is just within the slept process. No luck either.

Does

System.at_exit(fn _status ->
  IO.puts("Bye!")
end)

IO.puts("Interrupt me!")
Process.sleep(:infinity)

print “Bye!” in your machines using C-g q?

Ah, I also tried in a Docker container that runs Linux, same.

This may be of interest to you: https://stackoverflow.com/questions/42324276/catching-exit-signals-in-elixir-escript

AFAIK erts will trap the TERM signal and call init:stop, but when you’re C-c ing that’s an INT signal. You should be able to achieve your desired behavior with a bash wrapper like in the linked stack overflow question.

However, several people in this thread say C-g q should work. Note that C-g does not send a signal, you get the control character 7. If that is possible, a shell script wrapper would be overkill.

I just need to interrupt a script in an ordered way. I have an at_exit callback, but could be taking down a supervision tree orderly or whatever.

A thing to realise here is that the handling of Ctrl-G and its commands only work when you are running a shell. Without a shell this interface is not generally not started. You can see this if you look in the elixir shell script and see that when an iex shell is not start then there is a -noshell argument to the resultant erl command. Doing elixir foo.exs does not start a shell and, hence, no handling of Ctrl-G and q.

1 Like

Temporary hack

Thanks @rvirding! That makes sense. Since Ctrl-G does not trigger a signal, I guess someone needs to be monitoring standard input to do something about it.

By now, I am moving on with a dirty solution, which is to monitor a file:

defmodule Halter do
  @halter "halt"

  def run(on_halt) do
    if File.exists?(@halter) do
      File.rm(@halter)
      on_halt.()
      System.stop()
    else
      Process.sleep(1000)
      run(on_halt)
    end
  end
end

Halter.run(fn -> IO.puts("done") end)

You leave the script running, and touch halt whenever you want it to stop. Enough for my purposes right now.

That solution prints these noisy traces, so it is definitely not clean:

** (exit) exited in: GenServer.call(Mix.ProjectStack, {:get_stack, #Function<10.12591541/1 in Mix.ProjectStack.peek/0>}, :infinity)
    ** (EXIT) no process: the process is not alive or there's no process currently associated with the given name, possibly because its application isn't started
    (elixir 1.10.3) lib/gen_server.ex:1013: GenServer.call/3

System.at_exit/1 is not invoked (that is why run/1 above receives a callback), I do not know why.

Is this a missing feature?

Not being able to do this easily surprises to me. It is a fundamental aspect of Erlang/OTP that you start applications and, symmetrically, shut them down orderly.

So, given that mix run starts the application by default, you’d expect that there would be interface to shut it down too, no? @josevalim is there a technical reason this is not in place? Or is it something that could be contributed?

BTW, if you do not call System.stop/1 and let run/1 and the script finish by itself, then System.at_exit/1 gets invoked, but the application is apparently not stopped. At least, I do not see terminate/2 callbacks being invoked.

There are two questions here:

  1. How to shut the system down cleanly

  2. How to handle certain signals (interrupt)

To answer 1, System.stop/1 is the answer. If terminate/2 callbacks are not invoked, make sure those processes are trapping exits.

To answer 2, since Erlang/OTP 20, there is OS signal handling: http://erlang.org/doc/man/kernel_app.html#os-signal-event-handler

It is a bit of a long answer. In a nutshell, you need to implement a gen_event handler and react to the callbacks you are interested. There is a more detailed guide here: https://medium.com/@ellispritchard/graceful-shutdown-on-kubernetes-with-signals-erlang-otp-20-a22325e8ae98

2 Likes

Thanks @josevalim!

However, System.stop/1 has three problems:

  1. It does not invoke System.at_exit/1, that is why my “halter” solution above needs to get passed a function.
  2. The script ends printing the noisy “exit” traces I quoted above. Maybe that is cosmetic, but a user does not know if “no process: the process is not alive or…” is a normal message or something to worry about, and in any case the output is not clean as you’d expect from running a program that finishes normally.
  3. The programmer needs to do this by hand. I am touching a file to finish the script! Alternatively, I could write a signal handler, but all this seems boilerplate for something mix run should have builtin in my view.

The question is: mix run starts the application by default, why does not provide a builtin way to cleanly stop the application too?

@josevalim ah, re terminate/2, what I said was that if you remove System.stop/1 and let the script finish by itself, then System.at_exit/1 is called, but terminate/2 callbacks are not invoked (meaning that is an observable property that suggests to me the application is not being shutdown, there’s just the VM halting.)

Could put the logic you want to execute on exit in a terminate/2 callback of a process in your app’s sup tree?

We could execute at_exit for System.stop as well… but we do have an issue in that they would be executed way too late in the termination process, when almost everything would be shut down, so it is likely not useful. That’s why System.at_exit is about the script exiting normally.

Another trick you could do is System.at_exit(fn int -> your_code(); System.stop(int) end). :slight_smile:

2 Likes

I have investigated a bit the solution with the signal handler.

Erlang executes :init.stop() on SIGTERM by default. That means that a simple kill PID shuts the thing down orderly.

However, you need to figure out the PID of the process, and open a different terminal to execute the command. Besides, custom code passed to System.at_exit/1 is not invoked by the event handler.

Ctrl-C sends SIGINT. The only shortcut that I’ve seen sends a different signal is Ctrl-\, which sends SIGQUIT. So, I am using this event handler:

defmodule QuitScript do
  @behaviour :gen_event

  def init(_), do: {:ok, nil}
  def handle_event(:sigquit, _state), do: :remove_handler
  def handle_event(signal, state), do: :erl_signal_handler.handle_event(signal, state)
  def handle_call(_, state), do: {:ok, :ok, state}

  def terminate(_reason, _state) do
    IO.puts("My exit code")
    System.stop()
  end
end

:gen_event.swap_handler(:erl_signal_server, {:erl_signal_handler, []}, {QuitScript, []})

Notice that the terminate/2 callback allows you to run custom code on exit.

If you throw that to a script and execute it for example like this

mix run --no-halt foo.exs

Ctrl-C still behaves as always, and additionally Ctrl-\ shuts the thing down orderly (you only see ^\ echoed in the terminal, not too bad).

2 Likes

Alternatively, we could move System.stop() to the callback:

defmodule QuitScript do
  @behaviour :gen_event

  def init(_), do: {:ok, nil}
  def handle_event(:sigquit, _state), do: System.stop()
  def handle_event(signal, state), do: :erl_signal_handler.handle_event(signal, state)
  def handle_call(_, state), do: {:ok, :ok, state}

  def terminate(_reason, _state) do
    IO.puts("My exit code")
  end
end

:gen_event.swap_handler(:erl_signal_server, {:erl_signal_handler, []}, {QuitScript, []})

Looks more natural to me this way, though in the previous example the order in which the custom code runs is more obvious.

2 Likes