Hello, I have an application that is built using distillery.
I have a systemd unit file that starts the app in foreground mode (logs through rsyslog, direct systemd control and other advantages).
Unfortunately, no matter what I try, I can’t find a way to terminate the application in foreground in a clean way.
The main problem is that the bash script trapping SIGTERM dies when the app starts (but not the rest), so there is only
beam.smp catching the signal, which causes exiting with code 1. I’m on otp 20 and elixir 1.6.5.
Anyone with ideas on how to achieve a clean shutdown?
The systemd file:
[Unit] Description=Winex Phoenix Server After=local-fs.target network.target postgresql.service Requires=postgresql.service [Service] Type=simple User=winex Group=winexweb WorkingDirectory=/opt/winex ExecStart=/opt/winex/bin/winex foreground ExecStop=/opt/winex/bin/winex stop Environment=LANG=en_US.UTF-8 Environment=MIX_ENV=prod Environment=RELEASE_MUTABLE_DIR=/var/opt/winex Environment=ERL_CRASH_DUMP=/var/opt/winex/erl_crash.dump LimitNOFILE=65536 UMask=0027 StandardOutput=syslog StandardError=syslog SyslogIdentifier=winex Restart=always RestartSec=10s TimeoutStartSec=10s TimeoutStopSec=5s [Install] WantedBy=multi-user.target
When the process is started, the following things are running:
● winex.service - Winex Phoenix Server Loaded: loaded (/lib/systemd/system/winex.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2018-06-07 04:26:04 UTC; 7min ago Main PID: 13698 (beam.smp) Tasks: 31 (limit: 1152) CGroup: /system.slice/winex.service ├─13698 /opt/winex/erts-9.3.1/bin/beam.smp -Bd -- -root /opt/winex -progname opt/winex/releases/0.0.17/winex.sh -- ├─13959 /opt/winex/erts-9.3.1/bin/epmd -daemon ├─13970 erl_child_setup 65536 ├─13989 inet_gethost 4 └─13990 inet_gethost 4
When is stopped, I get:
● winex.service - Winex Phoenix Server Loaded: loaded (/lib/systemd/system/winex.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Thu 2018-06-07 03:58:25 UTC; 1s ago Process: 12492 ExecStop=/opt/winex/cmds/stop.sh (code=exited, status=0/SUCCESS) Process: 12210 ExecStart=/opt/winex/bin/winex foreground (code=exited, status=1/FAILURE) Main PID: 12210 (code=exited, status=1/FAILURE) Jun 07 03:57:51 winehost1 systemd: Started Winex Phoenix Server. Jun 07 03:57:53 winehost1 winex: 03:57:53.540 [info] Running Winex.Web.Endpoint with Cowboy using http://127.0.0.1:4000 Jun 07 03:58:25 winehost1 systemd: Stopping Winex Phoenix Server... Jun 07 03:58:25 winehost1 winex: 03:58:25.455 [info] SIGTERM received - shutting down Jun 07 03:58:25 winehost1 winex: erl_child_setup closed Jun 07 03:58:25 winehost1 winex: [1B blob data] Jun 07 03:58:25 winehost1 winex: Crash dump is being written to: /var/opt/winex/erl_crash.dump...done Jun 07 03:58:25 winehost1 systemd: winex.service: Main process exited, code=exited, status=1/FAILURE Jun 07 03:58:25 winehost1 systemd: winex.service: Failed with result 'exit-code'. Jun 07 03:58:25 winehost1 systemd: Stopped Winex Phoenix Server.
Right now my temporary solution is to avoid
ExecStop and instead use
KillMode=mixed which kills the main process with SIGTERM and the rest with SIGKILL.