Rotating logs in OTP releases

Hello folks,

How are you rotating logs in OTP releases? Those that are in log folder

erlang.log.1  erlang.log.2  erlang.log.4  erlang.log.5	run_erl.log

Are you using system tools or it’s possible to configure through release options?

http://erlang.org/doc/man/run_erl.html has some environment variables which let you control those things.

However, I tend to run Elixir as part of a Systemd service in the foreground, and then that way the logs are integrated into the general system logging. From there you can forward them anywhere you like.

1 Like

Can you please share the systemd config?

Basically, I would like to avoid log-file reusing and seems that it’s not possible to setup with run_erl environment variables (thank you for the link, btw).

systemd is quite cool, you can do stuff like

journalctl -u myapp.service --since today
journalctl -u myapp.service --since 09:00 --until "1 hour ago"
journalctl -u myapp.service --since "2016-11-10 12:00" --until "2016-11-10 13:00"

etc. you get the idea :slight_smile:

Here is an example service:

[Unit]
Description=My app daemon

[Service]
Type=simple
User=username
Group=groupname
Restart=on-failure
Environment=MIX_ENV=prod "PORT=4000"
Environment=LANG=en_US.UTF-8

WorkingDirectory=/var/apps/myapp
ExecStart=/usr/local/bin/mix phoenix.server

[Install]
WantedBy=multi-user.target

just put it in /lib/systemd/system/myapp.service and control using systemctl, e.g. sudo systemctl status myapp.service

note the use of absolute paths and extra verbosity for utf support

4 Likes

You should make a new top-level post on how to set up systemd for others. Can be turned into a Wiki. :slight_smile:

/me uses either Windows or Docker

you got it :slight_smile: Elixir apps as systemd services - info & wiki

I actually wanted to write a blog post on this topic, but got no time for I don’t know how long, might as well post it here

2 Likes

I decided to try this approach but every time I try to start my service I keep getting a permission denied error i.e mix: /usr/bin/env: elixir: Permission denied then the process exits with status 126.
Am using centos 7 and Elixir 1.4.2 which was installed using a precompiled package. Below is my systemd config:

[Unit]
Description=My backend service

[Service]
Type=simple
User=user
Group=group
Environment=MIX_ENV=prod “PORT=7891”
Environment=LANG=en_US.UTF-8
WorkingDirectory=/PATH_TO_MY_APP
ExecStart=/usr/bin/elixir/bin/mix phoenix.server
Restart=on-failure

[Install]
WantedBy=multi-user.target

and placed this file in /etc/systemd/system.

Kindly assist figure out what I might have done wrong.

do you actually have the user “user” and group “group”? These are dummy values that you have to replace with correct user and group names. See who owns your app files and use this user / group, also the working directory must have the correct path to your app.

Thanks for your reply.

The user, group and path were just place holders,
this is my config file: (myapp.service):

[Unit]
Description=Myapp backend service

[Service]
Type=simple
User=wogembo
Group=wogembo
Environment=MIX_ENV=prod “PORT=7891”
Environment=LANG=en_US.UTF-8
WorkingDirectory=/test/ussd/apps/myapp
ExecStart=/usr/bin/elixir/bin/mix phoenix.server
Restart=on-failure

[Install]
WantedBy=multi-user.target

So the user and group in this config own the app file.

I think the error happens when systemd tries to set environment variables, but couldn’t
figure out why. Here is the output of sudo systemctl status myapp.service

[wogembo@hostname ~]$ sudo systemctl status myapp.service
● myapp.service - Myapp backend service
Loaded: loaded (/etc/systemd/system/myapp.service; enabled; vendor preset: disabled)
Active: failed (Result: start-limit) since Tue 2017-05-16 17:11:31 EAT; 3s ago
Process: 16311 ExecStart=/usr/bin/elixir/bin/mix phoenix.server (code=exited, status=126)
Main PID: 16311 (code=exited, status=126)

May 16 17:11:31 hostname systemd[1]: myapp.service: main process exited, code=exited, status=126/n/a
May 16 17:11:31 hostname systemd[1]: Unit myapp.service entered failed state.
May 16 17:11:31 hostname systemd[1]: myapp.service failed.
May 16 17:11:31 hostname systemd[1]: myapp.service holdoff time over, scheduling restart.
May 16 17:11:31 hostname systemd[1]: start request repeated too quickly for myapp.service
May 16 17:11:31 hostname systemd[1]: Failed to start Myapp backend service.
May 16 17:11:31 hostname systemd[1]: Unit myapp.service entered failed state.
May 16 17:11:31 hostname systemd[1]: myapp.service failed.

And this is seen in journalctl -xe

– Unit myapp.service has begun starting up.
May 16 17:15:00 hostname mix[16419]: /usr/bin/env: elixir: Permission denied
May 16 17:15:00 hostname systemd[1]: myapp.service: main process exited, code=exited, status=126/n/a

The file permission is as below:
drwxrwxr-x. 8 wogembo wogembo 4096 May 15 16:50 myapp

What’s the permissions for your elixir binary? Please follow the complete symlink chain and plot each permissions.

See below my elixir binary and its permissions.

[wogembo@hostname ~]$ which elixir
/usr/bin/elixir/bin/elixir
[wogembo@hostname ~]$ ls -al /usr/bin/elixir/bin/elixir
-rwxr-xr-x. 1 root root 3819 Feb 16 17:33 /usr/bin/elixir/bin/elixir

Am not sure how to plot the complete symlink chain.

You have that folder directly in the path? Does your services user have it there as well? Probably it has /usr/bin there, finds a sum link, tries to follow but isn’t allowed.

Please su into that user and try again.

Plotting the whole sum link chain simply means that if you find a sum link, plot it’s permissions, follow it and plot again until you are at an elixir executable.

1 Like

Thanks for your reply.

The elixir binary was not in roots path so I moved the elixir folder
from /usr/bin/ to /opt/ then symlinked to the elixir binary i.e

sudo ln -s /opt/elixir/bin/elixir /usr/bin/elixir
sudo ln -s /opt/elixir/bin/elixirc /usr/bin/elixirc
sudo ln -s /opt/elixir/bin/mix /usr/bin/mix
sudo ln -s /opt/elixir/bin/iex /usr/bin/iex

then updated my systemd config ExecStart to /usr/bin/mix phoenix.server
and now its works perfectly. :slight_smile:

Thank you all.