Hello! We recently upgraded a bunch of dependencies after being pretty far behind for a while.
We have more yet to go, but we’re seeing an issue with the first round that makes us hesitant to continue.
Since the deploy, we’re seeing a big spike in CPU use 1x or 2x per day. (we’re on Heroku, so it’s actually ‘dyno load’ which is just a little different). It ramps up to 50-100% pretty quickly, sustains for 15-20 minutes, and drops back down to normal levels & stays there. Normal levels are well under 1%.
These tend to happen when there is little to no traffic on the server. Not a result of a restart or crash.
We have 2 servers running at a time, and it happens to each of them at different times. So far, it hasn’t happened with both simultaneously. If there’s a correlation, it’s when traffic is low.
While the described behavior seems unlikely to be caused by dependency upgrades, the issue started occurring consistently on the exact same day that we deployed these, and hadn’t occurred for years before it.
Hoping someone here recognizes a known issue with a dependency listed below, or compatibility type issue. We’ve been through changelogs for the most likely culprits and haven’t seen anything interesting.
Here are the dependency changes per mix.lock.
appsignal 2.0.6 -> 2.2.10
appsignal_phoenix 2.0.3 -> 2.0.13
appsignal_plug 2.0.4 -> 2.0.9
basic_auth 2.2.4 -> 2.2.5
certifi 2.5.2 -> 2.8.0
connection 1.0.4 -> 1.1.0
cors_plug 2.0.0 - 2.0.3
cowboy 2.6.1 -> 2.9.0
cowboy_telemetry (new) 0.3.1
cowlib 2.7.0 -> 2.11.0
date_time_parser 1.1.2 -> 1.1.3
db_connection 2.1.0 -> 2.4.1
decimal 1.8.1 -> 1.9.0
decorator 1.3.2 -> 1.4.0
ecto 3.1.5 -> 3.1.7
ecto_sql 3.1.4 -> 3.1.6
erlsom 1.4.2 -> 1.5.0
ex_aws 2.1.3 -> 2.2.10
ex_aws_s3 2.0.2 -> 2.3.2
ex_phone_number 0.2.0 -> 0.2.1
hackney 1.16.0 -> 1.18.0
hashids 2.0.4 -> 2.0.5
html_entities 0.4.0 -> 0.5.2
html_sanitize_ex 1.4.0 -> 1.4.2
idna 6.0.1 -> 6.1.1
jason 1.2.1 -> 1.3.0
joken 2.0.1 -> 2.4.1
jose 1.9.0 1.11.2
mime 1.3.1 -> 1.6.0
mochiweb 2.20.0 -> 2.22.0
nimble_parsec 1.1.0 -> 1.2.1
parse_trans 3.3.0 -> 3.3.31
phoenix 1.4.1 -> 1.4.17
phoenix_html 2.13.1 -> 2.13.4
phoenix_swagger (removed)
plug 1.7.2 -> 1.13.0
plug_cowboy 2.0.1 -> 2.5.2
plug_crypto 1.0.0 -> 1.2.2
ranch 1.7.1 -> 1.8.0
sentry 7.0.1 -> 7.2.5
sweet_xml 0.6.6 -> 0.7.2
telemtry 0.4.0 -> 0.4.3
timex 3.6.1 -> 3.7.2
tzdata 1.0.3 -> 1.1.1
unicode_util_compat 0.5.0 -> 0.7.0
Any ideas?
