Phillipp
Increasing resource usage with Phoenix
Hey, I am using Phoenix for a still quite small app. All the app does is getting some input, then starting a bunch of processes per entity and after a certain time, stopping these processes again (DynamicSupervisor).
We are not live yet, so the system is most of the time in “idle” mode, except a few times a week when the customer is testing some stuff.
I am monitoring the application with Prometheus and noticed some increase in resource usage over time. You can see the BEAM dashboard in the provided screenshot. It shows the last 7 days. The resource usage goes up over time. I redeployed one time, that’s when the usage goes straight down.
For the memory part, it is mostly process memory that increases. It cannot be my processes since they get cleaned up if an entity is not “tracked” anymore.
It’s weird that the IO, GC and Load also increases.
While typing this, I may have an idea of what it could be. The Prometheus metrics are filling up over time, for example when bots hit URLs that don’t exist. But the Prometheus package stores the data in ETS and the ETS memory stays at around 1,9mb the whole time while process memory goes from 12mb to 33mb.
Just noticed that the process memory dropped to around 15mb overnight (screenshot is from yesterday), while all other metrics stayed high.
I am really confused about what’s going on here.
Quick summary of my setup:
- Phoenix 1.4
- Using DynamicSupervisor to start and terminate “tracking” processes
- No real load on the system yet
- Hackney pool size of 1000
- prometheus_ex package for collecting metrics
Most Liked
michalmuskala
Debugging a system without a way to inspect that system is kind of a lost cause 
You could potentially try some of the web-based observer replacements, but YMMV.
michalmuskala
The graphs indicate the memory is primarily allocated to processes - have you explored which processes use that memory? You could obtain the top offenders either with observer or some other tool (with recon, e.g. :recon.proc_count(:memory, 10))
NobbZ
As docker usually doesn’t have a graphical display server running, most erlang images (especially those that target small image sizes) strip out wx and dependants. So you either need to install a version of erlang that includes wx and other missing applications, choose another base image that has them already, or drop them from your :extra_applications.
Popular in Questions
Other popular topics
Latest Phoenix Threads
Latest on Elixir Forum
Sponsor Spotlight
Supporting innovation across the BEAM ecosystem.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #javascript
- #code-sync
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Real-time error tracking, performance insights, and observability for devs.
Practical resources that improve the lives of professional developers.
Develop your skills with books, videos, and courses.
Enabling companies to succeed by building software people love.
Courses that'll move you from confusion to "Aha, now I get it!"