tmbb
Incendium
Easy profiling for your Phoenix controller actions (and other functions) using flamegraphs.
Example flamegraph: Example flamegraph — Incendium v0.5.0
Documentation can be found at https://hexdocs.pm/incendium.
Rationale
Profiling Elixir code is easy using the default Erlang tools, such as fprof.
These tools produce a lot of potentially useful data, but visualizing and interpreting all that data is not easy.
The erlang tool eflame contains some utilities to generate flamegraphs from the results of profiling your code
But… eflame expects you to manually run a bash script, which according to my reading seems to call a perl (!) script that generates an interactive SVG.
And although the generated SVGs support some minimal interaction, it’s possible to do better.
When developping a web application, one can take advantage of the web browser as a highly dynamic tool for visualization using SVG, HTML and Javascript.
Fortunately there is a very good javascrtip library to generate flamegraphs: d3-flame-graph.
By reading :eflame stacktrace samples and converting them into a format that d3-flame-graph can understand, we can render the flamegraph in a webpage.
That way, instead of the manual steps above you can just visit an URL in your web application.
Usage
To use incendium in your web application, you need to follow these steps:
1. Add Incendium as a dependency
def deps do
[
{:incendium, "~> 0.2.0"}
]
end
You can make it a :dev only dependency if you wish, but Incendium will only decorate your functions if you’re in :dev mode. Incendium decorators won’t decorate your functions in :prod (profilers such as eflame should never be used in :prod because they add a very significant overhead; your code will be ~10-12 times slower)
2. Create an Incendium Controller for your application
# lib/my_app_web/controllers/incencdium_controller.ex
defmodule MyApp.IncendiumController do
use Incendium.Controller,
routes_module: MyAppWeb.Router.Helpers,
otp_app: :my_app
end
There is no need to define an accompanying view.
Currently the controller is not extensible (and there aren’t many natural extension points anyway).
Upon compilation, the controller will automcatically add the files incendium.js and incendium.css to your priv/static directory so that those static files will be served using the normal Phoenix mechanisms.
On unusual Phoenix apps which have static files in other places, this might not work as expected.
Currently there isn’t a way to override the place where the static files should be added.
3. Add the controller to your Router
# lib/my_app_web/controllers/router.ex
require Incendium
scope "/incendium", MyAppWeb do
Incendium.routes(IncendiumController)
end
4. Decorate the functions you want to profile
Incendium decorators depend on the decorator package.
defmodule MyAppWeb.MyContext.ExampleController do
use MyAppWeb.Mandarin, :controller
# Activate the incendium decorators
use Incendium.Decorator
# Each invocation of the `index/2` function will be traced and profiled.
@decorate incendium_profile_with_tracing()
def index(conn, params) do
resources = MyContext.list_resources(params)
render(conn, "index.html", resources: resources)
end
end
Currently incendium only supports tracing profilers (which are very slow and not practical in production).
In the future we may support better options such as sampling profilers.
5. Visit the /incendium route to see the generated flamegraph
Each time you run a profiled function, a new stacktrace will be generated. Stacktraces are not currently saves, you can only access the latest one. In the future we might add a persistence layer that stores a number of stacktraces instead of keeping just the last one.
Here you can find an example flamegraph with explanations about how to interact with it .
Where Next?
Trending in Announcing
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
Deploy Phoenix to your own VPS with PaaS ease: push-to-deploy, zero downtime, auto SSL, managed Postgres. 14-day free trial at potions.io
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #blog-post
- #ai
- #phoenix_html
- #elixirconf-us
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #hex
- #security
- #metaprogramming
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Practical resources that improve the lives of professional developers.
Producing high quality Elixir screencasts since 2017.
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 10- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
tmbb
Some questions “for the audience”…
Should I add a way of saving profiler stacktraces for multiple function runs and aggregate them all in the same flamegraph?
Should I add a way to compare flamegraphs between different implementations of the same function? How should I do it? I can’t find a good way of comparing flamegraphs…
Any suggestions for a good sampling profilers to use? (as opposed to a “tracing” profiler)
lud
This is nice
But how could it be used if an app does not use phoenix framework ?
tmbb
Well, you can use the private API to generate HTML files and dump them somewhere.
Or maybe I could change things so that Incendium was a standalone phoenix app locally (listening to a different port, of course). That would make it independent of the user’s app, and it would allow you to profile any Elixir code running locally.
That’s probably the easiest way
tmbb
PS. This was written as a very simple way of profiling routes in a Phoenix app, and although I’ve tried to make it generic and usable from other apps, I’ve always thought of making it a phoenix controller instead of a standalone phoenix app
tmbb
Incendium can now (v0.3.0) integrate with Benchee to generate reproducible benchmarks with performance data displayed as flamegraphs (using samples from multiple iterations, of course).
The logical way would be to create an Incendium formatter for benchee, but unfortunately that’s impossible. Incendium needs to run the benchmark twice (one without profiling so that we get accurate runtimes and another with profiling so that we get flamegraphs). This means I need control before execution (to set up profiling and create some processes that will gather the relevant metrics generated by the profiler), during execution (so that I can run the benchmark suite tiwce) and after execution (as a normal benchee formatter, to generate the HTML report
The limitations above mean that you need to call
Incendium.run(benchmark, options)instead ofBenchee.run(benchmark, options_with_incendim_specific_parts).Example HTML report here: https://hexdocs.pm/incendium/0.3.0/assets/Example.html
Note that the flamegraph width is scaled according to the scenario runtime (longer runtimes create wider flamegraphs). The scaling of flamegraph widths can be disabled with an option.
Pinging @PragTob because he might be interested in this. I’d appreciate a way of running this with
Benchee.run/2instead of having to callIncendum.run/2.The old Pheonix-based features still work as before.
PragTob
Thanks for the ping.
Sad things first, I’ve been battling hand/arm problems for close to 5 months and haven’t done any OSS or gaming or general fun things in the time, hence benchee might appear stale… and there’s also a lot of unreleased stuff on main from even before that
We should definitely have that, and we might already have it!
There is a feature that has been on
mainforever (thanks to idea from @josevalim and implementation by @pablocostass ) to haveprofile_afterwhich runs builtin profilers after benchmarks initial PRIt should be semi easy to extend it to support non builtin profilers/that you can give it custom args. Maybe it already does, can’t check right now.
That said, I worry about my hands and if the past is an indicator of the future won’t get to make a new release or changes any time soon. Sorry
It’s scary for me so I’m minimizing what I can to have a chance to be “good” again. Once I am, fixing that damn mac bug in nano time measurements releasing a new versiona nd brushing up benchee (and simplecov) is top of the priority list.
tmbb
Oh, hope you get better soon!
Does the profiler have access to the results of the original suite so that I can report the original runtimes and do some sanity checks on whether the profilers have distorted the relationship between the runtimes of different scenarios?
PragTob
Profile is run last - it has access to everything. However, a quick look at
Profileshows that we are only calling the profilers themselves with the functions. So, some further work would be needed there.That said, one of the easiest things might be for you for now to do:
Benchee funs always return the
suitewhich has all the aggregated information throughout the benchmark run.tmbb
If calling the profiler on the suite is all I need, then I don’t think I’ll need anything else
tmbb
Another question: do you think that scaling the flamegraph widths should be the default, so that longer runtimes result in wider flamegraphs? Or should all the flamegraphs have the same width by default?
This will be configurable, of course, I’m just asking you what the default should be