I’ve been trying to track down odd behaviour that I’m experiencing with running my Elixir application in Production where by the first request to that specific module/function in my Phoenix controller is slow, and all further requests respond in the expected time.
At first I thought it was an issue with my code around fetching data from ETS, but then figured out with the help of a fellow Alchemist on the Slack channel that it happens even to a raw plug that does no interactions other than passing a short string of JSON exhibits the same behaviour.
It was his idea to attempt to use Distillery to create a release and see if that had any affect, and the odd part is that it does.
The first screenshot is the odd behaviour in question, a simple 5 line plug responding in 28ms.
The second shows how this is not an issue when running my application via Distillery.
This is probably due to code loading. The Erlang code loader supports two modes: interactive and embedded. The former loads BEAM files on-demand, the latter loads all required code for a release upon startup. See http://erlang.org/doc/man/code.html
When you run with mix, interactive mode is used, which causes some modules to be loaded upon the first request. Distillery creates a release, which uses embedded mode.
That flag changes the some aspects of the compilation process, perhaps in support of using embedded mode, but it doesn’t actually enable that mode. Calling :code.get_mode() will return :interactive even in prod.
I’m not sure it is possible to use embedded mode outside of a release. Passing --erl "-mode embedded" or ERL_FLAGS="-mode embedded" always result in a crash of the boot process for me.
@cbarratt: as you’ve seen, one fix is to build a release. The alternative would be for you to ‘warm up’ your application yourself, as part of your application initialization or through deployment tooling…
This doesn’t seem super viable because each unique code path may load some modules and not others. It isn’t as simple as just hitting a health check endpoint because there’s tons of modules that such a request won’t hit.