Same App, Performance between 1.8 and 1.9 on Docker

Questions / Help
, , ,
1

I have a Phoenix & Absinthe application I have been running in PROD for a while. I run the app on AWS ECS in a docker container using elixir:1.8.2. Performance is good, everything is good.

I decided to update the docker image and started using elixir:1.12 to check it out (knowing that it’s only in RC state), however, I noticed a performance issue with the application. I Switched back to 1.8.2 and the issue resolved. I put it down to using pre-release, however wanted to investigate more and i found that i did not have to update the image version much before the performance issue re-appeared. It turns out when i switch the docker file to use elixir:1.9 i see quite the degradation in performance in my load test.

Looking through the changelog for 1.9 there was nothing that stood out, but wanted to ask here and see if anyone knows of anything obviously I should be looking at.

Just to confirm. Same app, built exactly the same way, same deps on the same hardware (CPU and Memory allocation). elixir:1.8.2 good performance. elixir:1.9 not so good performance. To fix/cause the issue, all i have to do is switch the version of elixir in the docker file.

HTTP Load Testing usings Artillery.io. Same test scenario and load.

elixir:1.8.2

Shortest call 107ms
Longest Call 442ms
Requests per second 1.75
Avg response time 231.5ms

elixir:1.9
Shortest call 516ms
Longest Call 7747ms
Requests per second 1.57
Avg response time 1971.5ms

Other information about the app - It’s a simple GraphQL server that calls on MongoDB to get data. The only slightly unusual thing it does is use System.cmd to call out to a nodejs application. This does occur on every api request, so any change to performance characteristics of System.cmd would cause this.

Thanks for any thoughts or suggestions for further research.

2 Likes
2

To confirm, is the Erlang version the same in both? Or is it being updated too? Also, can you reproduce it by using Bob’s build of Erlang+Elixir? GitHub - hexpm/bob: The Builder

FWIW, nothing should have changed to cause said slowdown. :slight_smile:

2 Likes
3

Thanks i was not aware of Bob’s build, switching to the alpine version (tried with 1.9 and 1.11 of elixir) of that base docker image has sorted the problem and the response times are now back at around the same mark as the 1.8 version using the elixir base image.

I will continue to investigate more and see why the 1.9 base image is having this effect with my project and and AWS ECS combination and feedback into the github issues if i find anything.

Thank you for your help

1 Like