prxssh

prxssh

Any tips or learning resources for writing highly performant Elixir code?

I’ve been writing Elixir and Phoenix professionally for the past year and have grown quite fond of them. I’ve read books like Elixir in Action, Concurrent Data Processing in Elixir, and Designing Elixir Systems with OTP.

I’m keenly interested in writing highly performant code in Elixir. Obviously, achieving performance levels like Go, C++, or Rust isn’t possible, and I understand that. However, I suspect there might still be some hidden tricks to make Elixir code more performant.

Could anyone point me towards any source material, books, talks, etc., that dive into the performance aspects of Elixir?

First Post!

Eiji

Eiji

They are - just use NIFs :wink:

There are lots of possible improvements, for example:

  1. Depending on specific case i.e. you have 2 or more algorithms, benchmark them using benchee and use desired solution
  2. NIF and related language features not only can speed up code, but also allows to use many non-Elixir libraries.
  3. Stream, Flow and other modules instead of Enum - instead of storing whole enumerable in memory we can read and process them separately.. Similarly we can process multiple list elements at the same time. There are lots of solutions depending on what you want to achieve
  4. Metaprogramming i.e. improve runtime code by for example generating a pattern-matching
  5. Try to limit piped Enum calls, so iterate one list as few times as possible, often [head | tail]-based recursive function or Enum.reduce/3 is useful in such cases.

Please remember that there is no #1 optimisation. Sometimes you optimise resources usage and sometimes you use all available resources to speed up the work. Also it’s worth to mention that raw speed it not always welcome if you have to write very strict code. Often writing more generic code without every possible optimisation may save you huge amount of time in case you would be rewriting or enhancing it by supporting more structs and so on … Again, there is no a single good solution here as it’s very case-specific topic.

Most Liked

rkallos

rkallos

Without much more information about what you want your code to do, I can only offer more general advice to think about when writing code, as a supplement to Eiji’s great advice.

Do less.

It should not be surprising that code that does less usually finishes faster than code that does more. Applying this advice could look like:

  • Refactoring traversals of collections to traverse less, ideally once. (Echoed in Eiji’s post).
  • Performing fewer deep structure updates. While updates to immutable data are efficient from a VM perspective, they’re not free. Certain updates to data structures are much cheaper than others, and you should try and use those whenever possible. For example, building iolists can be much faster than repeatedly constructing binaries.

Using :ets, :atomics, and more.

Erlang’s standard library contains modules that can be a great help when writing code that needs high performance. :atomics and :counters are great for working with collections of integers that must be updated atomically. :persistent_term is very useful for accessing read-only data from many processes.

ETS is a more general-purpose tool, and learning to wield it well can dramatically improve performance in many situations.

Read the Erlang Efficiency Guide

Reading, and more importantly, understanding the advice written in the Erlang Efficiency Guide should take you pretty far along your journey to writing high-performance code that runs on the BEAM. The guide does a great job of explaining why certain code runs more slowly, and communicates useful insight to the BEAM VM that you can keep in your mind when you code.

Know when to stop

While it is very satisfying to write code that runs very quickly, certain optimizations and refactorings done in the name of performance can have a strong negative impact on the readability, testability, and maintainability of your code. To quote the late, great Joe Armstrong:

Make it work, then make it beautiful, then if you really, really have to, make it fast. 90 percent of the time, if you make it beautiful, it will already be fast. So really, just make it beautiful!
– Joe Armstrong, Erlang & OTP in Action

11
Post #3
dimitarvp

dimitarvp

The most performance I managed to eke out of Elixir was:

  1. When I parallelized the algorithm. The BEAM VM just absolutely excels at parallel programming.
  2. When I made sure to not copy and pass a lot of data around i.e. if you need to periodically access stuff that’s several kilobytes it’s probably best to put it in ETS and just pass names / references to your workers – and not the data itself. That’s also quite true for any programming language btw; you can make an otherwise quick Rust program crawl down to JS / Golang level if you just constantly copy / clone data. (BTW it really must be said that this very strongly depends on the data, the algorithm, the amount of workers etc.; I also had success with just directly passing the data to workers and was confused as to why using ETS didn’t net me a performance win… until I realized that pulling data out of ETS copies them as well – so it’s all copying in the end but many other parameters in the equation can tilt the result one way or the other. Just measure.)

There are many more that could be inferred by production experience but these were my top 2 every time. I’d say you’ll have more success if you just try your hand at something and if you are not satisfied with the results, come back to the forum and we’ll give you guidance on per-case basis.

lawik

lawik

Nerves Core Team

Michal Muskala who built Jason and the new Erlang :json gave a talk about a lot of how he made it fast. It was given at Code BEAM Berlin last year and should show up online in January or February I believe.

I was there, heard it. Found it quite interesting :slight_smile:

Last Post!

venkatd

venkatd

Some great advice in this thread!

I agree here I need to start optimizing slow parts of the app where performance has business value.

We have a service that runs entirely in memory (except writing to an event log) and it runs very quickly. I’m sold on the approach :slight_smile:

I think you’re saying yourself, but it feels like Plan A should be to try to make things run as fast as possible before resorting to NIFs because NIFs add complexity and you lose some advantages of the BEAM.

For the next thing I need to optimize, i will do just this. I am honestly looking for general advice at this point because performance is an area of more interest to me now.

It feels like as I am improving this skill, I should first focus on becoming better at diagnosing pref bottlenecks rather before getting too involved in solutions to improve performance. Optimizing seems like two skills - identifying what is slow, then a good mental model of knowing what changes are likely to speed up a particular chunk of code.

I will try to be patient here!

You will be happy to hear that I have been doing this recently. I was inspired by a talk on YouTube called “Moving IO to the edges of your app”. I think patterns like Ecto.Multi are intended to encourage this pattern right?

Where Next?

Popular in Chat/Questions Top

Fl4m3Ph03n1x
I am doing some exercises while learning Elixir using Exercism.io. Now, my objective is to do all exercises, extras included. This shoul...
New
zervis
Hello, I’m about to dive into web development. I was thinking about Laravel or Ruby on Rails, but then I found Phoenix. Do you recommen...
New
woohaaha
I’m coming from Ruby and Rails. I have read some Elixir and Phoenix books. They shed a lot of light about building applications in Elixir...
New
dogweather
Can anyone recommend books/courses/videos that use real-world Elixir? E.g.: Idiomatic error handling design, whether it’s {ok/error, .....
New
koen_vb
Hi, I was looking for a pointer of how I could most easily start with phoenix regarding deploying it to something like linode or google c...
New
loganhelms
A while back, I read a great book by Luis Atencio titled, Functional Programming in JavaScript. In section 7.3, he discusses memoization ...
New
Sujit
Hi Team From the title, I am entirely new to programming and i am interested in learning elixir but not sure where to start. Can someone...
New

Other popular topics Top

JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New

We're in Beta

About us Mission Statement