PragTob

PragTob

You may not need GenServers and supervision trees

Hey everyone,

this has been brewing in my head some time and it came up again while reading Adopting Elixir.

GenServers, supervisors etc. are great technologies that help you solve problems. They’re one of the things that is most special about elixir/Erlang. As a result lots of conference talks, blog posts etc. focus on them and it seems everyone wants to use them.

However, do you need them all the time? At least while using a framework (like Phoenix), chances are you don’t. Of course, until you got a problem that these help you solve.

Building a relatively standard CRUD web application with Phoenix? No need.
Just using channels for a chat like applications in Phoenix? You’re good.

The hidden detail of course is that you are using GenServers and friends without even knowing it - Phoenix runs every request and every channel in their own processes. Ecto uses poolboy for your database connections. It’s already parallelized and you don’t need to take care of it. That’s the beauty of it. What I’m saying is that in the standard situation the eco system takes care of you.

Why am I picking up this topic?
It feels like we talk so much about GenServers etc. that people who come to Elixir feel like they need to use them or they are not “really” using Elixir. I hear people say something to the tune of “We’re still using this like Rails - we should use GenServers” - without any need (granted they mostly don’t know what Phoenix & friends already do under the hood). At worst (as I’ve seen in some questions here) people create a single GenServer that then essentially all traffic needs to go through complicating their code while also adding an unneeded bottleneck. Maybe they just complicate their code, that’s also bad.

To get back to “Adopting Elixir” an example from it:

A new developer team started building their Phoenix applications.
They had always heard GenServers could be treated like microservices but even
tinier. This “wisdom” led them to push all of their database access control to
GenServers .
(…)
performance was abysmal. Under high-enough load, some pages took 3 sec-
onds to render because they built a bottleneck where none existed. They
defeated Ecto connection pools because all access happened through a single
process.
In essence, they made it easy to create global, mutable variables in Elixir. They
essentially crippled the single biggest advantage of functional languages, for
no gain whatsoever.

Which is also what I’ve seen around a bunch of times. The book also provides some guidance as to what to best use GenServers for:

  • Model state accessed by multiple processes.
  • Run multiple tasks concurrently.
  • Gracefully handle clean startup and exit concerns.
  • Communicate between servers

So, what do I want in the end?

Well, I want to discuss with you all about this and hear your opinions!

I think we should make it clearer that you don’t have to use GenServers and that doing so might actually be harmful. My 2 production applications include no single GenServer written by us. They run fine. In general the eco system takes good care of you so you’re using them without realizing it (which is good imo).

I’m not saying you shouldn’t learn about GenServers. You should. But know when to use them and when not to.

Lastly, if you disagree I want you to scream at me and teach me the error in my ways :smiley:

First 10 of 39 Posts Switch mode

jordiee

jordiee

I agree with you that for the most party you don’t need to use any otp stuff in servers but people should also understand some of the awesome things you can do with it. Having an distributed cache(mnesia), tracking state real time without the need to hit a database in genservers. Also depending on the structure of your code you don’t “have” to take a performance hit. Running everything through a genserver…sure its going to be quite slow. spinning up a new genserver per “user”…your performance is probably still going to be pretty good and now you can track things in memory for users. Again I agree that you, 99% of the time you do not need to use anything otp related but when you are in that 1% its really a game changer that makes elixir as great as it is.

I for example heavily use otp things(mnesia,ets,supervisors,genservers) in a saas product I am working on and it has got to the point where I do not think I could get the same performance out of the box with any other language.

PragTob

PragTob OP

Thanks for your response! Totally agree, it can be a total game changer with what you’re able to do with it and I myself still have to get acquainted with large parts of OTP. Knowing that all of this has proven reliability and performance in large system is also very comforting :slight_smile:

Oh definitely - didn’t want to come across as saying they’re bad for performance. On the contrary, I usually think of them as improving performance (by parallelization). The fact that they’re so incredibly cheap to spin up is just sprinkles on the cake!

yurko

yurko

I think that’s “kinda” true :smiley: You sure don’t have to use them for the sake of using them, but the only way to feel their awesomeness is to actually use them, otherwise you’ll just never get the feeling of needing to use them (if that makes sense).

We have a bunch of apps and only the first one, a tiny microservice actually has no GenServers. Even things that needed cronjobs / sidekiq profit from them and these are not exotic features.

sync08

sync08

I almost wrote about this not long ago but while writing it out I came to some unfortunate conclusions. I’ll just cut right to the chase and say if you’re not using Elixir for the genservers, supervisors e.t.c. then it’s probably not worth using the language.

As a language, I love Elixir. It has made some compromises but nothing major. The only real issue I have with it is that it’s dynamically typed. I’m pretty sure I linked @PragTob’s blog post in a thread the other day talking about this.

As much as i wish I could just use the language and forget about complex supervision trees e.t.c. it just doesn’t make sense. It’s not that fast, it’s not that safe, it’s not light on system resources, it’s editor tooling isn’t great, it’s not suitable for scripting, it’s not suitable for GUIs and most importantly it’s growth has peaked. The only thing it currently makes sense to use Elixir for is fault-tolerant server side apps.

cpgo

cpgo

If I understood @PragTob point is that if you are using something like Phoenix or Ecto you are already taking advantage of the faul-tolerant side of elixir and you dont need to try and shove otp stuff where you dont need.

As a very bad comparison, I can use Rails without any Ruby metaprogramming black magic on my code but still Rails is using it (very) heavily for me.

sync08

sync08

While I understand that, having just Phoenix and Ecto be fault tolerant buys you nothing. Any modern web framework (regardless of language) can handle bad requests without bringing down the server.

My point is that if you’re not using the built in supervision tools then there is pretty much no benefit to using Elixir but many downsides.

jordiee

jordiee

Not completely true, You at least get a language that runs across cores pretty nicely out of the box. Not saying other languages can’t do that but its really nice.

amnu3387

amnu3387

So what are the many downsides to it?

Previously you mentioned “is not that fast/light/safe”, which can be true about anything else as well, it basically depends on what you compare it against, nothing else. It can also be much faster/lighter/safer. So what are your points of comparison?

I personally feel very happy writing in it - I just wrote a queue system for a game in half a bunch of lines, split between genservers and ets tables…

edmz

edmz

I agree with your feeling and I would like to extend this to processes too. They say that when you are a hammer everything looks like a nail. So I see people coming to elixir becoming hammers and trying to use process for everything and getting frustrated in the process (ha!, no pun intended).

Its like getting into ruby and somehow convincing/forcing yourself to use threads everywhere when in real life you would’nt really be using them that much.

I think its an issue of communication.

sync08

sync08

Yeah but I get that with many languages now. I’m still half in the C# world and despite the hype about Erlang/Elixir multi-core usage, it’s actually not that great at it. I’ve done some benchmarks and saturating all cores caps out at about 75% CPU usage. In C# I can get almost 100% usage. Even if I could push Elixir to utilize all 100% (you can’t, it’s fundamental BEAM overhead) you still won’t see the performance come anywhere close. I’m talking orders of magnitude difference in most tasks.

What I will say is that concurrency in Elixir is way easier and safer.

Where Next?

Trending in Discussions Top

AstonJ
As the title says, please share what you’ve been up to with Elixir. Whether that’s been learning it, looking into it, making stuff with i...
2977 91561 914
New
byu
@chrismccord : I just saw the Extract AGENTS.md from Phoenix.new into phx.new generator commit to the phoenix project. My initial shotgu...
New
arcanemachine
I was working on an Ecto migration and I needed a timestamp. So, for the nth time, I looked up the different data types for timestamps, a...
New
AstonJ
Just a general thread to post chat/news/info relating to AI/ML stuff that may be relevant for Nx now or in the future. Got anything to sh...
New
type1fool
I just stumbled on a newly redesigned elixir-lang.org. :tada: It looks like @Software_Mansion did the work, and I think it is generally a...
New
juhalehtonen
There has been a thread to discuss the Stack Overflow Developer Survey on this forum every year since 2018, so here’s yet another one for...
New
alexslade
Fly’s CEO posted this recently - Turn And Face The Strange · The Fly Blog It says that Fly is going all-in on sprites, which is a worry ...
New

Other Trending Topics Top

JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve. They are GUI (Emerge) and State management (S...
New
ausimian
Emily is an Elixir library that runs Nx computations on Apple’s MLX. Install it as the default Nx backend and Nx, defn, Axon, Nx.Serving,...
New
akoutmos
@hugobarauna and I (Alex Koutmos) have been hard at work on writing a book on Nerves that takes you from simply blinking LEDs to building...
New
wintermeyer
There are three potential reasons for members of this forum to have a look at https://vutuv.de You are tired or annoyed of LinkedIn. Yo...
New

We're in Beta

About us Mission Statement