noam87

noam87

When to use one-off processes vs long-running servers?

Hi, so I’m reading Elixir In Action and on page 159 it hints that spawning off concurrent processes is a code smell.

Scenario: User clicks button to generate a schedule -> a schedule is generated in the background -> an email task sends him an email with the schedule -> at the same time a db task converts this schedule into db-friendly format and stores it.

Approach 1: Have a bunch of long running servers, ScheduleServer, EmailServer, DbConverterServer, all waiting for messages to process in its inbox. User actions simply send a message.

Approach 2: Every time a user does one of these actions, I spawn off a new instance of ScheduleGenerator (which itself spawns the other processes that come after), so that each user request is done in parallel. When the task is complete, the process dies.

Which would be the “elixir way” to do this? What are some rules of thumb for when to choose one approach over the other?

Most Liked

sasajuric

sasajuric

Author of Elixir In Action

That’s certainly not what I meant to say :slight_smile: The paragraph you’re refering to just warns that if you’re optimizing a sequential piece of code, you should first consider algorithmic and technology-specific optimizations.

The reason I added that comment is because I’ve seen people frequently trying to optimize a suboptimal algorithm by running chunks in multiple processes. While that might improve performance, it is often better to first consider whether algorithm can be improved. For example, if you can reduce complexity from say polynomial to logarithmic, then you should get much better savings.

However processes should certainly be used to run different independent or loosely dependent things. Per your description, the output of e-mailing is not depending on storing to db, so you can do those two things in separate processes. This will not only improve efficiency (the VM might run those things in parallel), but also fault-tolerance (if e-mailing fails, storing to db might still succeed).

When it comes to your question, I agree with @dom’s response. You need a long-running server only if it needs to keep some state or if actions need to be serialized.

dom

dom

Approach 1 makes sense if schedules have to be processed in order. The point of the process then is to serialize the calls. Otherwise, it’s better not to go through one process, because it creates a potential bottleneck.

I would use a supervised task: http://elixir-lang.org/docs/stable/elixir/Task.html

jwarlander

jwarlander

Are you refer to the last part of the page?

“If you can’t make message handling fast enough, you can try to split the server into multiple processes, […] This should be your last resort, though. Parallelization isn’t a remedy for a poorly constructed algorithm.”

If so, I think he just means that exactly; start by optimizing the algorithm itself, making sure each single message is handled efficiently. Then, if the algorithm can meaningfully be parallelized, try to split into multiple processes if need be.

There are other reasons for concurrency, of course. Some things, like incoming web requests, are naturally concurrent. Other things may need the fault isolation that a separate process gives you.

If you were to make a game, for example, it’s quite likely each player’s input would be handled in a separate process. Any fault in the input handling code could then only crash that process, and wouldn’t affect any other player. For a game with room-based navigation, each room might be a process as well - players could affect the state of the room by dropping items, picking them up, maybe destroying things, etc. In this case as well, you’d benefit from the fault isolation; code execution that’s triggered in the room process only affects that room and it’s inhabitants. With the right monitors, crashes in either of those parts can be safely handled with minimal impact.

Email sending is probably a typical thing though where you need to limit concurrency with a pool, or just have a single persistent email sender process, depending on your scale.

Where Next?

Popular in Questions Top

aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
marius95
Hello everyone, I try to use an Javascript Event Handler in my root.html.leex file. Therefore I created a function in the app.js file: ...
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
mgjohns61585
Could someone help me? I’m making my first elixir program, number guessing game. I can’t figure out how to convert the user’s guess from ...
New
shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New
nobody
How to bind a phoenix app to a specific ip address? could not find anything about that, nowhere, unfortunately, but for me this is quite...
New
LegitStack
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New

Other popular topics Top

sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
985 42920 311
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New
hariharasudhan94
lets say i have a sample like a = 20; b = 10; if (a > b) do {:ok, "a"} end if (a < b) do {:ok, b} end if (a == b) do {:ok, "equa...
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
Qqwy
Update: How to use the Blogs & Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3271 126479 1222
New

We're in Beta

About us Mission Statement