PragTob

PragTob

Blog Post: Careful what data you send or how to tank your performance with Task.async

I ran into an interesting problem recently where simple concurrency on the BEAM via Task.async made my application a lot slower and a lot more memory hungry. This blog post illustrates the issue with a short example, where processing 3 non-trivial actions on a list in parallel is slower than doing it sequentially, and then explains why this happens and what can or can’t be done about it.

Most Liked

dimitarvp

dimitarvp

Come on now. Unless you have a strict one-thread-pinned-per-core runtime then you can’t claim it for any runtime, Golang’s and Rust’s tokio’s included.

Fact is that most parallel runtimes do parallelization on a best-effort basis and they do a damn good job at it. There was a rather hilarious article a while ago reposted on HN how the Linux kernel never used more than 4 (or 8?) cores for a while, some years ago. What can a runtime do if the kernel is lying to it? But that’s a separate topic.

So… 100% guarantee? No, but it’s at least 90%.

lud

lud

No as long as you stay in the same process then you do not care, basically, as the data is not copied. The closure will use the same pointer to the data as the parent scope does.

D4no0

D4no0

Using the term parallel in this context is incorrect, the correct term would be concurrently and this has some very important implications:

  1. You are not guaranteed that the spawned tasks will be running on a separate physical core, as this is decided by the scheduler, there are ways to configure this manually, however you will be breaking the abstraction level of concurrency used in elixir and potentially introduce locks;
  2. As the scheduler switches context between running processes, every process will get a slice of execution time, naturally having more processes on the same thread will make them run slower.

Where Next?

Popular in Blog Posts Top

mudasobwa
Wrote some rant on Business Process Driven Development with Finite Automata (and Finitomata library in particular.)
New
brainlid
There is a new community resource available on writing “Safe Ecto Migrations”. When we get a migration wrong, it can lock up your product...
New
brainlid
Building a LiveView powered chat app is easier than ever when using Streams! Sophie DeBenedetto shows us how in this article. She createa...
New
JohnnyCurran
I’ve been writing LiveView since 2020. In that time, I’ve seen the same three form mistakes at multiple companies. Here’s what they are a...
New
rms.mrcs
https://medium.com/the-realreal-product-tech-blog/monitoring-hackney-connection-pools-with-telemetry-3aaeafa8eeb8 Hi, everyone! In this ...
New
ConnorRigby
Just finished a post for Embedded Elixir and I thought it should be cross-posted here: https://embedded-elixir.com/post/2018-09-25-mocks...
New
brainlid
This post asks if we can remove Alpine from the PETAL stack. Can we do everything we need with just LiveView? Also, let’s explore an area...
New
mudasobwa
Blogged about the motivation and reasoning behind my idea to create yet another FSM library. Long story short: I did it in a proper way :...
New
brainlid
Dialyzer is a tool that you’ve probably heard about in the Elixir community. You may have even used it. However, adding Dialyzer to an ex...
New
bemesa21
Add drag-and-drop functionality to your Phoenix LiveView app with SortableJS and LiveView Hooks! This posts shows you how to create an in...
New

Other popular topics Top

sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 29603 241
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
boundedvariable
I am going through the kafka architecture. All the features what the kafka is providing are already in Erlang. I would like hear your opi...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New

Latest on Elixir Forum

Elixir Forum

We're in Beta

About us Mission Statement