PragTob
Blog Post: Careful what data you send or how to tank your performance with Task.async
I ran into an interesting problem recently where simple concurrency on the BEAM via Task.async made my application a lot slower and a lot more memory hungry. This blog post illustrates the issue with a short example, where processing 3 non-trivial actions on a list in parallel is slower than doing it sequentially, and then explains why this happens and what can or can’t be done about it.
Most Liked
dimitarvp
Come on now. Unless you have a strict one-thread-pinned-per-core runtime then you can’t claim it for any runtime, Golang’s and Rust’s tokio’s included.
Fact is that most parallel runtimes do parallelization on a best-effort basis and they do a damn good job at it. There was a rather hilarious article a while ago reposted on HN how the Linux kernel never used more than 4 (or 8?) cores for a while, some years ago. What can a runtime do if the kernel is lying to it? But that’s a separate topic.
So… 100% guarantee? No, but it’s at least 90%.
lud
No as long as you stay in the same process then you do not care, basically, as the data is not copied. The closure will use the same pointer to the data as the parent scope does.
D4no0
Using the term parallel in this context is incorrect, the correct term would be concurrently and this has some very important implications:
- You are not guaranteed that the spawned tasks will be running on a separate physical core, as this is decided by the scheduler, there are ways to configure this manually, however you will be breaking the abstraction level of concurrency used in elixir and potentially introduce locks;
- As the scheduler switches context between running processes, every process will get a slice of execution time, naturally having more processes on the same thread will make them run slower.
Popular in Blog Posts
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance









