sukhmeetsd
All in all, from what I understand, it is better not to use GenServer.cast when we want some concurrent operations to happen for sure, because we don’t know when the messages get lost in the air. So what is better to be used in case of concurrent operations? (I am trying to find all the Vampire Numbers in the range 1000000000 to 2000000000 using the power of concurrency)
Trending in Challenges
Other Trending Topics
Hey there,
I haven’t seen a list of useful Elixir specific skills for the popular AI tools yet so I’m starting one here with my first.
...
New
Hi everyone :waving_hand:
I wanted to share something I’ve been building — NowBlind, a random chat app designed to help people discover ...
New
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
(I just needed to vent somewhere and LinkedIn is full of hope, or hype, I’m not sure which exactly)
AI dream has many faces, but general...
New
Localize is the next generation localisation library for Elixir. Think of it as ex_cldr version 3.0. The first version will be released ...
New
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #code-sync
- #discussion
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #ai
- #elixirconf-us
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #blog-post
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #security
- #elixirconf-eu
- #processes
- #api
- #forms
- #hex











Showing Posts 1 to 9- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
NobbZ
This is true for any message sent. Erlang messaging between processes is like sending mail to a friend.
If you don’t receive a reply, you’ll never know if your letter got lost or the reply, or if your friend just didn’t sent it.
But still for such a thing I consider something like GenStage or Flow is probably the best approach.
peerreynders
It’s important to realize that on
GenServer.call/3processing in the calling process blocks:sentSo the calling process isn’t doing anything between the sent and return message.
Sometimes it makes more sense to monitor the process responsible and using a timeout to determine whether the result will be available (in time). That way the casting process doesn’t block (but the process interaction protocol is more work).
sukhmeetsd
I tried using handle_cast everytime I was checking a number and I would update the state whenever I found a Vampire number. Later at the end I would make a Genserver.call to get the final state. But most of the time the genserver.call timed out. When I tried printing things inside handle_cast, I found that the request coming to handle_cast stopped coming even before the range (arg_n to arg_k) was over. Hence I was a little confused about the working of handle_cast and GenServer altogether. That led me to your post here. Code snippet below:
I am now planning to
spawn(__MODULE__, :fun_to_check_vamp, [number]). This will potentially create a new process for every number that will be checked in the range.peerreynders
If I interpret your code correctly you were just piling up
:check_vamprequests inpidprocess’s mailbox. It’s just a single process and code inside that process is strictly sequential.Likely you where processing a request immediately and the next request wouldn’t be touched until the current request finished - so the process was too busy to run
handle_castagain (while more and more requests sat in the mailbox). You might as well just have run your code within a single sequential process.By the time you hit
call/3pidwas hopelessly backed up with:check_vamprequests so that it couldn’t respond tocall/3within 5 seconds.Please have a look at Task:
That being said these type of coding challenges typically rely on algorithmic improvements that will often outperform simple parallelization of brute force methods.
sukhmeetsd
Thank you for that amazing realization! I was under the impression that because every request going to Genserver.cast is being handled asynchronously, so there will be no interaction between them and the code execution would be quicker, but the process was stopping abruptly in between.
I am definitely going to adopt the Task approach but for now I am running the following code for 10e7 to 2*10e7 and it is taking a long time as well.
I am afraid if I did something wrong. I have switched to check-till-squareRoot approach now instead of checking all permutations (which is a huge improvement when working on one number) but still, for that range it is taking some significant amount of time.
EDIT: It took 30 minutes!
peerreynders
You are still running sequentially.
You are spawning a single process then entering a blocking receive and waiting for the result from that process. Only after the blocking receive is exited can the code advance and spawn the next process.
Now the BEAM can handle a million processes but given the circumstances it isn’t efficient.
Have a look at
Task.Supervisor.async_stream/6. It allows you to constrain the number of tasks to the number of schedulers/cores to have the most efficient execution.https://medium.com/@dinojoaocosta/elixir-findings-asynchronous-task-streams-7f6336227ea
sukhmeetsd
I am awestruck at the improvement! From 30 minutes to 7 minutes. Thanks again for this wonderful thing!
It gives us the freedom to decide the number of concurrent processes too (as you very generously highlighted). Just to make sure I am not leaving any other stone un-turned, here is what I am doing:
It is printing the result in vamp_check itself hence the nil in Enum.map.
(To be noted that my laptop has System.schedulers_online=12)
sukhmeetsd
I noticed something while execution of the individual processes - they use 11.5 of my 12 cores effectively. I am doing this:
(without the receive blocking this time).
It sounds very disadvantageous to know that when I use Task.Supervisor.async_stream or Task.async_stream (as above), I am only able to utilize maximum 2 cores effectively. Is there a reason for that? Can it be improved?
Here is my Task.Supervisor.async_stream code:
I am using the time command in Debian shell on my Windows system to calculate the effective CPU cores being used ((user time+sys time)/realTime).
peerreynders
It’s kind of difficult to judge without access to your code - if for some reasons you used blocking operations those 12 processes may not be generating enough work to warrant more cores.
This script floods the cores of my 2016 MBP with work:
Note: