yurko
GenServer advice needed - relationship between handle_call and handle_info that use the same state
Say I have a message queue implemented as a GenServer, something like this
#... def start_link ...
def init(state) do
{:ok, schedule(state)}
end
# schedule processing every 500 ms
defp schedule(state) do
Process.send_after(self(), :process, 500)
state
end
def handle_info(:process, state) do
{:noreply, process_queue(state)}
end
# catch all handle_info
def process_queue(state) do
# concurrently processes each entry and deletes it from the state
# schedule(state)
end
def put(data) do
GenServer.call(__MODULE__, {:put, entry})
end
def handle_call({:put, entry}, _from, state) do
{:reply, :ok, [entry | state]}
end
The question is about the relationship between handle_call and handle_info that both use the same state. From what I understand there is no way the state breaks, meaning while process_queue does its job (which may take a while) any put calling handle_call will have to wait, is that right?
If I want put to return immediately and add entries to queue after handle_info is done, I’d have to switch from call to cast, and I’m good?
Is there a nicer / simpler approach to achieve that or is this one good? Shouldn’t I handle such state using ETS (that’s what I’m switching from at the moment)?
Most Liked
Qqwy
Correct!
Yes! 
Not necessarily. Using ETS makes sense if you have a lot of data that you might want to search through. If you have a queue of ‘jobs’ (of whatever kind), then simply using a single process whose state is a list, map or queue should do the job just fine, without adding complexity of using a system ETS for the developer.
Depends on what you want to archieve exactly: If you want the items to be processed and they can be processed completely separately, then you could use Task.async or Task.async_stream to do the job, which means that process_queue will not block for long.
On the other hand, if it was possible to process items completely separately from the start, one could create Tasks right away, and not need to bother to use a queue at all.
So: What is this job queue for, exactly?
OvermindDL1
For the map version earlier on, if the actions you perform are already communicating to another process then you are already async, so you definitely should not use Task then as that would just slow it all down.
yurko
Some background: I am using parts of a working app in another, bigger app so some solutions have to be refactored or reimplemented.
I have lots of data, but it’s pretty uniform and I have to handle each entry all the time, no searching. Complexity of ETS wouldn’t be a problem though, because the current implementation already uses it, still makes sense to stick with GenServer’s state?
I will use Task.async_stream here, there is no question about that The jobs are separate and each one might have to work with an external API so concurrent processing is a must, but still even asynchronously it will take some number of milliseconds to finish.
It was a “no queue” implementation originally I don’t want to bore you with too much details but here is one case that justified the use of queue: since I have to use an external API it might have temporal problems or even one time errors, so at times I need to fallback to a simpler handler for a specific task and at other times - just keep an item in queue and schedule it for the next run since the error must be temporary but I don’t want to block for long.
Anyway thanks for the feedback, it seems there are no obvious problems with the above solution and I might use it.
I might also stick with mutable ETS state, the only problem with it is that it’s somewhat verbose to iterate over. Which one would you go with, @Qqwy ?
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Phoenix, LiveView, multimedia pipelines, distributed systems. 400 engineers, Elixir since 2017. Available for hire.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking for Elixir devs who love to ship. Start your free account.
Develop your skills with books, videos, and courses.
Producing high quality Elixir screencasts since 2017.
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"