polypush135
Proper use of Task.async?
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a large list of similar but different tables.
One suggestion I’ve received for dealing with this task was to create a module that would query concretely each table then aggregate them into a single list at the app level vs at the db via unions and view tables. The suggestion was to use Task.async.
Since my max concurrent users will never go above 1000 at any given time I felt this could be a good solution for my use case. I should have enough pool workers to deal with the load of requests to query the db concurrently. I realize this will have cpu overhead but the trade off of how I could do the query will be a good tradeoff since I have a very large list of tables that I will need to manage and rewriting a very long union query does not seem like the best way to manage this. I know little about Tasks and I’m just starting to learn.
So back to my question:
If I have a single function that delegates multiple concurrent queries to compose a single result set, is this a good use case for Task.async?
Also I’ve seen Task await but still lack a good comprehension about it. If I want to block my function until every async task has completed and then composed a new list of results what does that look like?
IE: if a wanted to make a function that took a list of number and squared each value and returned a new list of all the squared values (“regardless of order”) via using Task.async what would that look like?
Thanks for your help in advance, this community awesome !
Best - Josh Chernoff.
Trending in Questions
Other Trending Topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #elixirconf-us
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #hex
- #performance










First Post!
idi527
Maybe something like this
Note that
Task.awaitby default waits for up to 5 seconds, then it times out (exits).Most Liked
crazymevt
I personally like using async_stream, I have found I use this quite a bit
This will just keep kicking off the functions with a maximum number at a time (in this case 10) and then gathers all the results into a list. No need to call task.await at all.
peerreynders
A slightly more complicated example:
peerreynders
Caveat:
Compatibility with OTP behaviours
If you need to go that route have a look at:
Last Post!
nikitaavvakumov
@peerreynders Thanks for your example! It really helped me to work through a similar scenario.