LJ1990
How do you push data to external APIs?
In one of my side projects I want to index some data to Redis whenever a DB entry is created/updated/deleted (the same scenario is also applicable if the external API is a Elasticsearch/Algolia endpoint).
For this I am calling a function in my Redis module within my create/update db functions.
Since indexing does not need to happen synchronously, I thought about speeding up the e2e response time by wrapping the call to the Redis client in Task.async/1.
My question is now: What happens if a lot of DB entries are created? In this case it would be better to bundle all the data to be indexed and then send it over the wire within a single batch request.
Would you then reach for different modules like GenServers? Is Flow or Broadway a good use case for this?
Would you use Task.async in my scenario at all? Or do it with the Task.Supervisor module to add some additional resilience?
Also keeping in mind that you do not want to exhaust the external API.
Thoughts?
Where Next?
Trending in Questions
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
We build reliable cloud platforms for business-critical systems.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #blog-post
- #elixirconf-us
- #elixir-ls
- #ai
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #hex
- #security
- #metaprogramming
Our Sponsors
Build Elixir applications with speed and confidence.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Elixir, Erlang & BEAM conferences from Code Sync - join us in 2026!
Master modern tools, write cleaner code, and tackle real-world challenges.
Bringing learning, work and compliance together since 1987.
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 4- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
hauleth
Do not use
Task.async/1as this spawns task outside of supervisor tree, useTask.Supervisor.async/{2,3}.Yes, it would be better to do so.
Yes. Create cache that will run function to reindex data that will debounce/throttle data. It can be
GenStage, but do not need. All you need could be achieved via:gen_statem.I would never use
Task.async/1in production.ityonemo
I would use Task.async in tests, or in the REPL.
hauleth
In production
ityonemo
Sorry I read op as asking if you would use Task.async in “any” situation, misread “my”.