Advise about an approach for background data processing

  1. It’s a web application. In a database there’s some data that needs to be processed by sending http requests to a remote server in the background, say, every 10 seconds. And depending on a response from a server, the flag “one_more_iteration?” for each record of the data should be set false or remain true. If it’s true, one more request should be sent and so on.

When new item is inserted in the db, it’s “true” by default.
There might be a time when there’s no data to process.

That’s it.

There’s no direct interaction with an end user for the GenServer. A user can only insert a row in database by pressing a button on UI and see data from the db, and how it changes.

What would you recommend: GenServer? With Task.await/yield or spawn? Or anything else?

  1. Is it a good idea to make a db request upon application start to feed it to a GenServer?

    def start(_type, _args) do
    import Supervisor.Spec

     data_to_be_processed = where(...) |> Repo.all() 
     children = [supervisor(MyWebApp.Repo, []), supervisor(MyWebApp.Endpoint, []),
             worker(MyWebApp.MyGenServer, data_to_be_processed)]
    
     opts = [strategy: :one_for_one, name: MyWebApp.Supervisor]
     Supervisor.start_link(children, opts)
    

    end

If not, then how else?