I have a use-case where I want to initialize a GenServer to manage the process of fetching data from a third party data source. A single GenServer should manage a single user’s data. I want to supervise the GenServer and I want the event which initializes the GenServer to be receiving a webhook JSON message on my Phoenix http server.
I’m having trouble grasping the best way to implement this based on my knowledge of supervision trees. For example, should I initialize a supervision tree with a fixed pool of GenServers when the application starts and allocate those servers as resources, or should I spawn a new server under the supervisor every time I need to fetch a user’s data and terminate the server afterwards? In the latter case, I could potentially have thousands of servers running under a single supervisor - is this acceptable?
My user needs to complete a workflow including an authentication. It’s possible that a webhook response with the user’s data is received before he completes the workflow. Embedded in the user workflow is information critical to processing the webhook response. I figured a server was the best way to maintain state until I had sufficient information to process the webhook response. The server will also be useful to manage a call to the third party data source.
I find architecture questions helpful, so I’m gonna close the loop here on how I solved my problem.
Thank you, dom, for pointing me in the direction of Tasks.
I have a separate application that launches with my phoenix webserver. This application is initialized with a supervised GenServer. When phoenix receives a JSON webhook message, it sends a message to this GenServer with an access token as the message body. The GenServer stores this access token in its state and spawns a supervised Task to fetch the corresponding data for the token. The GenServer receives messages from the Task (either success or failure) and either deletes the token from state (indicating it has been processed successfully) or keeps it for reprocessing. The GenServer constantly numerates through the tokens in state trying to fetch data for them.
