Hello, I’m doing an umbrella project, in which I have doubts about the architecture that I have defined, given my still little experience with Elixir, I would like you to give me your comments about it and they will help me with a little light on my way.
The project consists of an API that after a small process, writes the result in SQS. I have other “workers” which consume the SQS information, process and write in a DB.
My question is mainly focused on the issue of supervisors, since I understand that Phoenix manages its own supervisor, but I have doubts about whether I should have a Supervisor above it, to “monitor” the API and the workers.
Do you have experience in architectures of this type ?, where can I start to modify and improve?
Finally, if you could also support me with resources to improve my “thinking” for architectural design with the “Erlang / elixir” ecosystem, it would be incredible. Thanks for the time and support
You probably don’t need a supervisor above your Phoenix application. The OTP work in Phoenix has already been done for you: each request is handled concurrently and failure contained. As long as Phoenix is part of the umbrella you’re probably good to go.
You might want some kind of supervision tree above the SQS workers depending on how you’ve implemented it. You could use something like Conduit to do the message queue OTP work for you. Your publishing mechanism can also be ran through Conduit. That’s probably your lowest effort option. Just wire it up and you’ll have a solid application. You could also implement the worker pool yourself with Poolboy but expect that to be more work.
If you want to learn more about worker pool patterns I recommend reading David Sulc’s PoolToy series and Learn You Some Erlang.
Thank you very much for the answer, I think that leaves me in a better perspective to continue, since you clarified my main question with the subject of a supervisor over what Phoenix already has. And with the material that you suggest, I think I can have more clarity for this type of architectural design.