Many applications and few services or many services under one application?

I have this one application in an umbrella project that has 12 workers under one supervision tree: Auth, Guard, Queue, Cache, Storage, Mailer… etc. They provide a “platform” for my other apps (in fact the application is named :platform)

Should I split those GenServer services in their own application with their own supervision tree?

Sincere question: how do you know when you should create a new application or just append your genserver to an existing app? Do you have a rule of thumb?

2 Likes

The simple answer is when communication is a problem.

Let me try to explain.

First of all, the Erlang actor model and OTP makes this a different conversation than say Ruby, PHP, or the like.

If we look at the big picture, splitting off services is about isolating functionality into separate parts. Then the hard part is communicating between these separate isolated programs.
This is what Erlang does. In your case, each GenServer is an isolated program with possible access to a shared storage, which is also another GenServer. So your application is already a set of services with the communication handled by Erlang.

When should you split functionality out? On a non-OTP application my answer would be:

  1. You have multiple communication avenues, with different requirements, e.g., Multi-Language Environments.
  • You need to perform async tasks and workers aren’t enough or don’t want the overhead of reloading your entire application into a worker to send a simple web request.
  • There’s a series of complex tasks that often change independent of the application.
  • When you are deploying 2-3 times a day for different components and there’s enough overhead to be annoying.
  • You’re planning a rewrite and can’t afford to take your huge complex system and supplant it with another huge complex system.
  • You’re worried about one systems uptime independent of other systems.
  • Your organization’s communication structure is compartmentalized. (See Conway’s Law)

Now, with Erlang & OTP you basically rule out reasons 2 and 6. Erlang pushes you into proper use of the actor model plus has the ability to hot code swap, so these are problems that you may be more likely to solve using the language.

EDIT: I meant to add that 3 and 5 are solved by proper use of the actor model.

Which really just leaves your large deployment is a huge pain, multi-language environments, and organizational structure.

Deployment problems are complicated and diverse, you may or may not be better off splitting things out. More often than not DevOps and deployment just needs time or money thrown at it.

So, the simple Erlang/OTP answer for me is communication. When two applications need a common communication layer or when the communication structure in your organization would be better suited with different applications.

6 Likes

That’s a really helpful overview of the issues to look at when considering whether to split out a piece of functionality into a separate deployable entity—the traditional sense of “application”.

It seems to me, however, that @rms.mrcs intends to stick with a shared Erlang/OTP architecture—I think he was asking when it is appropriate to split out modules into their own OTP application—one with its own supervision tree, and managed as a separate dependency possibly.

1 Like

Thanks @mjadczak, that’s a good point. Rereading what I wrote, I definitely wrote for more of a overall multiple application and not OTP umbrella applications.

So, I’ll try to expand and clarify.

If there is a unique way you need your GenServers to communicate then create a new application. (e.g. Queue is actually a complicated worker queue serving more than the :platform application)

If it would be easier to hot deploy one GenServer separate from the others, make a new application.

If separate teams in your organization are siloed to separate GenServers, make a new application.

Finally the tried and true, I need to access this application to be used with several other applications. (i.e. I need to extract this library to use somewhere else.) Which is a more nuanced Organization reason.

Of course, there’s always the subjective I just feel like it reason.
In which case go with your gut instinct, it’s probably the right answer. Mostly because there’s no wrong answer and you get to be happy doing what you wanted to do.

But, those are all just my rules of thumb and how I decide. Keeping them all together looks fine because you have no concrete reason to split them besides it feels like a good idea. Which is just as viable a reason to split them up.

The one warning I would give about when NOT to split applications, don’t ever let two applications be codependent. If you have two codependent applications, then you should probably consider merging or refactoring them.

3 Likes

@Azolo thanks for the awesome answer :smiley:. As @mjadczak wrote, I’m talking solely about OTP applications in an umbrella approach.

This is a bit confusing for me because most part of the examples I see out there consists of very few apps, usually a server, lib and a web endpoint using phoenix.

Just to clarify: the :platform application has this one supervisor with all “core” features. There are other apps for the domains (user accounts, cms, profile etc.) and also for the API and Web components. I have :platform, :accounts, :web and so on.

If I understood correctly, if a feature is so important then it probably should have its own application and supervision tree.

Ah, this is a good point.

My only concern about team organization is the deploy process, as different people will be working on different parts and they need to be updated at some point.

So if one feature is strong candidate to be shared it’s a strong sign that it should be an application.

Sounds right, just like a library.

So wrapping it up, I have now this checklist:

  1. Do I need to update/deploy this feature independently?
  2. Is this feature complex enough to be a totally different project?
  3. Does this feature feels like it could be used somewhere else?

Of course it will vary on the specific case, and then I have to count on my feeling. But anyway you made an excelent explanation :smiley:

The good thing is that Elixir makes refactoring so damn simple that refactoring those features into new applications is actually pretty simple (and fun!) imho.

Thank you very, very much :heart:

Seems like pretty good guidelines. Glad I could help.

Just to point out, the examples I laid out in my second post is basically from applying the points from my first post with the addition of an application could also be a portable library.

The reason this works out so well is because my vision of OTP/Erlang is basically this:

Every long running process is a micro-service.
Every short lived process is a worker.
Anything else is just a communication mechanism.

But coming from a non-Erlang language, that’s just the lens that I view the applications I write with.

4 Likes