How would you scale up an Elixir/Phoenix application if only a specific part is responsible for that requirement?

Questions / Help
1

Hi everyone, I’ve got a question related to project architecture.

In the microservice sphere there is this concept where you can scale up individual services when needed. So imagine the following scenario:

     ┌────────────────────┐
     │RabbitMQ Message Bus│
     └────────────────────┘
       ▲            ▲
       │            │
 ┌─────┴───┐    ┌───┴──────────┐
 │Webserver│    │Expensive jobs│
 └─────────┘    └──────────────┘

The precise job doesn’t really matter, but let’s assume that we’re processing and/or modifying images or something.
The ‘expensive jobs’ box retrieves jobs from RabbitMQ and returns the result.
Let’s say that the expensive jobs get popular with our users, so we want to add another box that retrieves jobs from RabbitMQ.

     ┌────────────────────┐
     │RabbitMQ Message Bus│
     └────────────────────┘
       ▲            ▲   ▲
       │            │   └──────────┐
       │            │              │
 ┌─────┴───┐    ┌───┴──────────┐ ┌─┴────────────┐
 │Webserver│    │Expensive jobs│ │Expensive jobs│
 └─────────┘    └──────────────┘ └──────────────┘

We’ve now effectively doubled our capacity for handling these expensive jobs.

My question is as follows: how would you achieve something similar in an Elixir (Phoenix) monolith? We can use Oban to perform background jobs, but what if the amount of jobs becomes sufficiently large that the queue reaches its limit. We could theoretically scale up the entire server, but that doesn’t feel correct for some reason.

2 Likes
2

I’d argue at that scale the difference between a monolith and microservices becomes muddy. Because the remaining difference will just be “code organization” when you start to scale out a system (especially heterogenously).

Afaik both Oban Pro as well as the recently introduced FLAME library/pattern can help with adding more nodes (horizonal scaling) over adding more power to the current one.

3 Likes
3

Why? The expensive jobs will take a bigger slice of all the available resource. I don’t see anything got wasted. If you can get away with scale up, why scale out?

1 Like
4

I agree. Why take the penalty of managing multiple servers when you can have the Simple Life (r) with just a beefy one?

5

You could deploy a monolith 100 times and set aside 90 of those for jobs if you wanted. Don’t be bothered with whether there is one service deployed and running or many on a container / vm. There is a time for that, but not worth it early. Colocating services / data also is important if latency is there when scaling.

Scaling up is much easier than scaling out. Agreed with everyone else in the thread so far. You don’t want to be paying for giant machines to be idle 99% of the time though.

My preference would be deploy the monolith n-times and separate the usage / clustering. FLAME looks interesting also, but never tried it.

2 Likes
6

The BEAM VM has work-stealing schedulers (among other goodies) so bumping up a single server’s specs will naturally lead to increase of capacity in processing the expensive jobs, because they’re the ones demanding more resources, and the runtime will give it to them.

Delay scaling out as if your life depends on it. New and extremely annoying problems await you there.

3 Likes
7

:+1: Fair enough, and the BEAM has the advantage of scaling horizontally rather easily (compared to other solutions anyway) should you reach the limit of upwards scaling