Appologies, I found this reply apparently un-sent when I visited forum again. Don’t know if it was sent and cancelled again because it came from me or I never pushed the button.
Hey @julismz,
It certainly is a journey, not one event or decision. I’m sory to hear of your performance hurdles and poor results, but I’d need to get quite a bit deeper into what you’ve really been experimenting with.
There never was and still isn’t any nginx in my solution. How/why did nginx enter the mix for you?
As for the customers per node question you’re right on the money identifying that as the key question. It’s a lot like engineering a race-car’s drag coefficient. You can have an idea of the Cd for simple shapes but when many shapes overlap and interact it becoes a rabbit hole which if you do make it out of will leave you with unreliable predictions anyway.
The only application that performs without bugs is the null application. From there, everything your app has to do in order to add value will lower it’s performance. It’s up to you as designer to try avoid giving up large chunks of performance for no or questionionable value to the end-user. You cannot increase performance, only decrease it less by making smarter choices which avoids unnecessary work. In most cases, unnecessary work equates duplicated work so that’s usually a good place to start.
Perhaps your application is somple enough for you to know in advance what every user will be doing which means for you the only parameter becoes how many users are active. If that’s the case we can discuss strategies for that but there are probably many oters that can help you more than I can.
Each of my users use my application in their own unique way which I can neither predict nor directly and it varies day by day. That variance impacts my “optimal” ration between nodes and users far more than my code does so as long as I don’t waste computing cycles I’m doing as well as I can.
That then engages the third element we can call pre-scaling if you want. As you might have seen above in this thread I’ve been criticised rather heavily for it as premature optimisation. The essense of it is to design your application assuming it will need to scale bigger than you could have imagined and then implement it at the smallest possible scale you can pull off. The rationale is simple. Even if you cannot predict how many live customers a single node will service you can measure how it’s doing and if it starts to struggle you can add additional nodes without delay. You can only do that if even for your initial version you’ve made provision for the application to run on multiple nodes and in multiple regions. If you don’t do that from the start you will get trapped. Your users will demand better performance but to give them that you need to redesign your app or find a way to give your server more resources (vertical scaling) which will in the end put you exactly where the cloud providers want you - dependent on them for scaling.
I’ve seen too many designers and managerial types with unrealistic expectations of the agile and “first make it work, then make it fast” to not say anything about it when asked. I’ve even worked with a company selling an “Agile product” that declared war against Architects as the enemy of Agile. But as an Architect using Agile I did things with their product they couldn’t believe, so I developed this analogy: When tasked with bulding a dam it’s easy to find a hole in the ground, fill it with water and say that wasn’t hard at all, now we just need to evolve it into a dam at the desired scale. If you cannot see the issues with evolving a filled dam or the simple matter of where the water will come from if your initial hole in the ground is not in an opportune geographical location, you deserve the disappointment that will follow. If you need a dam, your starting point is to find the right location, then to design whole dam, then the way to divert the water while you build the dam and only then to start construction of the dam even if you build it in phases on top of the full foundation started in the right spot.
Design your application’s value to your customers and let that guide you as to how to deliver that value at a lower cost than what your customers pay you for it.
A lot of Kubernetes’ facilities are there to accomodate legacy applications such as the monolythic web servers of yesteryear and “micro service” stuff written in C++, Java and go. Don’t get distracted by all that noise. Erlang, Elixir and Phoenix already provides a tonne of facilities which those applications never had and had to either implement at application level or get from a container environment when they needed to head for the clouds. As a result, a lot of the concepts like nginx (as Ingress controller) and NodePorts have wide-ranging capabilities in order to adapt to the enormous variance with which application designers had gone about they business before they involved Kubernetes. Your Phoenix app needs surprisingly little mangling to do well in Kubernetes.
Away from the public cloud providers on your own clusters MetalLB is your friend and you have a few good options as to how to create your kubernetes cluster. Once you are using a public cloud, use their kubernetes stacks and load balancers directly. It’s silly to implement your own stack on cloud-based servers.
You also need to choose a database strategy and a storage strategy to suit your cloud-agnostic ideals. It largely depends on your applications’s actual needs and the capabilities of your own hardware. For my purposes I chose to a “standalone” PostgreSQL deployment in Kubernetes using Percona’s Operator for PostgreSQL local to each regional kubernetes cluster and share data betweeen them at the application layer only. As for storage I’ve gone and stuck with Ceph because that’s what best suits my hardware but because Kubernetes abstracts that so well through storage classes and providers I’d use whatever the cloud provider offers where I need to use a public cloud’s kubernetes stack.
I don’t know if I’m really answering your questions and I don’t even know if all your questions have the kind of answers you’d like them to have. Both cloud-native and distributed computing (very different things, but related anyway) can overwhelm you exactly like the monsters they’re reported to be. They certainly are powerful but they’re mostly misunderstood which makes them scary. Once you know enough about them they’re incrediby good friends to have on your side. It just takes a little work and empathy to get to know and understand them.
Good luck with your journey and dpon’t hesitate to reach out (again) if you feel stuck or uncertain. Just bear in mind that the bulk of the information you will encounter on the interwebs have been written for purposes other than your own. This too. My purpose is the hope spark a kinship among fellow travellers on the good ship Phoenix sailing towards large scale independent federated systems such as my own. At some point we’re going to need others who look at the problems we’ll face through the same colour lenses.
