I’m looking to make an app that has to be as fast & fault tolerant as it can be.
Speed & fault tolerance will be the most important metrics of this project. With that in mind, I plan on a distributed node structure using OTP.
I’m currently using VueJS on the front end. I’m learning Websockets now to setup communication between Phoenix & VueJS.
I just came across ZeroMQ. From the surface, it looks like it would be perfect for a distributed node structure. However, I’m speaking from a newbie perspective. With that in mind…
Is it worth architecting the project using ZeroMQ from the start? Anything I’m losing by going with ZeroMQ from the start?
P.S. I got hella lot out of the last Websockets / AJAX thread… I appreciate the expert perspectives here.
I think that the best choice you can make here, is the one where you do not commit to either technology upfront.
Does this sound strange? Then I advise reading the article ‘A little Architecture’ by Uncle Bob.
Your application should not be dependent on the messaging technology you use. Rather, you should specify exactly what your application requires from a messaging layer (in Elixir, using a Behaviour is the common way to do this) and then implement this behaviour for one or more messaging layers. This will allow you to try out multiple possibilities, and maybe switch later if something changes or an even better alternative comes along.
Great article. So in practice for this particular problem, it would look something like this:
I define the exact messaging needs of the application.
I segregate the messaging needs.
I make a separate layer with functions for each need.
I plug & play ZeroMQ, RabbitMQ, or whatever into those functions. Making it so they can be changed out easily with little refactoring, in one central place.
Yes, exactly! I do not think you are missing anything. Another advantage of this approach is that it becomes trivial to test: Swap the messaging implementation out for a fake one that does not send messages to the outside world to test your app’s message sending, and call the real one’s behaviour functions directly with simple examples to test that that side as well, if desired.
You can specify which messaging adapter you use in your application’s configuration file, to make it as decoupled as possible.
My gut reaction is to say you probably won’t need an AMQP system unless you have several services and need a Service Bus of some kind. I find there are better tools for other use cases unless you already have an AMQP system in place.
Truer words have never been said for almost every project out there.
Just a tip, fast isn’t a good metric. The metric you should be looking at is slow. I could spend all day making something faster, and gain a couple of milliseconds and make my code a mess. But, if I look at what is too slow for my needs then I will probably find easy optimizations. On top of that, development speed and happiness is a better metric when my application is already fast enough.
What particular thing are you trying to do in ZeroMQ? Can it be accomplished with OTP?
I think you’re going to fall into the trap of “I don’t understand how to do X, what tool can make it easier?” OTP has really good node-to-node communication. You probably don’t need an extra layer on top.
Development speed. Learning another technology is a huge time sink.
They’re all going to stay active enough to be stable. They have all been around for a long time and are used actively. Technologies die, but once they’re being used by people they are usually longer lasting than most projects.
You will probably learn a lot of new things or new requirements pop up and you slowly rewrite your application because the tool is no longer useful for your use case before they die.
I would love to tell you the trick for future proofing and maintainability, but I don’t think there is one. Your intuition and judgement are the best tools and those come with experience, usually bad experiences at that.
If speed is more important, then work on making a system that is fast first. You can add reliability later. I would say that speed isn’t that important though. Likewise, when is fast enough good enough? We are probably talking about an overhead of tens of milliseconds again.
Also Task and Task.Supervisor are probably enough.
This advice shouldn’t be just applied to external dependencies. This becomes easier with experience, don’t try to be perfect the first try. Usually this also happens when writing testable code. Just make something that works, then feel the pain when you have to change it.
Likewise, consider the overhead. Sometimes it’s easier to have to change things in 5 places than figure out how to do it in one place.
Shipping is hard, but it’s usually easier to just make something in the worst way possible then rewrite it than it is to add all these dependencies to figure out your initial design decisions weren’t the best.
