I think that the best choice you can make here, is the one where you do not commit to either technology upfront.
Does this sound strange? Then I advise reading the article ‘A little Architecture’ by Uncle Bob.
Your application should not be dependent on the messaging technology you use. Rather, you should specify exactly what your application requires from a messaging layer (in Elixir, using a Behaviour is the common way to do this) and then implement this behaviour for one or more messaging layers. This will allow you to try out multiple possibilities, and maybe switch later if something changes or an even better alternative comes along.
Yes, exactly! I do not think you are missing anything. Another advantage of this approach is that it becomes trivial to test: Swap the messaging implementation out for a fake one that does not send messages to the outside world to test your app’s message sending, and call the real one’s behaviour functions directly with simple examples to test that that side as well, if desired.
You can specify which messaging adapter you use in your application’s configuration file, to make it as decoupled as possible.
My gut reaction is to say you probably won’t need an AMQP system unless you have several services and need a Service Bus of some kind. I find there are better tools for other use cases unless you already have an AMQP system in place.
Truer words have never been said for almost every project out there.
Just a tip, fast isn’t a good metric. The metric you should be looking at is slow. I could spend all day making something faster, and gain a couple of milliseconds and make my code a mess. But, if I look at what is too slow for my needs then I will probably find easy optimizations. On top of that, development speed and happiness is a better metric when my application is already fast enough.
What particular thing are you trying to do in ZeroMQ? Can it be accomplished with OTP?
I think you’re going to fall into the trap of “I don’t understand how to do X, what tool can make it easier?” OTP has really good node-to-node communication. You probably don’t need an extra layer on top.
Development speed. Learning another technology is a huge time sink.
They’re all going to stay active enough to be stable. They have all been around for a long time and are used actively. Technologies die, but once they’re being used by people they are usually longer lasting than most projects.
You will probably learn a lot of new things or new requirements pop up and you slowly rewrite your application because the tool is no longer useful for your use case before they die.
I would love to tell you the trick for future proofing and maintainability, but I don’t think there is one. Your intuition and judgement are the best tools and those come with experience, usually bad experiences at that.
If speed is more important, then work on making a system that is fast first. You can add reliability later. I would say that speed isn’t that important though. Likewise, when is fast enough good enough? We are probably talking about an overhead of tens of milliseconds again.
Also Task and Task.Supervisor are probably enough.
This advice shouldn’t be just applied to external dependencies. This becomes easier with experience, don’t try to be perfect the first try. Usually this also happens when writing testable code. Just make something that works, then feel the pain when you have to change it.
Likewise, consider the overhead. Sometimes it’s easier to have to change things in 5 places than figure out how to do it in one place.
Shipping is hard, but it’s usually easier to just make something in the worst way possible then rewrite it than it is to add all these dependencies to figure out your initial design decisions weren’t the best.