Distributed elixir or Message broker for microservice communication?

As in distributed erlang each node has to connect to each other node, you’ll end up with connections you do not need.

Therefore, I’d go for REST, a message broker or any other ISC that does not require a fully meshed network.

A simple difference to think about is simply the pros/cons of using one technology vs. using two. From the standpoint of someone new coming into your system if you use Erlang and RabbitMQ (ignoring the fact that Rabbit itself uses Erlang) then someone must understand two technologies, two conceptual models, to fully get what’s going on. You also need someone to keep RabbitMQ up and running - and doing things like performance balancing would require tweaking configs in two different technologies.

If you use an external system for message passing then you have a layer in your system where you have to translate, or externalize, messages for the message bus, and where you transform them again when they come back in. (You might argue, however, that you would need such a layer even if the transport mechanism is erlang messages).

I am fairly certain that this only becomes a problem if you have 50+ nodes connected to one-another.

My suggestion is always, only use distributed erlang if it is for a control plane.

If you are talking between services there is a lot to say about being able to reuse existing infrastructure for **** like load balancing (like http and recently grpc, which I’d suggest personally, has more support).

Distributed Erlang gives you a single non-multiplexed connection between each node, so workloads that you’d expect to work mostly fine if you were doing http, like with large payloads, will fall apart.

There is work on improving distributed Erlang (chunking large messages was recently added and I guess will be in OTP-22) and there is partisan (http://partisan.cloud/) for a high performance distributed Erlang. But for what it sounds like you are building you will likely have success using http or grpc.

it seems that the answer is inside the question. You should decide for yourself: is your architecture is microservices or not.

In microservice architecture you are not limited for single language for each and every service, also your messaging bus also can be called microservice.
In this situation it should be obvious to have message bus as a side application.

If you are creating monolith in Phoenix, that you decided to start not on single node, but on 2 or 3 - it’s not microservice architecture, for sure. In this situation, empd is a simple and obvious solution, which will not bring you any problems with administration and orchestration.

So, as for me - the answer is inside an question.

Yeah whilst most of the stack is elixir/phoenix there are some other languages in there (with no gaurenteed interop however), so your right the best course of action is most likely going to be to use some sort of message broker

cheers!

Well a message broker like rabbitmq provides many features out of the box exampled priority queue, disk persistence, acks, complex routings etc. You will need to weigh in these features before deciding upon using pure erlang as you will need to write a lot of code for the above mentioned features.

Hey,

Your question is really interesting! I’m more surprised by the answers

From what I’ve read and heard. Erlang has been built for distributed systems, so from what I can understand it should be a perfect technology to build services/micro services. Eg: WhatsApp, Discord.

@NobbZ, I may be wrong, but I assume that this companies have more than 50 erlang nodes connected. No? Why we should not use erlang message passing for building micro services architectures?

It’s not very clear to me. Can someone resume me, when we SHOULD use erlang message passing / rpc vs other protocols (message broker / http / grpc). I’m only talking about elixir/erlang services talking to each other’s. I agree that if you want to link services built in different programming languages, it’s more convenient to use the « other protocols ».

@Ciboulette the way I see it, you would want to use distributed elixir when you are building more of a single distributed system where alot of the components will be interacting with each other. Essentially when you connect all the nodes up, it becomes one big system/application which has acess to all other parts of the system (correct me if I am wrong here though).

Whereas on the other hand, if you are building a set of small, de-coupled micro services that don’t really need to be apart of a distributed system, then using something like a message broker might be the best bet.

I think personally I would typically use the distributed section if I had a single application that I need to be HA, (e.g. deployed over multiple servers) but also wanted it to be able to communicate with all other running versions of it’s self (for things like interval jobs for example) and then use a message broker to communicate with other services/applications in the wider network. So basically, a combination of both if that makes sense. However there are definitely still cases where it might make sense to have two seperate applications connected via distributed erlang/elixir but I think it’s more of a problem specific question in that case

I think this is exactly right. If you’re building a stateful service as part of a larger micro-service setup than you might use distributed erlang for the internal communication of that stateful service and RPC or some other interface for the communication with the outside world.