What is easier to scale, Go with Docker and Kubernetes or Erlang / Elixir + OTP?

erlang
otp
go
#21

When presenting Elixir to some devs, they already replied to me with something around this lines, and the last one I remember said that AWS auto scaling already gives them the the distributed and fault tolerance I advertised in the BEAM.

So as you said devs that do not know the BEAM will not see the value of it, once the current cloud offer already gives them the “distributed” and “fault tolerant”/“high available” bits for their application to be resilient and high available.

2 Likes

#22

I can understand your skepticism and I would not venture myself into discussion over communication practices between pods as I don’t know enough about it yet.

But something I do know is that the growth of IoT is such that a large share of the workload will have to be delegated outside the CSPs’ infrastructures, on edge devices. And once again, the industry actors that are not willing to go through this paradigm shift will not be able to sustain future requirements.

And in the context of billions of small user devices, inter-process communication opens the path towards M2M collaboration and this is why I would be very interested in knowing what specifically you would consider as a
architectural flaw in that case?

Best,

Igor

1 Like

#23

(1) Distributed Erlang was primarily intended for physically colocated hardware that was networked in a “secure” manner (e.g. isolated from outside interference; think each node running on a separate board communicating through a common backplane).

(2) The following is a mere rationalization of this experience.

  • The cumulative code deployed to a single node aggregates to a single system. That system could simply be a sub-system to a larger whole but running on a single Erlang runtime acts as a single autonomous unit no matter how many processes exist on the inside.

  • For the purposes of fault tolerance (provided (1) is satisfied) distributed Erlang (i.e. process communication between nodes) makes sense because the systems are identical. There is no issue with implementation coupling because the systems are identical. The nature of the communication traffic is closer to intra-system rather than inter-system communication.

  • If two systems operating on two separate nodes implement different capabilities then more mainstream communication protocols should be preferred. Using distributed Erlang leads to implementation coupling because now all the collaborating systems have to be either implemented as BEAM systems or at least have to communicate through proxy technologies such as C Nodes, jinterface, or Pyrlang. Also more mainstream protocols may make it easier to use more common network monitoring (or perhaps even governance) tools.

Another consideration is the protocol design between systems. Ideally messages between systems should have a coarser granularity and occur much less frequently than messages inside the system. From that point of view it makes sense to move inter-system communication to an altogether different network protocol rather than using inter-process messaging. In the IoT space MQTT seems to be the preferred protocol (Visualizing Home Automation with GRiSP).

I would imagine that in the IoT space Erlang/Elixir isn’t that impacted by the Docker/K8s prejudice. However other languages and runtimes are available in that domain. The issue is that often tools that optimize for a low barrier of entry and fast first iteration delivery and deployment are adopted over tools that strive for all around (long-term) quality. I’ve already mentioned PHP and I think Python for Machine Learning fits there as well. I suspect that Python and JavaScript are popular with the “easy to get started” IoT crowd - despite of the technical short comings that may impose on the resulting solutions.

3 Likes

#24

It is one of the biggest reason why I try to avoid all the Go I can. Underwhelming error handling with total lack of detectability of failed or hung goroutines.

I might not be missed when you compare with other languages that don’t even have such features, but it’s such a glaring omission for an Erlang old timer that it’s like (if I can be allowed the flippant comparison) looking at a car with no brakes or seatbelts while the driver tells you “if you’re just careful and time manual engine compression right, you don’t actually need any of that”

I understand what you mean by “not missed because you don’t know about them”, but frankly it still boggles my mind that nobody actually pines for them in these communities. The debugging stories I’ve heard and seen are just maddening.

9 Likes

#25

Totally true statement–ask my neighbor about a truck ride through the mountains with no brakes-haha!

2 Likes