Processes+messages VS Functional APIs - are they different?

Chat / Discussions
, , ,
1

Background

Some time ago I read a huge post about pragmatic Dave and his components idea (along side with a healthy dose of his videos). His idea was that we should code a lot of small applications that communicate together instead of one big app.

Having worked in a lot of monolithic projects (hello C#) this felt like an really good idea to me. Dave went ahead and even made a Components library:

Erlangers

The problem here, if you see the library he made, is that it uses processes for everything.

This is great! I thought. I remember from watching Joe Armstrong that the general idea in Erlang is that processes solve all your problems. If they don’t solve your problems, then more processes solve them. I have even seen old school erlangers advocate that people are usually afraid of harnessing the true power of erlang and that they generally don’t use enough processes.

Of course processes are not good enough, not alone. You also need well defined Protocols to go along. But the main point here is that an architecture based in Processes and Protocols is the preferred way to go with erlang.

Elixirers

But hold on!
If you read the first few threads and responses, you will see that the Elixir community overall prefers a functional approach with functional APIs rather than the processes oriented approach. Proeminent people such as @sasajuric warn about the dangers of using processes and recommend a more pure and functional approach to problem solving, via libraries without state and process trees and well defined APIs.

Basically, if you don’t really need a process, don’t use one.

What do I want?

I can see the value in both approaches. I come from a somewhat functional background but I thought the way to go with Elixir was to embrace the Actor model with all it’s distributed problems, like erlangers do.

So I really don’t understand all the friction with Dave’s idea - it seems rather natural if you come from erlang, right? Or perhaps I am missing an important detail from the discussion?

What is the preferred paradigm in Elixir? Is it the same as in Erlang? Are both communities that different at all?

7 Likes
2

It feels like you’re takeing the advice @sasajuric has a little to far into a black-or-white area of thinking. From my experience people frown upon using processes for code organisation purposes or data hiding. What processes are advocated for is rather for runtime behavior like isolating errors and organising parts of your system into supervision trees, which allow you to define which parts of your applications can fail independently and which parts need to fail together (e.g. cleanup is needed). Another runtime behavior a process gives you is serialization of requests.

9 Likes
3

I think the assumption that processes+protocols are preferred in Erlang is flawed. If processes and protocols were the way to go in Erlang, we wouldn’t have a gen_server (which is a functional API) as the default way to build processes in Erlang/OTP and used by the majority of the Erlang community.

I believe Joe would like to move more towards processes and protocols and I personally would like to see it explored but we do need some additions in order to make it more productive (and Joe explores some of them in his thesis).

Dave’s approach is functional. A lot of the discussion was when your functional API should be backed by processes or not.

I think both communities would generally agree to not create processes when they’re not necessary. Dave even says so in the readme of his component library:

If you don’t share your state with anybody then good news, you don’t need processes and you don’t need this library (for now). You will live a happier life than the rest of us.

Similarly, Joe is not arguing to use processes for everything. He is arguing to not hide our processes behind GenServer, components, etc. Perhaps they both want to achieve the same goal, which is to streamline the development experience related to processes, but they are approaching the problem in two different ways.

PS: I renamed the title because this is not a discussion about Erlangers vs Elixirists.

11 Likes
4

I’m sure he’s going to chime in when he has some time but for the time being I’ll just express my impression as a reader of the book.

There is an expectation that a reader of Elixir in Action may have some OO background and the essence of the message that I received is:

  • Choosing boundaries for processes is different from choosing boundaries for objects/classes.

In particular:

  • Processes are units of computation, messages implementing protocols are coordination - i.e. there is a very clean separation between the computational model and coordination model.

  • OO designed according to CRC leads to very different boundaries for Objects/Classes primarily motivated by encapsulation and while collaboration can be seen as a form of coordination, collaboration tends to happen primarily in the computational domain.

While both have state, processes and objects (classes) are even more different than apples and oranges.

Aside: Process and Stateness are they object? - #8 by peerreynders

10 Likes
5

This is an extremely good summary of what I’ve been trying to say in other threads and in my To spawn or not to spawn post!

We have two paradigms in Elixir: functional and concurrent. These two paradigms don’t compete with each other. They serve different purposes. Therefore, the reasons for splitting by functions and modules are different from the reasons for splitting by processes. Sometimes a single function might encompass multiple processes (e.g. if you’re using Task API with anonymous functions), other times the logic of a process might be spread across multiple modules (which is exactly what happens in my blackjack example).

Embracing processes is certainly the way to go. I frequently say in my talks that there’s a lot of potential for concurrency even in the smaller systems. In the second part of my To spawn or not to spawn article I use processes extensively. The same holds for the examples in Elixir in Action. Heck, the main focus of my book is about processes.

That said, I don’t advise using processes without tangible benefits, most notably the separation of latencies and separation of failures. If those benefits are not gained, I advise using a single process. For organizing the code I advise using modules and functions.

I believe that the preferred approaches are exactly the same for both languages. Most of the Erlang and Elixir code I’ve seen in the wild follows these principles, which have not been invented by the Elixir community. They have been around probably since Erlang exists.

I personally didn’t discuss the entire Dave’s concept. I only disagreed (and I still do) with the notion that encapsulation is enough of a reason to do a process split.

5 Likes
6

A bit of a tangent, but are there examples of using dependency injection and contracts/behaviours for the runtime behaviour? I’d like to abstractly define the run-time contract such as which messages need to be serialized, what modules and functions are consumed, and then configure the runtime implementation. I’ve found it’s straightforward enough to put Persistence and Query concerns behind an interface or behaviour, but it’d be interesting to do the same with Runtime concerns.

2 Likes