Excuse the longish post - I too watched Dave’s talk and it brings up some important points.

In this posting I want to talk about composabilty.

To me the unit of composabilty should be the process and NOT functions - to be clearer, of course functions should be composable but this problem is nicely solved ( F1 |> F2 |> F3 |> …

I think the gold standard for composabily were unix pipes, and the beautiful way they could be composed in the shell.

a | b | c | d ...

The principle design idea was “the output of my program should be the input of your program”

This allows a b and c to all be written in different languages - but this has a few nasty problems:

1. text flow across the boundaries
   so there is a lot of extra parsing and serialising involved
2. if something in the middle fails (say c) there is no nice way to close the pipeline down

One excellent feature is that (say) b does not know who sends it inputs and does not know to whom the outputs should be sent.

Now consider Erlang - one of the above problems gets solved - text does not flow across the boundaries but Erlang messages. X ! M sends a message, receive M → … end receives a message so no parsing and serialising is involved and it’s very efficient.

Processes do not know where they get messages from (=good) but have to know where they send messages to (=bad).

A better way would be to use ports, call them in1, in2, in3 for inputs and out1, out2, out3 for outputs and control1, control1 for controls

We can now make a component - assume a process x that has an input in1 which doubles its input and sends the result to out1 - this is easy to write in erlang

   loop() ->
      receive 
          {in1, X} ->
              send(out1, 2*X),
              loop()
      end

Clever people can write this in Elixir as well

All the component knows how to do is turn numbers on the in1 port into output on out1 but it does not know where in1 and out1 are.

Now we have to wire things up.

The pipe syntax X | Y | Z means “wire up the output of X to the input of Y” (and so on)

The important point is that a) components do not know where they get their inputs from and do not know where they send their outputs to and b) “wiring up” is NOT a part of the component.

Elixir has a great method for wiring up functions X |> Y |> Z but the X,Y’s and Z’s are functions
NOT processes.

We can imagine components to be processes with inputs (in1, in2, in3, …) outputs (out1, out2, …) control ports (control1, control2, …) and error ports (error1, error2,…) - what are the error ports?

Error ports are for (guess what) errors - sending an atom to the in1 port of my doubling machine would result in an error message being sent to error1 (or something).

All of this can be nicely specified with some type system -

Machine M1 is

 in1 x N::integer -> out1 ! 2*N :: integer

etc.

With this kind of structure software starts looking very much like hardware and we can make nice graphic tools to show how the components are wired up. The reason we do not program like this in sequential languages is because all the components MUST run in parallel (which is what chips do)

There is actually nothing new in the above - these ideas were first written down by John Paul Morrison in the early 1970’s (see Flow-based programming - Wikipedia) –

This (flow based programming) is one of those ideas we could (and should) revisit and cast into a modern form.

All of this means a bit of a re-think since most frameworks are structured on top of essentially sequential platforms.

Really we should be thinking in terms of “black boxes that send and receive messages” and “how to wire up the black boxes” and NOT functions with inputs and outputs, the latter problem is solved.

Think - “messages between components” and “what messages do I need in my protocol” NOT “input and output types” and “what functions and modules do I need”

(I called this Concurrency Oriented Programming a while back - but the term did not seem to latch on

As Alan Kay said “the big idea is messaging”

Cheers

/Joe

Sorry for being pedantic but, since you are asking someone else to build on their arguments, I would like to point out that you have not explained why any of the points above are true. I.e. it is not clear why the approach you mentioned above is easier to test, change, or reason about.

Just to make an absurd point, maybe the contract you define between those components pass all of the arguments between functions via the process dictionary. Maybe you do this:

# in the controller
Process.put(:product, product)
NewProductHandler.call()

# in the handler
def call() do
  product = Process.get(:product)

Now your code is broken between many files but it is still coupled and hard to test!

Every time somebody brings the argument that a module should only have a single function in their domain, I propose this: what if we remove Enum.map/2, Enum.reduce/3, etc and instead we introduce Enum.Mapper.call/2, Enum.Reducer.call/3 and so on?

To clarify: I am not saying breaking your software into multiple files is bad a thing but programmers clearly leverage different mechanisms to group functionality together. We need to identify when those mechanisms are measurably better (i.e. you can prove it leads to less coupling and better tests) and we need to identify when we feel the code is better.

Theoretically speaking, there is no difference between how easy to test or change is a big module MyApp with a 100 functions call_a, call_b, etc compared to 100 modules with a single function MyApp.A.call, MyApp.B.call, etc. There are, however, differences on how a developer understands and communicates with another developer in this codebase, which is what causes some people to feel one approach is superior than the other, but I can’t prove one is better than the other.

It is very important to make a distinction between proofs and feelings because they lead to very different discussions.

I think it is important to clarify that, and I think Dave would agree, is that GenServer is not wrong. Rather we are wrong in using GenServer to build higher level components, often with domain/business logic, and we need higher level abstractions to solve those cases. When building low-level components, the distinction between client, server, and the low-level callbacks are important, but in your application domain you want to get rid of the bureaucracy and put the focus on the domain code. There are also other libraries attempting to tackle this particular issue.

I agree with Dave in many of the problems he raised: yes, we need to improve configuration (there was a whole other thread about it in the forum). Yes, it would be helpful to know if an application is stateful or is a singleton (and as I said in this thread, I don’t think looking at the supervision tree gives the complete answer). As Dave said, there is a lot to explore.