OTP is Bad

Flamebait-y title, but Joe Armstrong doesn’t use OTP, so I am not in uncharted territory.

But… I have been thinking (dangerous).

I read a blog post a while ago, by a Lisp guy, who said that programming patterns are bad, because they are used to patch language weaknesses.

I imagine, in Lisp-world, patterns are packaged into nice little macros and all is good in the world because Lisp is flexible like that.

Perhaps too, this is where Elixir could really take a leap beyond Erlang.

By packaging OTP behaviours into macros.

Then again, perhaps the analogy is completely wrong, because behaviours aren’t really design patterns in the first place (although, I feel they aren’t far away).

Nevertheless, if I am making a little sense, I would love to hear some more experienced people’s views.

Distributed programming is very much a design pattern. I hate design patterns as much as the next rational person, but you can’t effectively assemble a distributed system without it. Even throwing a bunch of stuff behind ZeroMQ or load balancers is “a pattern”.

People don’t need to seek out OTP to use it “because OTP” but Elixir/Erlang’s design decisions facilitate everything needed to make OTP work so well for it’s purposes. You’re welcome to own a sports car with access to the Autobahn and not use it…but why? :slight_smile:

3 Likes

I would be interested in where seeing what Joe has said about not using OTP.

I think that OTP is “bad” in a number of ways. (bad in quotes because obviously a lot has been done with it)

  1. Using callbacks eliminates the value of the process mailbox. you cannot queue messages in a GenServer because of the handle_info callback.
  2. it’s difficult to have two types of messages to be going to one server, without knowing the message structure. again if a library is not using gen call then you are back to using the handle_info callback.

Elixir macros do allow some help to this because you can write macros and use them on the left side of pattern matches. e.g. the following psudo code that could be possible

receive do
  Gen.call_received(from, message) ->
    Gen.reply(from, "thanks")
  TCP.packet_received(socket, packet) ->
    TCP.send(socket, "Gottit")
end
1 Like

I’d say any non critical belief that something is good or bad is a bad thing, that might be the evil behind some design patterns but it’s “not their fault” so to say :slight_smile:

Without (a little bent) MVC and OTP we’d have problems maintaining our Elixir apps, YAGNI and KISS help us think about problems (or maybe they help us think less), so why are they bad? I like them!

That being said I find myself mostly relying on the “nested pipelines” pattern when writing elixir apps, that means I mostly think in terms of input, output and small functions in between, but that’s in part because these big fat patterns took care of other concerns.

1 Like

I had a very interesting conversation with a fellow developer a while back. He has a background in building engineering, and told me the following story:

  1. Architect designs a building.
  2. The building engineer tells the architect “you really need to make this column three times as wide, or otherwise the roof will collapse”.
  3. The architect says: “Aww, but it ruins my Feng-Shui”.
  4. The final building is constructed from the compromises that the architect and the building engineer agreed on.
  5. People believe that the whole building was built single-handedly by the architect.

He said that what people in Computing Science call ‘design patterns’ (probably since the famous ‘design patterns’ book written by the ‘gang of four’) are actually ‘structural patterns’: “build the column this way or the roof caves in.”.

As such, they have nothing to do with beauty, but everything with practicality (which, in computing science, translates to speed/memory efficiency, ease-of-understanding, ease of refactoring).
Many programmers boast that “they solved problem A by using patterns X, Y and Z”, but that is useless. Patterns are something to recognize and use when it is useful to do so, not something to boast over or to cram as many as possible in your application of.

In the end, the main use of these named patterns is that two developers can have a conversation about a possible solution for a problem, and easily describe to each other how these work in general terms, just like two architectural engineers might converse and say: “to prevent this wall from collapsing, we could use a buttress.”

Tl;Dr: do not use patterns to solve a problem. do solve a problem by using patterns.
Patterns are good nor bad, but programming is about solving problems, not about cramming as many patterns as possible in your codebase.

As for the ‘do patterns exist in functional programming?’-debate: Yes, they do! (To be exact, structural patterns do). Some common ones include recursion, map/reduce, higher-order functions and even the illustrious monads. However, these patterns have a much lower impact on how everything around them is designed than many of the OOP-patterns are, and all of these patterns more-or-less arise naturally (i.e. even when you do not know about them, you might already have ‘reverse-engineered’ them).

The fact that many people do not recognize these as patterns might indeed mean that it is so natural to use them in your language, that indeed they can be considered language features.


As for OTP: Message-passing is a complex thing. OTP packages this in the best way Erlang could. I completely agree that this does not feel like the most idiomatic way in all circumstances.

There do exist Elixir libraries that wrap GenServer & friends using macros to make it easier to use, such as ExActor. This works fine for 95% of the cases. For the other cases (more logic in the outward-facing function before calling handle_*, or needing to keep the dependencies of your library as small as possible) simple OTP is fine.

I would say that behaviours are something completely different from structural patterns: Behaviours make different parts of your code agree in what way something can communicate. This in no way restricts what else is going on in those parts of your code.

Finally though, recognizing patterns is actually important. This amazing TED-talk explains why better than I could :slight_smile: :

18 Likes

I find myself mostly relying on the “nested pipelines” pattern

In some places, nested pipelines is called the ‘builder pattern’.

Both try to achieve the same thing.

One is instantly understood when you lay eyes on it, the other is found in the pages of design pattern books…

Perhaps same thing goes for OTP and the mythical language semantics which could replace it : )

2 Likes

Actually, I’d dare say that the ‘builder pattern’ is the poor man’s ‘partial function application’ (to be precise, the definition of the builder is a way to introduce ‘currying’, and the locations where the builder are used are doing ‘partial application’), while Elixir’s pipelines are a more natural way to write to function composition (which, itself, of course, is a more specialized form of function application).

5 Likes

By packaging OTP behaviours into macros.

What does this mean? We already deal with some OTP boilerplate with macros. Macros are a compile time construct, OTP is run time process patterns.

3 Likes

OTP is concurrency model (actor model). Each language has some concurrent models.
https://pragprog.com/book/pb7con/seven-concurrency-models-in-seven-weeks

1 Like

I do think the analogy is wrong, here. I don’t think OTP tools constitute programming patterns at all–they’re a fault-tolerant framework that builds on the BEAM VM’s concurrency primitives and other language features.

The OTP tools, in their implementation, may employ different programming patterns, and as a framework they limit the patterns you can use with them because they decide how you interface with them. OTP Behaviours are just one pattern to implement the definition of an interface, you can do this different ways (as Protocol does). You can re-implement OTP tools using those primitives, a custom interfacing mechanism, and different programming patterns if you want to get different results, if you want.

There is a “pattern” for this–it’s called syntactic sugar. The OTP framework is bundled with the language, so one could conceivably extend the language with custom syntax for hooking into OTP, or even your own custom concurrency tools. There are three ways in Elixir that you could create elegant hooks into them:

  • Parser level: customize the tokenizer and parser with new syntax and define new semantics for them, possibly by extending the list of language operators
  • Compiler level: create inlined functions and macros that do what you want, perhaps around overriden or custom operators
  • Runtime level: just use normal functions and possibly operators like this guide demonstrates

You’ll notice Elixir does do this already! In particular, only through the latter two mechanisms, without any custom operators. I think is a wise choice because it keeps parsing separate from syntax and semantics, and leaves the operator field wide open for custom use-cases, like your envisioned concurrency toolbelt.

  • Behaviour, Supervisors, GenServer, and GenEvent are macro-wrappers around existing OTP tools (you don’t have to use them to define OTP-compatible modules, and could write your own if you wanted!)
  • Tasks and Agents are higher-level abstractions around existing OTP tools
  • GenStage and Registry are completely new, non-OTP concurrency tools responding to some pain points in GenEvent and gproc, respectively

With that background, to respond to your post point-by-point…

That’s because prefers to reinvent his own concurrency tools and behaviours from language primitives, or just use the primitives. If he packaged and published his tooling, you’d have just another OTP-like toolbelt in the ecosystem, which would be great. But the core appeal of OTP is that it’s done and battle-tested for you.

I think some tools are built to assuage language weaknesses. But some tools, like OTP, are built to make using the language’s strengths easier. I also think Lisp’s core language weakness is that it doesn’t have many standard tools built in, so its core language strength (metaprogramming) is used heavily to compensate. “programming patterns are bad” seems like a silly way to defend a core language-tradeoff that LISP made.

That would make sense if we wanted Elixir to be a LISP-world. Instead, Elixir’s made some trade-offs in it’s core library (like, for instance, having one) so that developers can be more productive out of the box, without navigating a package manager or auditing third-party code to decide how to do Enumerables, for example. Elixir inherits the OTP tools as well as adding its own.

Where it takes a leap beyond Erlang in it’s design, is that it’s hackable with macros to let you do anything you want to, within its parser and lexer limitations, if you’re willing to write the macros. This is how the pipeline operator, conditionals, and tools like with work–they’re totally new syntax and semantics, fitting within the lexer. You could build your own macro-powered concurrency tools that feel as native as those macros.

As mentioned, Elixir has already has done this, although purely on a semantic level. However, nothing’s stopping you from a very interesting investigation into pairing your own concurrency tools, or pairing existing OTP semantics, with custom syntax. Just learn your way around macros and color in-between the lines that the lexer gives you. If you have some ideas in this front, I’d love to see your experimentation!

5 Likes

Thank god someone said it.

Patterns being a bad thing is something that seems to be perpetuated by people who got burned because they took the idea of design patterns and “best practices” to mean “use all the time”, and then get mad when their square peg doesn’t fit in their circular hole, and they had to do frankenstein work to make it fit.

Formal design patterns are just a way to communicate different architectural approaches to solving a specific problem, not even to architect a whole application.

7 Likes

:grinning:

6 Likes

Nothing is perfect unless its perfect for the job ahead
@Qqwy has the best analogy I have ever heard on the subject.

patterns aren`t generally bad. they are pointers to how to get a specific thing done. if something doesnt smell right using a pattern then no problem. We are programmers we of all people should be able to think for our selves as problem solving is 90% of our work.

There is shit that needs to be done and there are a number of ways to shovel it. Don`t be a tool and use the hammer ! And a shovel is good enough for small projects

ps : I am not saying the hammer is bad its awesome for dealing with nails in wooden planks, Im just saying Id prefer a shovel.

3 Likes

Thanks for the enlightenment guys - lots of nice nuggets.

I reckon there’s a lot of scope for Elixir to use macros to simplify concepts or perhaps more importantly use them to enforce things that could otherwise trip people up.

There are quite a few Erlang OTP books on my bookshelf. Distributed computing is tough - but isn’t Elixir in the game of bringing OTP goodness to the masses? : )

I think diving into Lisp macros more and perhaps experimenting with those ideas in Elixir could be fruitful…

This strongly reminds me of philosophy (because it is. Really!). It’s common for someone to say “I had a thought about the way the Universe works, but then found out someone else already had that theory”. I’ve done it. Surely, everyone has? With patterns and programming, someone with little skill may tackle a problem that they see with an idea that turns out to be a pattern. They might not know this at the time, but there it is. Then, at the other end of the scale, we have the philosopher, who is well read and educated in theories. The pattern expert, who knows many patterns theorised by many computer scientists.

The point is, a pattern is just an idea that someone noted on paper for solving a given task. The philosophical theory that suggests a reality. Fact is, other people have probably had the same idea prior and further people will have the same idea without realising it was already a known thing. However, it fits the requirement and it allows the process to continue, so it is used.

Ultimately, we should not wonder whether patterns are good or bad; only, do they solve the problem at hand? If we know the pattern before hand or we devise the pattern for a new problem, it makes little difference, since any solution to any problem can be identified as a pattern, and is guaranteed that status if we tackle a future yet similar problem with the same approach.

2 Likes

I don’t know if Joe Armstrong doesn’t use OTP or not. But his focus is most likely different than yours. He’s interested in language development and providing useful primitives to build stuff like OTP. You are, most likely, interested in building applications that use appropriate abstractions.

I remember back in the day, when I worked in early Rails or PHP projects, I inherited a e-commerce solution from other programmer. I was chosen because I speak Polish, and code comments were in Polish. What no one told me, was that the programmer did write whole e-commerce store in single PHP file, with no database (but files as storage) and there was no single function or object definition inside. It was large if-clause, wrapped in case, wrapped in if, and then used bunch of goto labels to redirect the flow.

I met the guy at some point, told him that his code is crazy and we needed to write it from scratch, because no one could get their heads around his “solution”. Why would you do something like that? - I asked. “It’s because I am really assembly programmer, and that’s what I do best, and I don’t really believe functions, objects or database have any significant value for me”.

So here you go. If you want to write code that will make you proud, take the bare bones and build your own abstractions on top of that, that suit your taste. If you want your code to be understood by other programmers - use the patterns and libraries that they are familiar with.

6 Likes

Just to reiterate from above, as there are a lot of posts --> use exactor.

If you look at Sasa’s code you will notice a lot of macros just like “Lisp-world”.

If you are new to OTP and the syntax and conventions are a bit overwhelming this can really clean things up. If you have used OTP enough, you may just dismiss this lib as OTP may then at that point feel simple to you.

Language / framework designers always have to decide on how to balance the needs of people starting out and people that are very well experienced. Some of Sasa’s libraries fills some gaps. The Core team is actively working on parts of this…

For example look at the proposal around Experimental.DynamicSupervisor

We have a couple goals by introducing a dynamic supervisor:

- Simplify the API and usage of both Supervisor modules. Most of the documentation     in the Supervisor module is full of conditionals: "if the supervisor type is :simple_one_for_one, it will behave as X, otheriwse as Y." The differences in behaviour with little surrounding context makes supervisors hard to learn, understand and use;
- Provide out-of-the-box supervisor sharding for cases where the supervisor itself may be a scalability concern;
- Provide a built-in registry to avoid developers unecessarily using dependencies like gproc or incorrect dependencies like global;
- Implement the GenStage specification so dynamic supervisors can subscribe to producers and spawn children dynamically based on demand;

While you are at it, take a look at the fsm library as well. Gave me a great option when I did not want to use gen_fsm but instead being able to use an FSM w/o having to create a process. Apparently there is a new module gen_statem as well.

Also con_cache is cool to though I haven’t had a chance to play with it.

Part of learning Elixir is learning the functional programming building blocks, map, reduce, filter, take, head, tail, and if you really want to “learn Elixir” to do fault-tolerant, distributed work learning the OTP building blocks and the new Elixir Experimental backend libs :wink:

So yes indeed, OTP is waay baad a***

2 Likes

I know most people on this forum are fairly nuanced towards OO programming, but I submit that encapsulation in the sense of combining data and methods was flat out a terrible design decision. Ironically, it created brittle and tightly coupled code. As experienced OO programmers repeatedly banged their head against the consequences of this, they created the famous Design Patterns for minimizing the harm of encapsulation: inversion of control, dependency injection, fascades etc., Most books on OO programming style seem geared towards getting this water back in the dam.

Design Patterns became the touchstone of good programming. “Oh you’re a programmer? Have you read Design Patterns?” But as functional programming grew more popular and the limitations of OO programming became more apparent, that started to reverse. Functional programmers began to sneer at patterns as lipstick on a pig. Now patterns are bad. See also: the lisp quote referenced at the start of the thread.

It’s ironic because SICP, which is how most LISP programmers learned their trade, is all about patterns. It’s most the pattern-centric book on programming I’ve ever seen. And if you are going to create objects, then GoF patterns are also really good. OO + GoF >> OO. Patterns are good.

3 Likes

Exactly that. Interestingly, it is not what encapsulation was meant to mean. Thank you, you enabled me to finally find a way to wirte my rant on RPC vs Message Passing in a way that make sense !

1 Like

Patterns are good.

Yes… where needed. Not as a checklist (“oops, forgot to use Abstract Factory, now where can I shoehorn that in?”), not where inappropriate (“hey, let’s make each instance of this class its own Singleton!”), etc… but mainly to make up for deficiencies in the language. Hence the confusion of, for instance, Java programmers migrating to Ruby:

  • “So how do you do Dependency Injection?”
  • “You just… inject your dependencies.”
  • “Uh, right, but, ummm… what frameworks do you use for that?”
  • “Why would we need a framework to do something so trivial?”

Or from any typical OO language to any typical FP language… finding so many of their classical patterns Just Don’t Apply.

1 Like