Problems with Elixir Recursions?

Hello Everyone!

Just came across this article:
https://dzone.com/articles/inversion-of-coupling-control-1

It points out that:

  1. Recursion in FP Can Quickly Fill the Thread Stack (which leads to StackOverFlowError)
  2. Recursion Does Not Release References (which leads to OutOfMemoryError)

Would like to hear about the opinions of the experts on Elixir as to how true this is.
Thanks!

The topic you want to look up is Tail Recursion (e.g. https://stackoverflow.com/questions/33923/what-is-tail-recursion).

Elixir supports Tail Call recursion optimization which can prevent the stack overflow problem.

In general, Functional Programming does not work with references in the Object-Oriented sense. You aren’t maintaining pointers to objects that would prevent them from being released so the second issue is less prevalent - though I suppose you could run into such a problem with large binaries and the garbage collector.

The author of that article is more discussing the topic of using Functional Programming techniques in Java. Java does not support tail call optimization on recursive functions (more accurately the JVM does not support it - though languages like Scala and Clojure provide work-arounds) and Java is definitely prolific in its use of object references.

6 Likes

http://erlang.org/pipermail/erlang-questions/2016-October/090663.html

2 Likes

Like other have pointed out it doesn’t have this problem because of TCO tail call optimization.

The problem the article is criticizing is when a programming language is originally not functional and decides to add functional aspect to the language. IIRC examples is Scala and Python. You can do recursion style on those language but they are not TCO.

6 Likes

First of all I’m no expert.

I just calculated the 2_000_000th term of a fibonacci series(just for the sake of it) and the erl process didn’t go beyond 30% cpu and 27mb ram with an i5 7400 and 8gb ram. It took about 13ish seconds to run though.

defmodule Fib do
  def fib(a, _, 0 ) do a end
  def fib(a, b, n) do fib(b, a+b, n-1) end
end

# iex> Fibo.fib(1, 1, 2_000_000)

But functional languages are optimized for this, just like imperative languages are optimized for their paradigm.

I think it’s good that he points that it’s not a good idea to use a screwdriver to hit a nail, but I also think that blaming the screwdriver for not being a hammer is not very clever.

He then proceeds in the comments section to say that FP is good mostly for financial systems because of… reasons?

In my opinion, the article is completely biased.

3 Likes

In case someone googling in the future finds this thread, I want to specifically set the record straight regarding the main “conclusions” of the article:

  1. Recursion in Erlang/Elixir will not fill the thread stack. The stack for each process in BEAM is stored in a block of allocated memory that will grow as necessary (actually, at the other end of the heap for the process). In general, the runtime system is careful to avoid recursive calls in C code using the thread stack.

  2. The Erlang compiler will explicitly make sure that the stack does not hold any references to terms that will not be used in the future. The compiler does this by making sure that stack slots for stale references are either reused for new values or explicitly overwritten with an empty list. The compiler also does an optimization called stack trimming, where it tries to reduce the size of the stack frame before doing a recursive call. The result is that when a garbage collection occurs, the stack will only hold references to terms that will be used in the future.

12 Likes

The problem with that post is that it only covers Java and the JVM. Therefore it of course has to fail, as Javas compiler still does no optimisations on any kind of recursion. It follows the function calls literally. In the past 20 years there have been a lot of requests to add such optimisations, but for a unknown reason (to me at least) sun/oracle refuses to add this.

Further the author of the article seems to assume, that the rest of the world works exactly like the Java compiler and the JVM and therefore concludes it is a bad thing. I do second the author in the regard, that the functional style has its problems in Java, but that does not make the functional style a bad thing per se.

We could write similar blog post, taking the immutability of the BEAM and write ugly code which looks a bit like Java-Object-Oriented-Style and benchmark it. Then judge Java-OO-ish coding based on this code, and I think this will not end well for OO… But does this make OO bad? No, it only tells us, that the BEAM is not well suited for Java-OO…

And still OO has its value. For some at least. Personally I try to avoid it though, as pretty much too often some global state is touched without beeing properly documented. Or even objects passed in as arguments get mutated, while the object the method was called on isn’t…


Disclaimer: OO in this post of mine does always refer to Java-Style-OO, which simply seems to mean that “objects” know their methods and only live to mutate themselfs. It does explicitely not mean the message-passing OO style.

5 Likes

The stated reason is two-fold:

  • It removes stack frames, and thus harms debuggability (I personally think this is a load of bunk, but whatever)
  • It doesn’t work with the security sandbox contexts built in to the JVM.

Of course there are third-party JVM’s that have TCO though, so whatever Oracle says should be taken with a grain of salt as usual.

2 Likes

I heard of the first one in the python context, and even had a blog post in my favs once, by the former python BDFL, but I can not find it anymore.

Can you share some sources about these statements in the Java world? Talks, Blogs, Tickets, something else?

Not off hand (I’m posting from my phone), just remember coming across it quite a lot back at my last job (where I worked with Java excessively), it was in the issue trackers, a document somewhere, etc… etc…

We’re getting off on a tangent, but I also recall from verbal discussions at Java conferences those reasons. Also see https://stackoverflow.com/questions/3616483/why-does-the-jvm-still-not-support-tail-call-optimization

2 Likes