There are a couple of facets to the let-it-crash story.

At the core of it all is the idea of failing fast. This is not something exclusive to Erlang, and I believe it’s generally a good practice. We want to fail as soon as something is off. By doing this, we ensure that the symptom and the cause are one and the same, which simplifies the problem analysis. By looking at the error log, we can tell both, what went wrong, as well as why.

Now, of course, we don’t want our whole system to crash due to a single error, so we need to isolate a failure of a single task. In many of popular languages, this is done by wrapping the task execution in some sort of a catch-all statement, or by running the task in a separate OS process. So for example, as someone mentioned here, a typical web framework will indeed to this to make sure that the error is caught and reported properly.

However try-catch is not a perfect solution due to a couple of things. First, if shared mutable data is used, a task which fails in the middle could have left the data in an inconsistent state, which means that subsequent tasks might trip over.

Moreover, a task itself could spawn additional concurrent subtasks (threads or lightweight threads), and we need to make sure that the failures of these threads are properly caught. A great example of this is go language. If a web req handler spawns another goroutine, and there’s an undeferred panic (aka uncaught exception) in that gouroutine, the entire system crashes.

In contrast, using separate OS processes helps with this, but you can’t really run one OS process per each task (e.g. a request), so we usually group them somehow (which to me is what microservices are about). Now, you need to run multiple OS processes, and you need an extra piece of tech (e.g. systemd) to start these things in a proper order, restart failing OS processes, and maybe take down related OS processes as well.

With BEAM, all of these issues (and some others) are taken care of directly in our primary tech. If you don’t want a failure of one task to crash other tasks, you’ll typically run the task in a separate process, and fail fast there. With errors being isolated, a failing process doesn’t take down anything else with it (unless you ask for it explicitly via links). Shared-nothing concurrency also ensures that a failing thing can’t leave any junk data behind. Moreover, it ensures that whatever crashes, the associated resources (memory, open sockets or file handles) are properly released. Finally, a termination of a process is a detectable event, which allows other processes (e.g. supervisors) to take some corrective measures and help the system heal itself.

As a result, Erlang-style fault-tolerance is IMO a one-size-fits-all. We use the same approach to improve the fault-tolerance of individual small tasks (e.g. request handlers), as well as other background services, or larger parts of our system. I like to think that supervision tree is our service manager (like systemd, upstart, or Windows service manager). It give us same capabilities and same guarantees, it’s highly concurrent, and it’s built into our main language of choice.

In contrast, in most other technologies, you need to use a combination of try/catch together with microservices backed by an external service manager, and in some cases you might need to resort to your own homegrown patterns (e.g. if you need to propagate a failure of one small activity across microservice boundaries). Therefore, I consider these other solutions to be both more complex and less reliable than the Erlang approach.

HTH

Evaluating “let it crash” in a situation that is largely stateless to begin with (http requests) misses somewhat of the origin story this philosophy.

A good place to start would be to read The Zen of Erlang since anything most of us are going to say by way of an introduction to the topic will largely amount to a rephrasing of that post.

Notably, even in your stateless http requests there is a major difference: In Java, a crash due to your request will take out OTHER people’s requests too, since any single Java thread is handling many different requests concurrently, and if it dies due to your error everyone else will also go down. In Elixir / Erlang, crashing your process leaves everyone else’s utterly unaffected.

This makes crashing in Java expensive. It’s expensive in terms of performance because threads are heavy weight, and it’s expensive in terms of the cost to your users, since crashing is not isolated merely to the user who’s state caused the crash. Consequently, design patterns that utilize crashing as a way to ensure state guarantees are not readily available to Java. For more on those design patterns, I recommend the aforementioned link.

To point a few differences:

• “Let it crash” helps centralize the error handling in a supervisor process, rather than have it spread around your business logic. This diagram in An Open Letter to the Erlang Beginner (or Onlooker) represents this:

error-path.png497×681 35.2 KB

• All resources are owned by a process in Erlang, and the VM guarantees clean-up of resources once the process dies. In Java, Python, etc., if your response handler opens a file then throws a NullPointer at some random location later, the framework’s exception handler will catch it and the application will seem OK… until it runs out of file handles after this happens too many times. I’ve seen a Twisted app lock up hard because of this. So you have to be more careful with your error handling, and spread try/catch over the business logic.

• This extends to other types of resources, for instance registered names, DB connections, or locks of all kinds. In most languages, if you hit a problem where threads are grabbing DB connections then throwing an exception without releasing the connection, your app will quickly run out of connections and die. In Erlang the connection (a process) can keep a monitor on the owner (another process), and release itself as soon as the owner dies, no matter the reason. So the robustness of your application depends on the correctness of the DB connection’s code, which is rather small, rather than depending on the correctness of every single request handler that uses a DB connection. In the Erlang world this is called the error kernel, and you want to keep it as small as possible.