Structuring an OTP application

Processes can definitely serve a few different purposes when designing an application, and sometimes it’s fun to just play around a bit, trying out a totally different way of doing things. Like implementing an OO approach with processes..

However, for me it tends to come down to a few core points:

Concurrency

This is of course the first thing one tends to learn about the BEAM – concurrency is implemented by processes exchanging messages, after all. There’s no point in launching 2 million processes just because you can, however; try to stay close to the natural concurrency of the problem you’re solving.

If devices, end users, etc are connecting to your application, then that’s usually a good start – each of those probably need at least one process, for as long as they’re interacting. Do you have recurring tasks, cleanup tasks, etc? They probably also need their own processes. But.. each semi-involved task of some kind that’s initiated by a user of the system? It might not be warranted a process of its own; perhaps every user has a single “background worker”, effectively serializing tasks per user in order to democratize resources somewhat, making sure a single user can’t bog down the entire system..

Fault Isolation

This one didn’t really “click” for me until I’d started learning Erlang a long time ago, and after reading a couple of books and trying some examples, decided that I needed to build something “for real”.

As I was toying around with a small online game prototype, juggling things like TCP connections, command processing, long-running environmental effects, room-based communication and navigation etc, I finally realized that processes are a very powerful tool for fault isolation – especially in combination with proper supervision trees.

I was used to C/C++ mostly at the time, where errors usually are a bit of an all or nothing affair; either you anticipate and handle the error where it occurs, or your whole application comes crashing down. Naturally you’ve got ways to carefully navigate around that, but it’s not easy either way.

With the BEAM, instead, we can start to think about what parts of our systems that can safely fail, and how we can best handle that. User input processing and TCP communication? That all happens in user-specific processes, and if something fails, we just need to make sure that we’ve structured our processes so that everything that needs to be stopped / restarted is properly linked.

As a consequence, you can start thinking about separating very simple, reliable core parts of your application into their own supervision tree(s), so that even if everything else comes crashing down, those parts will survive, either to make sure everything else is properly restarted again, or if nothing else, to safely shut down, perhaps persisting critical data etc.

Serialization

Another important concern is if you have some part of you application that can’t handle concurrency at all; this can then be wrapped in a process that manages whatever it needs to do, allowing other processes to send messages at will but always confident that you’re processing them one at a time.

Naturally this can turn into a performance bottleneck, so it requires some careful thinking. If you’re writing to a transaction log, for example, perhaps it’s tolerable to keep the last few transactions in memory, only flushing to disk every now and then, in order to achieve better throughput. Yes, you risk losing some transactions - but depending on what you’re doing that may be perfectly acceptable.

Currently I’m following the “to spawn or not to spawn” approach – use functional and module deal with thought concerns, and processes deal with runtime concerns.

And try to use registry with name or id, instead of direct linking or storing pids. It’s pretty similar to the service discovery pattern over configuring ip addresses.

I believe it’s the safest and cleanest way of doing things currently. If you are trying to play your business on BEAM, this is absolutely recommended approach.

But from time to time I’m also wondering use wrap processes to an object-oriented language. Though it’s not applausable across the community, it’s might suitable to make something interesting – for example an entity system with runtime concerns. Maybe like when you define an object use the wrapper, you’ll get: supervision tree, default recover behaviour, type, struct, methods, and possibly graphQL type resolver for free. Or you can write custom view or read model on this entity. And it’s might quite suitable for things like fast prototyping.

Thank you. This is the excellent!

Employees, timecard events, schedules, vacations, and overtime may be important concepts for structuring data and possibly even parts of application state but really don’t inform much in terms application behaviour.

Alan Kay (1998)

I’m sorry that I long ago coined the term “objects” for this topic because it gets many people to focus on the lesser idea. The big idea is “messaging” …

This comment highlights how people like to focus in on (static) “objects” because that is comparatively easy when in fact the application value is derived primarily by the (dynamic) “collaborations” that implement application behaviour.

Similarly in a BEAM application the ideal process structure is influenced much more heavily by the behaviour the application is meant to exhibit rather than the structure of the data it is managing or transforming.

This Erlang developer puts a different spin on it:
Lambda Days 2015 - Torben Hoffmann - Thinking like an Erlanger

Processes as the building blocks for protocols - so the big idea is designing protocols realized through communicating processes to implement application behaviour.

I think you’d be better off if you tried to solve a problem that actually needs concurrency at it’s core, like writing a reliable message queue consumer or a web crawler.

You might reference the talk so we can be on the same page with you. My guess, is that the application he is talking about is not some CRUD application manipulating and storing business data. More likely it is a soft real-time application, such as may be used in the control plane of a network appliance, or even a manufacturing facility. There is no abstract “best way” to structure such applications - it is all very specific.

I love it. I see the concepts you’re talking about thank you!!

The two major models are to map data to process or task to process.

For example: a chat app would likely have a process per person (data) while a banking app a process per debit/credit transaction (task). This allows you to expand the relevant bottleneck i.e. add more transaction tasks independently.

Processes can be rich or thin. A rich process would accept a wide range of messages that call out to other modules (not necessary other processes). A person process is a good example of this.

Task processes tend to be thin and focus on specific messages and to interact with other processes.

On the larger architectural front, splitting into nodes and applications is key. Fortunately it is easy to begin either top-down or bottom-up when designing the architecture.

My Elixir applications are traditional (if we even have such a thing yet). I was listening to an Erlang programming talking about how he does not use databases and evading is a process (gen_server) to store state — essentially replacing the database.

So to summarize, it is hard to answer this question because you are asking how to structure an application using processes when we are saying that you shouldn’t.

This is what has me highly curious what that looks like translated to Elixir. I think you’re right there is no right answer really.

Thanks for your feedback. Again, I’m not building anything. Just a purely conceptual question that popped onto my head after listening to some Erlang developer talking. He does everything in gen_server processes — I’m curious as what that looks like.