All of the processes which are started directly from the supervisor should be OTP compliant (aka special processes). This will allow them to work properly within the supervision tree, and to play nice with tools/modules such as observer, sys, dbg, …

Abstractions such as GenServer, Supervisor, but also Agent and Task are already OTP compliant. If none of them suit your needs, you could implement your own OTP compliant behaviour. Usually, it’s easiest to do this on top of an existing OTP compliant behaviour. For example, Supervisor, Agent, and gen_fsm are internally powered by GenServer (or gen_server).

If none of the existing behaviours serve as a good baseline, then you have to start from scratch and support all OTP requirements as explained in the link above. The core library by @fishcakez could simplify the task.

One advantage of rolling an OTP compliant process from scratch is that you can do selective receives. In other words, you can use pattern matching in receive to give higher priority to some types of messages. This is something that doesn’t work with GenServer.

To be honest, I never used this technique myself. In the singe case where I had this need, I just split GenServer in two processes. One process received messages from clients, and acted as a priority queue. Another process was the consumer that handled messages. The consumer would ask for the next message from a queue, and then handle the message. Meanwhile the queue can accept subsequent messages and rearrange them by priority. When the consumer is done with the current message, it asks the queue for the next one, and it gets the one with the highest priority. I guess this approach can have perf/latency issues with a large rate of incoming messages, and then perhaps a manual loop with a selective receive might help. But as I said, even with this approach, it’s best to make such process OTP compliant.

Cowboy, the erlang web server that powers Phoenix, uses special processes in several places. Loïc has some excellent content covering the why and how on his website:

I feel the answer is probably, “not too often.” Maybe if you are building trivial tools it’s fine, but for most serious projects I believe you want to be favoring OTP construction.

spawn(), send(), and receive() are basic building blocks. OTP uses them under the hood and it’s useful to learn a bit about them to help you understand the higher layers.

However, you probably don’t often fire up a telnet client to check your email. You could, assuming you know the protocol commands to send, but doing so is harder and more error prone.