When to use OTP vs 'bare' processes?

The more I read about Elixir’s and Erlang’s actor-model-based functionality, the more I am in love.

One thing I am wondering about, is when it is better to drop down to a ‘bare’ recieve-loop, over spawning something as a GenServer?

GenServer, Agent and friends do a lot of the behind-the-scenes work for you, but are there any drawbacks to using them that would make it better to use bare processes in some cases?

4 Likes

I feel the answer is probably, “not too often.” Maybe if you are building trivial tools it’s fine, but for most serious projects I believe you want to be favoring OTP construction.

spawn(), send(), and receive() are basic building blocks. OTP uses them under the hood and it’s useful to learn a bit about them to help you understand the higher layers.

However, you probably don’t often fire up a telnet client to check your email. You could, assuming you know the protocol commands to send, but doing so is harder and more error prone.

6 Likes

Well, I don’t know if this is the best approach, but when I have a simple process that just need to be running and doing some work from time to time, with zero incoming communication, I usually create a “bare” recursive process or a Task and add it to my supervision tree.

Of course that doesn’t qualify as “not using OTP”, as I still use it’s supervision tree in all it’s glory. I just think sometimes the GenServer is a overkill.

Although, if you need incoming communication, then I guess a GenServer or an Agent or any other abstraction the language provides are the way to go.

2 Likes

I do think there are scenarios where you should consider Elixir’s simplifications, like Task (especially with the help of Task.Supervisor) and, to a lesser extent, Agent. Honestly though, these are an even higher layer than OTP.

3 Likes

Yeah, I guess my reply would be more suited for a “Whether or not to use GenServers” discussion :sweat_smile:

2 Likes

All of the processes which are started directly from the supervisor should be OTP compliant (aka special processes). This will allow them to work properly within the supervision tree, and to play nice with tools/modules such as observer, sys, dbg, …

Abstractions such as GenServer, Supervisor, but also Agent and Task are already OTP compliant. If none of them suit your needs, you could implement your own OTP compliant behaviour. Usually, it’s easiest to do this on top of an existing OTP compliant behaviour. For example, Supervisor, Agent, and gen_fsm are internally powered by GenServer (or gen_server).

If none of the existing behaviours serve as a good baseline, then you have to start from scratch and support all OTP requirements as explained in the link above. The core library by @fishcakez could simplify the task.

One advantage of rolling an OTP compliant process from scratch is that you can do selective receives. In other words, you can use pattern matching in receive to give higher priority to some types of messages. This is something that doesn’t work with GenServer.

To be honest, I never used this technique myself. In the singe case where I had this need, I just split GenServer in two processes. One process received messages from clients, and acted as a priority queue. Another process was the consumer that handled messages. The consumer would ask for the next message from a queue, and then handle the message. Meanwhile the queue can accept subsequent messages and rearrange them by priority. When the consumer is done with the current message, it asks the queue for the next one, and it gets the one with the highest priority. I guess this approach can have perf/latency issues with a large rate of incoming messages, and then perhaps a manual loop with a selective receive might help. But as I said, even with this approach, it’s best to make such process OTP compliant.

14 Likes

I like to search Github for use cases. In this case, the relevant link is:

Does look like it mostly is used when …learning Elixir. I think non-otp processes and receiving a messages in a loop (or otherwise) directly with receive is sort of primitive construct that is good to know it exists. But once you develop taste for OTP, you’ll stick to that. It’s predictable, well known set of behaviors, and that affects both: your ability to structure your thoughts in code, and to read someone other’s code.

3 Likes

One nice thing about otp genservers (which are reaaly quite simple) is that they are just modules with functions and the message loop is handled outside of the genserver. This make them really easy to unit test.

Once you name your processes and put them in a supevision tree, you will need to use OTP servers to prvent using stale PIDs after one server has crashed and been restarted

1 Like

Having written the core helper library I never really use it. I really just use standard behaviours or build another behaviour on top of those.

4 Likes

Cowboy, the erlang web server that powers Phoenix, uses special processes in several places. Loïc has some excellent content covering the why and how on his website:

http://ninenines.eu/talks/beyond-otp/beyond-otp.html
http://ninenines.eu/articles/erlanger-playbook/

7 Likes

One example might be about a process that sleeps for some time and then wake up(timeout ) and collect data from external interface and send it to some external server and then again go to sleep again.

You can use GenServer & send_after for this, not complex at all, and still allows for other events to be received & handled asynchronously should that ever be needed.

1 Like

Correct me if I’m wrong, but some libraries require creation of non-OTP processes, such as HTTPoison's & HTTPotion’s support for streaming.

I was searching for a way to create an OTP-like process which will function well even if I make a receive loop & I found here @fishcakez’s Core library which he actually never used, but I will :laughing:

EDIT: Unfortunately the library is quite outdated, so I ended up using Task. :confused:

1 Like

I was going through this slide show. Does anyone know if there is a video for this talk? Also do people agree with the usecase that a supervisor + some state management is a good example for a special process.

I’m thinking of using a special process to solve this issue here https://stackoverflow.com/questions/44663817/how-to-reference-previously-started-processes-in-an-elixir-supervisor/44663920

You could also use a GenServer for that.

You could use Supervisor.which_children in the second child to ask the parent about the children, and then find the pid of the sibling you’re interested in.

However, you can’t do this in init/1, since you’ll deadlock, so you need to postpone this for after init. The second child can send itself a message (e.g. :post_init), and then in the corresponding handle_info ask its parent about the children, find the desired sibling, and store the pid in its own state. It’s a bit tricky, but does the job.

Make sure to use rest_for_one or one_for_all supervision strategy, or otherwise you might end up with a dangling reference to a non-existing process.

1 Like

Do all OTP processes have a reference to their parent? if so then that is something I have missed and sounds a helpful solution. Although It still seams messy to have a process know about the structure of a supervision tree it is in. In essence I am having to change the structure of a child after adding it to a supervision tree

There is an undocumented :$ancestors procdict key:

iex>  Process.get(:"$ancestors")
[#PID<0.50.0>]

Whether you want to rely on that is another matter :slight_smile: If you want to be more explicit, you can pass the pid of the parent to the child from supervisor’s init/1.

I agree it’s messy, but I also think it’s fine in those rare cases where siblings are tightly coupled. For other cases, I’d use Registry to discover processes in the system. I talked about this last year at ElixirConf EU. You can find the video here. There was no Registry back then, so I used gproc as the registry, but other than that, I believe the talk is still up to date.

Not sure I understand what you mean by this.

Regarding:

I’m not sure registry offers any value, I would still have to come up with a random key in the registry to avoid clashes required to start the top level supervisor more than once

make_ref() returns a nice unique key.

Yes. It’s matched against in the receive loop. When a OTP special process parent exits then the process must exit.

This is a little different than a parent in a way I don’t fully understand and the little I can understand, I can’t explain. But I’ll try anyway!

In iex when I use import_file to import an .exs file and in that file I start a linked process there’s an extra pid in $ancestors but I’m still linked to the iex process.

Task actually only meets one of the requirements for a special process.

Be started in a way that makes the process fit into a supervision tree