Fl4m3Ph03n1x

Fl4m3Ph03n1x

Save state in a different proccess?

Background

I was re-reading Elixir in Action yesterday and I came across a curious quote from @sasajuric:

If the code of your error-kernel process is complex, consider splitting it into two processes: one that holds state, and another that does the actual work. The former process then becomes extremely simple and is unlikely to crash, whereas the worker process can be removed from the error kernel (because it no longer maintains critical state).

Separating state into different processes

I get that by doing this you would gain error isolation - if the worker blows up, you don’t lose your state. Great, right?

But why use a clumsy, boilerplaty, GenServer for the task? I see some issues with this approach:

  1. The State process can still crash
  2. Communication with a GenServer is slow compared with using an ETS table
  3. Your GenServer state process is a single point of access (meaning it is a bottleneck) and a single point of failure.

Is this approach really viable?

I don’t think I have ever seen a case where this makes sense. What I have seen, is people saving state in an ETS table which then has a dump_process that writes the table’s content into disk every X minutes.

This approach is considerably faster than using a GenServer and works wonderfully with concurrent accesses.

Question

  1. So, why on earth would I save my critical state in a GenServer process? (examples would be appreciated)
  2. Isn’t using ETS tables a lot better?

disclaimer: the author may have given this hint because he only introduces ETS tables in a later chapter.

Marked As Solved

sasajuric

sasajuric

Author of Elixir In Action

The main reason why I didn’t mention ETS tables is indeed because they have not yet been introduced in the book at that point.

Using ETS table to store the snapshot of the state is definitely a valid option. If there’s no other logic in the process, then my goto approach is to use an ETS table, for the same reasons you mention.

That said, ETS tables might not always suffice, simply because they have limited support for atomic, isolated operations. So there might be some cases where the state process might need to be a GenServer, and you’ll need to serialize some actions (sometimes only writes, sometimes everything) through it.

So more generally, the part of the book you quoted is about thinking in terms of isolating error effects (which is the name of the chapter), while keeping the state in ETS tables would fall into a “beyond GenServer” category (which is the name of the next chapter :smile: )

Also Liked

keathley

keathley

Any process can crash in a system. Thats not really the point. The point is that a simple KV store in a GenServer would be the opposite of “clumsy”. It would be so simple that the likelihood of crashing is very low.

I agree with your points about ETS allowing for more performance long term. But there’s overhead in having ETS tables lying around so if you don’t need it then its waste. The idea that always using ets is just better than using a genserver isn’t a solid general rule.

In practice we tend to use a combination of the two. A GenServer will start an ets table and process all writes to that ets table. That way writes are applied serially. However all reading is done in the client process. As an aside, this is why the “boilerplate” of a GenServer is useful. All of this logic is hidden behind that “boilerplate” and is transparent to the end user of the api.

Neither situation is that difficult to test if you want to test in isolation. The api can take an optional name argument and look up the process and ets tables with that name. In production you would use the default name. Thats pattern tends to be less egregious than using something like a mock.

LostKobrakai

LostKobrakai

You’re not wrong, but adding ets to the mix is also one more moving piece in your system and it’s more complex than e.g. a simple Agent, which basically does nothing but read state. Not to mention the write side if you need that as well.

Fl4m3Ph03n1x

Fl4m3Ph03n1x

So, assuming that my state has a low number of accesses per second, and that using an ETS would provide me with the same failsafe resilience as using a process, I would still be better off using an ETS because it would allow me to easily get concurrent access in the future without ever worrying with bottlenecks.

My point is, if you are going to separate state from work, do it with an ETS table as it grants you more benefits on the long term (meaning you already have concurrent access to state when needed).

Where Next?

Popular in Questions Top

chokchit
** (DBConnection.ConnectionError) connection not available and request was dropped from queue after 2733ms. You can configure how long re...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
tduccuong
Hi, is there any work on GUI with Elixir, that is similar to Electron/Javascript? My idea is to bundle Phoenix and BEAM into a single se...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> somethi...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New
srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call t...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New

Other popular topics Top

9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
johnnyicon
Hi all, I’ve just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I’m trying to use Postgres...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 43622 214
New
RisingFromAshes
I’ve read in another post that it may be possible with a router helper - but I couldn’t find an appropriate one, and tbh, I’m still just ...
New
klo
Got a question about when to concat vs. prepending items to list then reversing to achieve appending. So i know lists boil down to [1 | ...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New

We're in Beta

About us Mission Statement