The changeset type is publicly documented, and so the data structure is part of the API. Changing the structure breaks the API, so it’s not an internal change.

Using struct matching would made this a bit more obvious:

%Ecto.Changeset{params: params, data: data} = changeset

You don’t get that even with GenServer. Peeking into the state and changing it is as easy as invoking :sys.get_state and :sys.replace_state.

I don’t think that cost is trivial at all.

First, by moving from functions to processes, you’re changing the paradigm. All of the sudden, instead of game = Game.something(game, ...) we’re just writing :ok = Game.something(game, ...). In other words we’re moving from immutable functional to side-effectful imperative.

This style now becomes similar to OO, except it has some extra issues. For example, it becomes easier to leak processes. If e.g. the game process traps exits, it won’t be immediately taken down when it’s parent terminates. If there’s some bug, the child might even linger on forever.

Another issue is that debugging becomes harder. If one process crashes, we’ll get cascading crashes (assuming no one in hierarchy is trapping exits), and that’s going to add a lot of log noise. If a crash happens in a “one-way” handler, the stack trace won’t be complete (i.e. you won’t be able to tell how did you arrive to that one-way handler). Tracing multiple processes is going to be harder than tracing a single one.

Going further, if an abstraction is process-based we can’t implement protocols for it. In a hierarchy of process-based entities, we can’t just invoke Jason.encode or :erlang.term_to_binary on the root element. Even ad-hoc debugging with IO.inspect(game) becomes unusable.

Yet another problem is passing the abstraction to other processes. If we pass data to another process, we make a copy, and two processes can safely work on the data concurrently. In contrast, when a pid is passed, it’s almost like pass-by-reference. So again we encounter a paradigmatic switch.

Moving to the performance realm, spawning a process and communicating with it is “cheap” (by some hand-wavy definition of cheap), but it’s still much more expensive than not using a process. In a tight loop where an abstraction is frequently accessed, the performance penalty might become really significant.

You might also experience weird timeouts here. Imagine you pass a million pieces of data to the abstraction in a one-way fashion (you seem to prefer that for mutations), and immediately after that invoke a getter. This might easily lead to a five-seconds timeout error.

Another problem is memory usage. A process overhead is about few kilobytes - an order of magnitude more compared to an empty map or a struct. So if we start creating a bunch of process based entities, and do this for every web request, the memory usage will skyrocket even in a moderately loaded system handling a few hundred or few thousand connected users.

This is further exacerbated by the fact that the data is copied across process boundaries. So in a two-way invocation of Game.do_something(some_data) we keep two copies of the data in memory, neither of which is garbage. Again, it’s usually not a problem, but overuse processes, multiply by the number of connected users, and you might find yourself in trouble.

Don’t get me wrong, I’m a huge fan of processes. Heck, the main focus of my book is on processes, and even in my aforementioned article I’ve used them extensively. Used judiciously, they can do wonders for our systems. But misused, they will bring a lot of harm with little to no good. I’m speaking from experience here, because I’ve spent my first few years of Erlang programming using processes for encapsulation (mostly influenced by my own OO heritage), and I’ve bumped into most of the issues mentioned above.

So tl;dr - no, I wouldn’t say that the cost of spawning a process is trivial

Glancing at the code of DirWalker, I see no reasons for it to be a GenServer. It could just as well be a plain functional module and still be able to keep state, perform read-aheads, and provide streaming. I didn’t thoroughly read the code, but I don’t see any current API concern which would justify using a separate process. If I was writing something like this, I’d make it a plain functional module.

I feel a similar way about your last post about state. You seem to argue that using a separate process for the game separates concerns and improves encapsulation. That’s certainly true, but I think that in your example you can get those same benefits by using plain functional abstractions (i.e. functions and modules). For a detailed discussion on this you can check out my To spawn, or not to spawn? post.

The gist of that long post is that I believe that design concerns, such as encapsulation, separation of concerns, or the shape of the API, do not play any weight in deciding whether something should run in a separate process or not. The reasons for spawning another process are IMO almost (if not completely) mechanical. I reach for a process if I want runtime benefits, such as concurrency and/or fault-tolerance. If I don’t get any of these benefits, then I stick with plain functions/modules.

If DirWalker had no process where would it keep its state: it creates the path list lazily: important on file systems with millions of files.

Internally the Hungry strategy uses async_stream, but it adds some stuff to it. First, it can be used in GenServers without messing up the message mailbox. It also adds convenient callbacks.

Finally, the overall approach above is synchronous. In want my component stuff to default to asynchronous for most uses, because that’s today’s world: event streams and reduces.

But, having said all this, I don’t think you’re wrong. I’m, exploring, just as everyone else is. I’m driven by this idea that things should be easier than we make them. The component abstraction is just the starting point for that exploration.

Cheers

Dave