I say ignorance isn’t an excuse for poor application design.
Now I’m not familiar with the Ecto codebase but I’d expect that by and large the module to give a hint as to whether or not a function call crosses the process boundary. And even when that isn’t the case the function parameters probably tell their own story - if you need to specify the repo you’re likely crossing the process boundary.
But isn’t this done everywhere in the erlang ecosystem? Do you really care if something uses a process, or an ets table or something else and how would you ever be able to change the implementation unless you hide the functionality behind an interface?
It is even recommended in a number of books to hide the fact that you are talking to a for example a gen_server.
I start with the API I want. Once that is set I can change the implementation anyway I want (and I often do because I don’t know the optimal solution up front) without having to change all the code using the library. Perhaps this is convenience over correctness but in practice I don’t see it working in any other way.
I think the answer is, as always, it depends.
For client-server architectures (where client can be anything short-lived, most commonly a process handling a single HTTP request), I think that most of the times it’s fine not to care about if it’s a function call of sending a message. In worst case scenario you will just time out. Of course there are various cases where this might cause a bottleneck or other problem, but I’d say that this should be the server’s concern, not the client’s.
The situation looks a lot different where it’s not client-server, but there’re multiple cooperating processes. Then, you are exposed to a lot of different problems (for example deadlocks) which are really hard to debug. In this case, the difference between a function call and sending a message (and, what’s the most problematic - sending a message and awaiting the answer) is huge.
I would expect a competent Erlang developer to realize on a conscious and subconscious level that (synchronous) API calls are in fact:
while at the same time using the “let it crash” philosophy and the tools of process links, exits (+ trapping), monitors, supervision, etc. to manage potential termination in some reasonable fashion.
Do you really care if something uses a process
I would at least prefer to know up front whether a function call has the potential to block process execution. Because at that point the decision needs to be made whether it makes sense for the process to be blocked - and if it isn’t, it’s time to spawn a separate process for the express purpose of being blocked (instead).
Elsewhere I already expressed my puzzlement about some places suggesting to choose “call” as a default over “cast”. All too often a “default” is interpreted as a strong preference or worse, a “99% of the time” choice - often accepted to avoid having to actually understand the nuanced consequences of all the available choices.
I think one of the best uses of process state is to store context and correlation information for pending asynchronous requests (i.e. cast messages expected to eventually result in some kind of return message or timeout).
Synchronous API calls have their place but my concern is that mindless adoption (for the sake of convenience) will waste valuable opportunities in process architectures.
In terms of Erlang vs. Elixir - in my experience Erlang educational resources tend to dwell much more on the details of working with the primitives of spawning new processes, sending and receiving (asynchronous) messages - giving the learner a better perspective how different (from more mainstream platforms) the BEAM environment actually is.
Elixir resources seem all too eager to instead impress with agents, tasks etc., without building a sense and appreciation of how really asynchronous the fundamentals are (sometimes leaving neophytes with a false sense of familiarity and security).
There are some very articulate thoughts about when it does and does not make sense to use processes to hold state above.
I have some intuition myself about this, mostly leading to using process state for ongoing interactions when the interaction model is time-bound and matches a single connexion or session. Those would be some types of games, web sessions, etc.
However I would be grateful if you could apply your opinion re. whether or not it’s a good idea to use process state to various examples just to make sure I understand what you mean. Especially if @peerreynders and @hubertlepicki could chime in, that would be very useful 
In his online Elixir course (non-free, Elixir for Programmers) Dave Thomas uses one stateful GenServer holding the state of a Hangman game (letters already guessed, game status, turns left, etc.) for each connected client.
Lance Halvorsen, in line with his book that got recommended on this forum on several occasions (Functional Web Development with Elixir, OTP, and Phoenix) presented his Elixir at a Walking Pace talk at ElixirConf US 2018. In it, he describes how he uses one process per item to be tracked in a wharehouse / shipping system, complete with on-demand cold start (read state from disk). Albeit he admits this is going to be hell to scale to a multi-node system in the talk and does not really give an applicable solution for this, he seems to see quite a few benefits in doing this in terms of architecture.
EDIT: There is a reference by @peerreynders to a thread on the PragProg forums that might address this at least partially (GenServer docs: “handle_cast … should be used sparingly” Why?) but those forums seem to be closed now and I could find no cache of this thread elsewhere.
On that last one, I’d especially would love to get feedback from @hubertlepicki, as you seem in agreement above with not using processes to hold state as some default model, but at the same time seemed to agree that using them to maintaing aggregates is fine. I might be seeing a contradiction that does not exist there, and would really like to hear it from the horse’s mouth
All this I guess eventually boils down to the (already mentionned above) excellent To spawn, or not to spawn? by @sasajuric
To frame it in terms of that article, is using a GenServer per DDD aggregate not a pure “thought concern” (red flag reason for using a process to maintain his state) because its role is to serialize access to that aggregate, and thus is a legitimate concurrency unit?
Dave Thomas uses one stateful GenServer holding the state of a Hangman game
In my view it’s an acceptable implementation largely because the state is ephemeral, i.e.:
he seems to see quite a few benefits in doing this in terms of architecture.
The primary problem would be finding the relevant state if you don’t know where it lives.
However in most cases it is just simpler to stick with a single node solution. Because the BEAM is so resource efficient you can pack a lot of stuff on a single node. Also keep in mind that multi-node solutions are primarily a reliability measure, i.e. maintaining availablity in the face node hardware failure. Distributing heterogenous task across nodes is possible but not the primary focus of BEAM distribution and can quickly get complicated if you don’t want a single node failure to cripple your system (because that node was the only node to provide some critical services).
it has been suggested that GenServers are a good fit for holding DDD-aggregate type state
I think the point is that they can be under certain circumstances, i.e. it depends.
Ultimately state is simply data and you will have functions in modules to inspect that data and to create new versions that evolve that data as a result of the sequence of events that the state has been exposed to over time.
That is all you need to represent state because that data can be handed to a function wishing to inspect the current state.
Placing that data inside a process gives it a location or place where the containing process can “evolve” the state while the process itself be contacted by other processes.
I think classifying a process as a container of state is an oversimplification - it makes it sound like the process only serves to maintain state. A process exists to enact a protocol in concert with other processes and the state contained by all the participating processes serves the protocol (or whatever the protocol is trying to accomplish).
Viewing processes merely as containers of state could easily lead to “ask-style” architectures (Tell, Don’t Ask) which would be far from optimal.
Many thanks for taking the time to share your insights.
OK, that connects with what I had in mind as well, re. this being time-bound and not having strong persistence requirements .
Yes, I’ve spent a lot of time during the last years doing distributed-systems related work, and despite the ambient kool-aid my advice to customers is to avoid at all cost doing distribution if they can avoid it. The BEAM is indeed a good tool to stretch what one can do on a single node, and even with Erlang/Elixir I tend to design by default for stateless server farms when horizontal scalability and/or HA and disaster recovery are required.
But that being said, all “stateful process” situations do not create that situation, a good example would be stateful client channels with Phoenix, as already mentionned in this thread.
Haha, yes, of course. This is interesting to get back to the good old “it depends” there, as the stateful vs stateless question tends to trigger some pretty dogmatic point of views if my recent online readings are any indication. No silver bullet etc.
I get the dumb container anti-pattern, and was exposed to the TellDontAsk / LawOfDemeter / OneResponsibilityRule corpus of principles years ago.
However, approching that set of issues from the “Tell, don’t ask” angle when using remote and/or message-passing systems and not strictly in the context of neo-OO (Java, C++) often leads me to some peripheral (to this problem) considerations, close to Steve Vinosky’s stance on RPC.
It’s actually quite funny that you choose to point to Alec Sharp’s take on it and not one of the more commonly linked articles from Martin Fowler or even the c2 pages. He probably talks about it with a Smalltalk-centric vision. Which for me, in an Elixir context makes the idea that CSP / Agents / Erlang processes might be the purest implementation of the Alan Kay vision of objects resurface. Which might seem paradoxical, because as mentionned in the article you linked, the TDA principle often is coined as being a reminder that Objects are bundles of data + functions at their core, which depending on how you read and interpret it might be at odds with Alan Kay’s “The important thing about objects is message-passing”.
Is this just a surface, language-level-trick apparent contradiction in your opinion or is there something more profound at work there?
EDIT: I reread myself and it seems that I was just rambling, feel free to ignore me
I came across the idea via Why getter and setter methods are evil (2003) - more recently Allen Holub has been going with the “Tell, Don’t Ask” moniker in his OO talks. And most sources seem to ultimately point back to pragprog.
Steve Vinosky’s stance on RPC.
I keep quoting Convenience over Correctness (2008) hoping somebody will listen.
interpret it might be at odds with Alan Kay’s “The important thing about objects is message-passing”.
Alan Kay’s insight is enlightening as it moves focus away from “the mighty object” but it could still be considered myopic.
I’m referring to Protocols as discussed in Thinking like an Erlanger.
The software industry got caught up with CASE tools while Class-Responsibility-Collaboration (CRC) Cards (1989) got largely ignored. Lots of attention to Classes (Objects) and Responsibilities but the dynamics of Collaboration - not so much. People focusing on individual trees and ignoring the forest.
PS: Some further discussion on To spawn, or not to spawn?
I guess that we should wrap this up before we get moved to an other thread as being off-topic, even though I feel we’re digging in a direction directly related to the original thread topic
Well, there is quite a lot of old ideas recycling in all those well-coined modern catchphrases.
I distinctly remember spending a long time thinking about how splitting objects in a system would work to reduce complexity, and how to design interactions between those objects and make sure we don’t fall into the bare-objects trap by ensuring we’re not querying state but exchanging events/messages/commands. This was after reading a pre-UML book by Grady Booch, the one with the plastic tracing ruler included, for drawing model diagrams And this was probably the early 90’s, I’m quite sure that i.e. Fred Brooks touched on those topics as well with similar conclusions.
And I thank you for that, one of your links prompted me to read that one again a few days ago. A good candidate for my “read once a year” list
I did not particularly enjoy that talk, not sure why, maybe because I was already convinced But I can definitely vouch for the excellent Principles of Protocol Design by Robin Sharp. As a consultant I tried to put the message out that thinking in terms of protocol yields many benefits (of course there are tradeoffs, as always), but with little success so far. Either I lack the proper approach / arguments or it’s simply too much of a mind-shift for my target audience.
Yup. And I fear all the Agile agitation this last decade, even though some of it was based on starting points that made sense, misled a generation of programmers (sorry, “craftsmen” ) to thinking that they can get away without doing any systems design and just focus on micro-optimization of their individual bricks, with the assumption that adhering to whatever communication method of the day exonerated them from thinking about the actual communication taking place, sometimes optimizing for consistence of call conventions rather than consistence in communications semantics. People coming from manufacturing systems, logistics or in general industrial computing however seem to have been more immute to that than the general public though (totally anecdotical observation, I have no proof of that).
Thanks for the heads up, I’ve read the original article but missed the thread on these forums. Will do
