Crowdhailer
Pachyderm - Virtual actor framework for durable, globally unique processes
Summary
Durable, globally unique processes (Entites) are created by controlling side effects.
(In short side effects are deferred until the state change of the entity has been committed to storage).
The purpose of an entity is to handle data that must be strongly consistent within a given execution context.
Example (as of 0.2.0)
Entity behaviour
defmodule MyApp.Counter do
@behaviour Pachyderm.Entity
alias MyApp.Counter.{Increment, ...}
alias MyApp.Counter.{Increased, ...}
def init() do
%{count: 0}
end
def handle(%Increment{}, %{count: count}) do
events = [%Increased{amount: 1}]
if count == 9 do
effects = [{MyApp.AdminMailer, %{threshold: 10}}]
{:ok, {events, effects}}
else
{:ok, events}
end
end
def update(%Increased{amount: amount}, state = %{count: current}) do
%{state | count: current + amount}
end
end
Calling Entities
type = MyApp.Counter
id = UUID.uuid4()
reference = {type, id}
{:ok, state} = Pachyderm.call(reference, %Increment{})
# => {:ok, %{count: 1}}
Design notes
A detailed discussion of why this API was chosen and further examples is available in the project README.
Why
The single global process is very easy to reason about, but has many pitfalls.
The dangers of the single global process.
Pachyderm tries to rescue this simple model by offering a standard way of managing some of the pitfalls.
Prior Art
The description of this project is quite difficult because there are lots of related but different approaches to this problem.
Other terminology includes:
- Single Global Processes
- The Virtual actor model Microsoft Orleans
- Resilient Actors
- An Actor Database Actor-Relational Database Systems: A Manifesto
- A database-less App framework
- Sticky Processes
Most Liked
tristan
It says it is based on Orleans so hopefully a safe assumption that it does storage similarly. If this is the case then it writes an etag with the state of a grain to storage when it saves state and uses the etag it read in on start to compare against the one currently in the store when doing the write. if the etags don’t match then it fails.
In sql this looks like https://github.com/erleans/erleans/blob/master/priv/blob_provider.sql#L60 for an update. This is from my project Erleans that is also based on Orleans.
pma
https://github.com/rabbitmq/ra implements Raft for consensus and allows defining side-effects that run on the leader only. Using it with disk_log as storage layer for an event sourced system
Crowdhailer
Ra looks interesting. At first I thought It might be cool to make an alternative Log storage. for Pachyderm using it.
However it could be even more useful, because it handles side effects as data in much the same way as Pachyderm. Just finished watching this talk. Which is really informative.







