Crowdhailer

Crowdhailer

Creator of Raxx

Pachyderm - Virtual actor framework for durable, globally unique processes

Summary

Durable, globally unique processes (Entites) are created by controlling side effects.
(In short side effects are deferred until the state change of the entity has been committed to storage).

The purpose of an entity is to handle data that must be strongly consistent within a given execution context.

Example (as of 0.2.0)

Entity behaviour

defmodule MyApp.Counter do
  @behaviour Pachyderm.Entity

  alias MyApp.Counter.{Increment, ...}
  alias MyApp.Counter.{Increased, ...}

  def init() do
    %{count: 0}
  end

  def handle(%Increment{}, %{count: count}) do
    events = [%Increased{amount: 1}]

    if count == 9 do
      effects = [{MyApp.AdminMailer, %{threshold: 10}}]
      {:ok, {events, effects}}
    else
      {:ok, events}
    end
  end

  def update(%Increased{amount: amount}, state = %{count: current}) do
    %{state | count: current + amount}
  end
end

Calling Entities

type = MyApp.Counter
id = UUID.uuid4()
reference = {type, id}

{:ok, state} = Pachyderm.call(reference, %Increment{})
# => {:ok, %{count: 1}}

Design notes

A detailed discussion of why this API was chosen and further examples is available in the project README.

Why

The single global process is very easy to reason about, but has many pitfalls.
The dangers of the single global process.
Pachyderm tries to rescue this simple model by offering a standard way of managing some of the pitfalls.

Prior Art

The description of this project is quite difficult because there are lots of related but different approaches to this problem.

Other terminology includes:

First Post!

Qqwy

Qqwy

TypeCheck Core Team

Very cool!

What is your approach in the face of netsplits?
EDIT: I’ve read through the README, and it seems, if I interpret the explanation in the deferred side-effects section correctly, that Pachyderm defers netsplit handling to the storage layer: The first event to be ‘committed to storage’ wins, and at that time other events that were based on a stale state will be dropped (and you can possibly retry them manually).

‘Commited to storage’ is not a very clear thing to say. What is the storage? What does it mean to be commited to it?
If the storage is distributed, then multiple nodes might both save things thinking they are the first. On the other hand, if the storage is only in one single place, you create a single point of failure and a bottleneck (since all events go through this storage).

Most Liked

tristan

tristan

Rebar3 Core Team

It says it is based on Orleans so hopefully a safe assumption that it does storage similarly. If this is the case then it writes an etag with the state of a grain to storage when it saves state and uses the etag it read in on start to compare against the one currently in the store when doing the write. if the etags don’t match then it fails.

In sql this looks like https://github.com/erleans/erleans/blob/master/priv/blob_provider.sql#L60 for an update. This is from my project Erleans that is also based on Orleans.

pma

pma

GitHub - rabbitmq/ra: A Multi-Raft implementation for Erlang and Elixir that strives to be efficient and make it easier to use multiple Raft clusters in a single system. · GitHub implements Raft for consensus and allows defining side-effects that run on the leader only. Using it with disk_log as storage layer for an event sourced system

Crowdhailer

Crowdhailer

Creator of Raxx

Ra looks interesting. At first I thought It might be cool to make an alternative Log storage. for Pachyderm using it.

However it could be even more useful, because it handles side effects as data in much the same way as Pachyderm. Just finished watching this talk. Which is really informative.

Last Post!

wanton7

wanton7

Reentrancy is not currently supported by Dapr but it’s coming Allow Reentrancy for Actors · Issue #21 · dapr/dapr · GitHub
Here are actor related docs for Dapr https://github.com/dapr/docs/tree/master/concepts/actors

Where Next?

Popular in Announcing Top

wmnnd
Hi there, for my project DBLSQD, I needed a file storage solution that is a bit more flexible than Arc. Because I thought others might f...
New
kelvinst
Hey everyone! Well, we made this lib a while ago and now we decided to finally go out and public with it! It’s a tool for creating and m...
New
danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 30486 241
New
maltoe
Hello! Came here to announce ChromicPDF, a pet project PDF generator I’ve been working on for the past few months. Why another PDF gener...
New
kip
Image is an image processing library for Elixir. It is based upon the fabulous vix library that provides a libvips wrapper for Elixir. I...
622 19610 194
New
scohen
Lexical Lexical is a next-generation language server for the Elixir programming language. Features Context aware code completion As-you...
New
MRdotB
I needed to reuse React components from my Chrome extension in my Phoenix/LiveView backend. I noticed that for Svelte/Vue, there are live...
New

Other popular topics Top

stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
AstonJ
Seen any cool LiveView demos, sample apps or examples? Please post them here! :003:
New
saif
Hello everyone, Long time lurker first time poster here. I’ve recently begun working on Elixir full-time again! :raised_hands: It’s been...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
AstonJ
Posting this to see if we can make things easier for people to get into Neovim. If you use Neovim and have a favourite distro please let ...
New

We're in Beta

About us Mission Statement