PaperTrail vs Carbonite vs Others?

Ash Forum Ash Questions / Help
,
1

So, I wanted to share my decision making experience on how I chose carbonite vs papertrail – for my own specific needs, others may have different requirements.

And maybe get insight from others in the community about other possible metrics, alternatives, or things to think about-- or even their own use cases when having chosen one solution or another…

Caveat that what I did was an amount of “playing around” and superficial analysis… nothing so technical or deeply exhaustive-- i can’t really know if I decided correctly until i actually build out the thing.

Problem Statement

A new Ash project is core to a new subdomain of our business, and of course, EventDriven notifications are a go to choice for signaling between different bounded contexts (outside of this project). And also, because of the writings/speakings of Pat Helland, Joe Armstrong and Rich Hickey, I definitely wanted to embrace the benefits – though not without challenges – of the ideas of immutability (where we can with data and identity), messaging/coordinations, etc. especially across bounded contexts, autonomous computing fiefdoms (per Helland).

I chose Ash for its Resource oriented declarative philosophy, and I’m lazy and don’t want to write GraphQL or migrations.

So I was looking for:

  1. Compatibility with Ash
  2. Emit events to messaging fabric – Primary Goal
    • Domain Events vs persistence CDC?
    • Transactional Outbox Pattern, w/ understood sender ordering guarantees?
  3. Have audit trail of changes – Major Goal
    • some resources we want stronger auditing on changes for debugging and customer support.
  4. Can we model immutability for general non-functional benefits?
    • And where we need mutability of resources, can we preseve past knowledge/fact?
  5. Revert Versions – not a goal
  6. Access past Versions – A mixed bag where it “depends” based on the problem domain… to balance append-only data (e.g. just log events) vs tracking mutating versions (e.g. changing of a webhook connection target – active, inactive, url changes).

Options

So for auditing and versioning in Ash, I found two immediate options:

Carbonite

Pros

  • transactional consistency – the change is persisted w/ resource all or none, unlike post-action notifications in elixir.
  • multiple ash resources changed in same transaction are explicitly tracked
    • “changes” table FK to “transactions” table, also includes postgres transaction ids.
  • we can set custom metadata on the transaction itself
    • specify actor of change
    • we can write the initiating ash-resource action… translate this to a “domain event”?
  • has transactional outbox processing

Cons, Gotchas, Coinflips

  • persistence level CDC, not domain level events
  • Implemented using DB Triggers – mixed bag here for me.
  • Outbox processing and ordering has subtle guarantees in ordering and processing.

PaperTrail

Pros

  • transactional consistency – the change is persisted w/ resource all or none, unlike post-action notifications in elixir.
  • strict mode foreign keys constraints for relating a resource to its versions
  • modeled as saved versions, and with the ability to look up old versions
  • straight forward wrapper around ecto
  • integration as Ash extension – still experimental though
  • has specialized originator and (of change) metadata to signify the actor of the change and the source processor of the change
  • open ended metadata for the change

Cons, Gotchas, Coinflips

  • different ash resources changed in the same transaction each get their own event row, but there is no strong correlation between the two rows.
  • I’ll have to write something completely custom to try for outbox processing.

Conclusion

I ended up using Carbonite because of my own use-case needs…

Related: I didn’t find anything in Ash to support event-sourcing itself … Though, maybe commanded has some sort of future integration with Ash… haven’t started any exploration in how commanded works-- and I don’t personally see the need for event-sourcing complexity in my own needs.

Mainly, the key deciding factor was the outbox processing and that different changing resources would all be strongly correlated by a transaction-- i.e. an action on an “Aggregate Root” would also manage other subresources such as changing “tags” (a different as resource, not actually embedded resource)… while for papertrail, I didn’t see the extra transactional outbox handling/sending, or the same correlation capability.

For “immutability” vs “versioning” … I think I can get away with, for my problem domain at least, modeling relevant resources as INSERT-only to provide immutability as needed… and that I only needed audit logging for the other mutatable entities/resource, and I would not need to revert or show the user past revisions.

As for compatibility concerns for Ash, while AshPaperTrail extension existed, I saw a past PR to get carbonite to work with ash… so just fine there.

Though, I’m still fleshing out how to best design “Domain-Level Events” out of this persistence level trigger-based CDC

  • Maybe that each transaction is annotated (metadata) with the AshResource action used, and translate that to some single Domain Event.
  • or maybe create additional domain events ash resources–persisted to also be captured by the carbonite CDC.
    • outbox processor handles the transaction, sees a domain event row inserted… and sends that out instead of the raw resource data change.
    • that are explicitly created as a result of a related ash resource action, which would give me possibility of many events to one action and better conformity to the domain.

And that is me mostly completely emotional-gut decision making, with a rationalization of arbitrary picked metrics.

I hope that eventually I’ll have enough experience working with it that I can have a more rational, experience/fact based, retrospective.

And I’d like to hear from others about if they had similar/different evaluations, share anything else in this field, or if there were other alternatives?

thanks!

4 Likes
2

Nice! Looks like a great write up. We’re also actively working on ash_paper_trail right now, working to make it better. We should include in that a way to pass down identifiers so that you can track the initiating event for any given version. With that said, the carbonite solution should be great as well. As for event sourcing strategies with Ash, I’ve played with a few ideas that have some interesting takes, but are not fully fledged. I’ve pushed them up to a non-production ready experimental package that may interest you: GitHub - ash-project/ash_events: An event-architecture extension for Ash (not ready for production use).

2 Likes
3

I hate to be a reply guy, but I need to point out that ash_paper_trail has nothing to do with the paper_trail hex package. I expect that they both implement pretty much the same behaviour inspired by the original paper_trail ruby gem.

1 Like
4

I stand corrected. My above analysis was just looking at the papertrail hex package itself.

I guess I didn’t dive into the ash paper trail properly. Now I’ll take a look to try to understand what ash-paper-trail itself actually does. thanks for the correction.

1 Like
5

Oops, I misread your bit on ash paper trail :joy: thanks @jimsynz

6

In case you want to use Carbonite with Ash, here is a quick way to do it.

First create a change module that will implement the Carbonite code like this:

defmodule MyProject.Ash.Changes.Audit do
  @moduledoc """
  This change will use `Carbonite` to create a audit of a specific action
  """

  use Ash.Resource.Change

  @impl true
  def change(changeset, opts, _) do
    type = Keyword.get(opts, :type, to_string(changeset.action.name))
    meta = Keyword.get(opts, :meta, %{})

    meta = Map.put(meta, :type, type)

    Ash.Changeset.before_action(changeset, fn changeset ->
      {:ok, _} = Carbonite.insert_transaction(MyProject.Repo, %{meta: meta})

      changeset
    end)
  end
end

Now, you can add this to the resources you want to use Carbonite:

  changes do
    change {MyProject.Ash.Changes.Audit, meta: %{actor: actor(:id)}},
      on: [:create, :update, :destroy]
  end

You can send whatever you want as the meta key, in this case I’m sending the actor id for easy audit.

2 Likes
7

I would actually suggest following the setup here:

That uses a transaction hook to ensure only one carbonite transaction is inserted. If you use changes like that, you will end up with conflicts when you chain multiple actions together.

2 Likes
8

But in that case you will apply carbonite to all tables in your database no? What if you wan’t to do this only for some?

Also, it seems that any other call for Carbonite.insert_transaction after the first one in the same transaction would be simply ignored anyway.

9

Yes, you’d want to use the strategy I gave if you were using it for all resources, although you could check the resource and action in the hook. If you only wanted it partially then your way should work.

10

But you would get into situations where if you have resource B call back to resource A to create one, you might get a transaction named like create_a even though the original thing you did was call an action on B.

11

I would be interested in your thoughts of Ash PaperTrail. I don’t remember why I went with Carbonite. It’s possible that I had already convinced myself that Carbonite was more mature, stand alone, and had reporting, but the downside is that it’s Postgres-specific. Happy Elixirin

12

As discovered in the thread, I wasn’t quite looking at Ash PaperTrail itself … rather it was a different paper trail.

But since I went with carbonite, I didn’t have a need to immediately re-evaluate Ash PaperTrail.

I would definitely look at Ash Paper Trail again if I had a need in a new project. But also, at that future time, I would also take a look at Ash Events – per zach.

Longer answer

And it would depend on the problem statement to solve. I could see different needs depending on the situation.

  • Is “versioning the resource” a first class concept for the feature, and exposed to the user? For example, a user may want to rollback/forward different draft versions of an Article?
    • Ash Paper Trail w/ version may look promising here
  • Or is it a non-functional issue like auditing?
    • I’ll want to re-evaluate the experience with carbonite at that time, see what worked or didn’t work.
  • Or how much we may need, or not need, “Event Driven Architecture”…
    • “Event Driven Notification” and/or “Event-State Carried Transfer” needed?
    • Do we want to keep resource state using “Event Sourcing” (Ash Events or Commanded)?
    • Would there be a path to use Event Sourcing, translate that State Event to a Domain (or Integration) Event for publication to Domain or other “Bounded Contexts”?
    • Would the problem even be complicated enough to need all the extra work/complexity for DDD and these kinds of Tactical Patterns?

Summary

But sorry :frowning: , @terris , I don’t have any thoughts on Ash PaperTrail for you at this moment-- maybe in the future.

3 Likes