Fl4m3Ph03n1x

Fl4m3Ph03n1x

Is this usage of Repo.transact an anti-pattern?

Background

I am studying the Transaction Script pattern, namely from a book called “Learning Domain Driven Design”.

For those of you unfamiliar with this pattern, here is a small description:

The Transaction Script organizes business logic by procedures, where each procedure handles a single request from the Public Interface of the application, aka, the presentation layer.

In the book, the author explains that one of the limitations of this pattern is when you have to take an action that must be atomic across several different storage/communication mechanisms. Imagine updating the database and sending a message via a broker - for the system to remain consistent, both must be done atomically.

  @spec execute_v1(integer(), NaiveDateTime.t()) :: :ok | {:error, any()}
  def execute_v1(user_id, visited_on) do
    Repo.query!("UPDATE \"user\" SET last_visit=$1 WHERE id=$2", [visited_on, user_id])

    MessageBus.publish(%{user_id: user_id, visited_on: visited_on})
  end

Question

In most languages, ensuring this behaviour remains consistent and transactional would be impossible to do. In fact this is the main premise to introduce CQRS and the Outbox Pattern in later chapters.

However Elixir does have a workaround for this limitation, that can make sure this set of operations is atomic and consequently ensures the consistency of the system (or so I believe):

@spec execute_v2(integer(), NaiveDateTime.t()) :: {:ok, [Postgrex.Result.t()]}
  def execute_v2(user_id, visited_on) do
    Repo.transact(fn ->
      update_result =
        Repo.query!("UPDATE \"user\" SET last_visit=$1 WHERE id=$2", [visited_on, user_id])

      :ok = MessageBus.publish(%{user_id: user_id, visited_on: visited_on})

      {:ok, [update_result]}
    end)
  end

By using Repo.transact Elixir allows us to make sure this piece of code is atomic. If publishing of the message fails, the Database operation is rolled back. In this specific instance, because we only send 1 message, I believe this workaround would work just fine.

Even though I am using this workaround in this context, I would like to make it clear I have seen people use it with all sorts of things. A few come to my mind:

  • Sending HTTP messages after writing to the database
  • Doing database writes on multiple storage systems (usually relational DBs and non-relational DBs at the same time)

However, I always have the same questions.

  1. Is Repo.transact supposed to be used this way?
  2. Isn’t this considered an anti-pattern by the community?
  3. What are the alternatives to this workaround, if any?

I am also curious to know if anyone else reading the code thinks this can fail in a way I have not yet predicted. Please let me know!

Marked As Solved

LostKobrakai

LostKobrakai

That’s code example is not consistent. You could publish to the message bus but fail to commit the transaction. This becomes more obvious once you look at sql queries behind this:

BEGIN TRANSACTION;
UPDATE …;
-- publish to message bus;
-- consider a failure here
COMMIT;

This is no different in elixir than it is anywhere else. The problem stems from distributed computing, which doesn’t care at all about what language runs on individual actors.

Also Liked

al2o3cr

al2o3cr

This is something that Oban can help with. Extract the external side-effect code to a background job, and then enqueue the job inside the transaction.

If the transaction commits, the job is executed.

If the transaction rolls back, the job rolls back with it and is never executed.

One limitation of this approach is that it can’t handle the “roll back the transaction if the external request fails” scenario.

garrison

garrison

Lol you guys, this is not a matter of implementation details. What @LostKobrakai was trying to tell you is that solving this is literally impossible.

You can have at-most-once or at-least-once but you cannot have exactly-once in these situations.

You can make one side of the transaction idempotent and then guarantee at-least-once for that side. For example, you could commit the row to Postgres first and then repeatedly attempt to publish the message with some sort of idempotency id until it succeeds. There was actually a pretty good post about this on the Tigerbeetle blog recently.

As a side note, @dimitarvp and I had a discussion about something similar a while ago and I want to point out that this sort of thing is what I was talking about. Gluing multiple databases together can be messy and hard to understand, even if you’re knowledgeable in this area.

An alternative approach is to just have a really scalable multi-model database and use it for everything. The database can then worry about atomic commits for you. FoundationDB was a pioneer of this approach, and it’s one of the things I’m trying to do in Elixir with Hobbes.

garrison

garrison

Yeah, unless someone does the enormous (ask me how I know) amount of work needed to serve as a base for another approach there is really not much you can do.

Ideally you just shove everything into Postgres as long as you can get away with it, as @al2o3cr alluded above. Postgres may have garbage consistency guarantees out of the box but at least its developers actually care about free software and won’t rugpull you at the earliest opportunity. And also at least it has a serializable flag even if nobody uses it lol.

If you need to hit an external API then you’re cooked but this is why good external APIs (e.g. Stripe) have idempotency as a first-class feature.

Where Next?

Popular in Questions Top

vegabook
I’m brand new to Phoenix and I have stripped one of the demo applications to the bone. I just want to get an svg up on the screen. Here i...
New
jaysoifer
Is there a way to rollback a specific migration and only that one (“skipping” all the other ones)? Would mix ecto.rollback -v 200809061...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
chokchit
** (DBConnection.ConnectionError) connection not available and request was dropped from queue after 2733ms. You can configure how long re...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New

Other popular topics Top

KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 36432 110
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
AngeloChecked
What learn first? Rust or Elixir Hi Elixir community! I’m here because i want learn a new language. I’m a junior developer and mainly i ...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New

We're in Beta

About us Mission Statement