I have never used CockroachDB (though I have spent far too much time reading about it…), but at some point you are going to run into transactions that require retries because the database is distributed and relies on clock synchronization for single-key linearizability, so if you perform reads/writes on the same key from different nodes within the clock skew interval it’s going to have to abort since it doesn’t know which came first.

In such a case you have to retry the transactions on the client because the SQL is conversational so there’s no way the database can reissue the transaction on its own. I don’t think Ecto has any provisions for retries, but you should be able to just wrap the transaction in a function and recursively call it if you get an error (perhaps with a counter to avoid looping forever).

It sounds like the problem you’re running into in this case, though, is simply a serialization error (i.e. the same you would get in Postgres if you used SERIALIZABLE), in which case taking a pessimistic lock on the row will just cause the transactions to block each other instead of aborting. So if you just SELECT FOR UPDATE and then UPDATE and COMMIT, you should be fine. If you’re taking multiple locks you have to be careful to take them in order or you’re going to get deadlock aborts instead, of course.

For retries, we have a transaction_retry function in our Repo module.

  def transaction_retry(fun, options \\ [], max_retries \\ 3) when max_retries > 1 do
    transaction_retry_rec(fun, options, 1, max_retries)
  end

  defp transaction_retry_rec(fun, options, attempt, max_retries) do
    transaction(fun, options)
  catch
    # serialization_failure
    _kind, %Postgrex.Error{postgres: %{pg_code: "40001"}} = exception ->
      Logger.warning(
        "Transaction failed due to serialization error. Retrying... (attempt #{attempt} of #{max_retries})"
      )

      if attempt <= max_retries do
        sleep_ms = trunc(:math.pow(2, attempt) * 4 + :rand.uniform(10) + 1)

        Logger.info("Sleeping for #{sleep_ms}ms before retrying...")

        :timer.sleep(sleep_ms)

        transaction_retry_rec(fun, options, attempt + 1, max_retries)
      else
        Logger.warning(
          "Transaction failed after #{max_retries} attempts. Giving up. exception: #{inspect(exception)}"
        )

        reraise exception, __STACKTRACE__
      end
  end

When calling this you always have to keep in mind that whatever code you have inside this transaction can be executed multiple times, and should generally be without side effects.

If you want follower reads, those can also be specified at the transaction level. We have a function in our Repo for that as well:

  def set_transaction_follower_reads! do
    {:ok, _} =
      Ecto.Adapters.SQL.query(
        __MODULE__,
        "SET TRANSACTION AS OF SYSTEM TIME follower_read_timestamp()"
      )

    :ok
  end

This can be called at the beginning of a transaction in order to enable follower reads for the whole transaction:

Repo.transaction_retry(fn ->
  Repo.set_transaction_follower_reads!()

  # [...]
end)

The lock is for pessimistic locks, e.g. lock("FOR UPDATE"). The hints keyword in from/2 is the documented way to add options after the from clause.

I see from your past posts (including in this thread) that you were doing this before. Was it not working? The Postgres support was cut into a release over a year ago.

BTW, this is possibly unrelated, but Cockroach doesn’t support AS OF SYSTEM TIME and FOR UPDATE in the same query. Which makes sense, I guess, since you can’t lock the past