tfwright

tfwright

Unexplained rollback after 2.21 upgrade

Soon after updating our Oban mix spec to 2.21, an error report came in about a unexpected return from a Multi transaction involving an Oban insert. None of the changes had any errors so there was no immediately explanation what was causing the error in the App/Ecto logs, just the rollback. After dropping down to pg logs the error was clearly a bad/missing migration to Oban v14 schema, causing an outdated enum definition that was breaking the insert. Is that expected? Our expectation was that class of error would be surfaced in Oban’s query execution or insert return value.

Most Liked

sorentwo

sorentwo

Oban Core Team

It’s expected because the changeset is valid, and it doesn’t know there’s anything wrong until it attempts to insert and a value is missing from the database enum. That’s unexpected, and something you should be able to identify in the test environment.

We’ve just added a migration verification step to help catch this automatically in tests/CI: Verify migrations at startup in testing mode · oban-bg/oban@6dbea1f · GitHub

sorentwo

sorentwo

Oban Core Team

No, I suppose it wouldn’t have caught the issue then.

This adds to the evidence that it’s better to use Repo.transact/1 rather than a multi, which have plenty of little edge-cases that can trip you up (not using the prefix passed to the transaction it’s ran in, for example).

Last Post!

tfwright

tfwright

If I am understanding the problem correctly (and I’m honestly not at 100% there) I don’t think Multi is at fault here. Repo.transact (and Repo.transaction) would have the same problem since they all share the same SQL transaction handling. It appears to mark the connection as failed and that is what triggers the rollback on next (or possibly final?) retry: db_connection/lib/db_connection.ex at 165ce62b9636ae56d4a0f0810c2aa3803b3cff1e · elixir-ecto/db_connection · GitHub

run/3 and transaction/3 can be nested multiple times. If a transaction is
rolled back or a nested transaction fun raises the transaction is marked as
failed. All calls except run/3, transaction/3, rollback/2, close/3 and
close!/3 will raise an exception inside a failed transaction until the outer
transaction call returns. All transaction/3 calls will return
{:error, :rollback} if the transaction failed or connection closed and
rollback/2 is not called for that transaction/3.

And some logs visualizing the transformation of the error. Looks like it does pass through until the final retry (also a short circuit probably would be preferable for this case, not sure if that’s practical though)

[Oban.Repo.transaction] attempt=1
[Oban.Repo.transaction] rescued error=Postgrex.Error message=ERROR 22P02 (invalid_text_representation) invalid input value for enum oban_job_state: “bogus”
*[Oban.Repo.transaction] attempt=2
[Oban.Repo.transaction] rescued error=DBConnection.ConnectionError message=transaction rolling back
[Oban.Repo.transaction] attempt=3
[Oban.Repo.transaction] rescued error=DBConnection.ConnectionError message=connection is closed because of an error, disconnect or timeout
[Oban.Repo.transaction] attempt=4
[Oban.Repo.transaction] rescued error=DBConnection.ConnectionError message=connection is closed because of an error, disconnect or timeout
[Oban.Repo.transaction] attempt=5
[Oban.Repo.transaction] rescued error=DBConnection.ConnectionError message=connection is closed because of an error, disconnect or timeout
[Oban.Repo.transaction] max retries reached, reraising
Expected exception Postgrex.Error but got DBConnection.ConnectionError (connection is closed because of an error, disconnect or timeout)

Where Next?

Popular in Questions Top

rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list. ...
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? Ecto.Repo — Ecto v3.14.0 has exampl...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New

Other popular topics Top

Qqwy
Update: How to use the Blogs & Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3271 130286 1222
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
AngeloChecked
What learn first? Rust or Elixir Hi Elixir community! I’m here because i want learn a new language. I’m a junior developer and mainly i ...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New
AstonJ
Posting this to see if we can make things easier for people to get into Neovim. If you use Neovim and have a favourite distro please let ...
New

We're in Beta

About us Mission Statement