Making sure data is successfully added locally and through remote API?

Questions / Help
,
1

I’m trying to create customers in Stripe using stripity_stripe and in my Postgres db at the same time. I want to make sure that if either fails to add the user, the user doesn’t get added anywhere and instead an error is returned. What would be the best way to do this?

2

Perhaps wrap the Stripe step in an Ecto transaction:

MyRepo.transaction fn ->
  customer = MyRepo.insert!(%Customer{})
  case Stripe.add_customer(%Customer} do
    {:error, reason} -> MyRepo.rollback(reason)
    _ -> {:ok, customer}
  end
end

But note that depending on your locking strategy you may now be throttling your database by the speed of the Stripe API.

3

Thanks! I’ll give that a try.

4

For who did not know the term yet: This kind of ‘making sure all operations together succeed or fail’ is also known as ‘making a sequence of operations atomic’. (Atom is greek, and means ‘indivisible’) Under ‘atomic operations’ or ‘atomic transactions’, there is a lot of information and guides to be found.

For working with Ecto, an Ecto.Repo.transaction is the way to go, and if you have a larger sequence of multiple queries and functions to perform, look into Ecto.Multi.

One problem I’ve faced, though, is that while the database-related stuff now either happens all-at-once or not-at-all, the functions that are run in the middle might still happen.

Example:

Repo.transaction(fn ->
  update_game_state_in_db(some, params)
  {:ok, results} = perform_remote_request()
  update_user_score_in_db(user, results)
end

If perform_remote_request() is an action that updates data somewhere, it will still have happened, even if update_user_score_in_db/2 fails after that (rolling back the update_game_state_in_db/2 results as well).

I am not yet sure how to solve this problem; as you quickly face the Byzantine General’s problem: Which one of the two remote APIs (or local GenServers for that matter) should send the final ‘ok, everyting went fine, carry out the changes’-response to the other party? And what if a failure happens during ‘carrying out the changes’?
If someone can provide any tips on solving these kinds of issues, I’d be very hapy :slight_smile: .

5

Maybe you should rethink the issue a bit. Is it really a big problem to have customer in your local db and not in an external API (stripe)?

I would do the following

  • create the customer in the local db, with a column synced_with_stripe = false (or something)
  • when the customer is successfully created call the external API to create a customer
  • when the customer is successfully created on the external API update the local customer record with synced_with_stripe = true

In the above case you can retry the external API (stripe) calls if they fail and you can always filter the customers if needed by synced_with_stripe

2 Likes
6

Yeah I really wish that Ecto.Multi had a ‘run’ that took two functions, one to do an operation and the other to undo it if a rollback is needed. That would solve some cases (though not all, would be nice to have a run that takes 3 functions as well, one for action and the other two for either commit or rollback).

7

Such thing wouldn’t work either…

Just think about this scenario, which in a similar fashion already happened to a friend of mine:

  1. start the session
  2. write order to db
  3. try payment, it sends ok
  4. try set order status to paid, but it fails because network went away and DB is on another host
  5. cancel the payment from the payment provider, network hasn’t recovered until now
8

Hence the ‘though not all’, that is one of the ones it would not. :wink:

1 Like