silverdr
Transactional user registration
I would like to ask for a “best practice” suggestion on how to wrap a user registration process in a transaction that will either:
- store the user data in the database AND send confirmation email
if both storing and sending is successful
- neither store the user’s data nor send the email
if any of the two (storing data, sending email) fails
I currently use new_user_changeset to validate passed params and hash the supplied password before doing Repo.insert() and that works fine. I also created a Mailer based on “swoosh”, which I tested from the console to work as expected. Now a) where to put the Mailer invocation so that it is invoked only once the the new user is stored and b) how to wrap the process in a transaction and roll the database insert back if the mailer fails?
First Post!
dimitarvp
Most Liked
benwilson512
This didn’t work in Rails either. You can do it, but it doesn’t actually work the way you think it does. The email might go out, and then the transaction might fail because the database terminates, and then the email is not unsent. This means that it is not actually transactional.
It does mean that if the email fails to send, the user will be rolled back. BUT the inverse isn’t true. The transaction might rollback after the email has gone out for any number of reasons, and the email isn’t reverted because it’s an external side effect.
Arguably, sending emails inside a transaction is actually an anti pattern because you hold a database transaction open while waiting on an external API call which is unaffected by the transaction itself.
The after create is in the transaction sure, but the mail server is not in your database. Eventually your “in transaction” calls become API or SMTP calls to the mail server, and at that point stop being managed by the transaction.
You can do the same psuedo transaction in Elixir:
Repo.transaction(fn ->
user |> Repo.insert!
user |> send_welcome_email!()
end)
This suffers from the same limitations though. If the sending fails, then the user rolls back, which is good. BUT if the email works, and the transaction aborts for any number of reasons, the user will be rolled back, but the email won’t be unsent. This is just how the transactions work.
sasajuric
In addition to the issues already mentioned here, there is another problem. If the connection between the app server and the mail server breaks before the mail server managed to send a confirmation response, you might end up rolling back a perfectly valid transaction, while the mail is already sent. This is just a how distributed transactions work. As @benwilson512 said, you can’t use db transaction mechanism to commit or rollback the e-mail.
This is basically impossible to guarantee. If the network is a bit slower, the network request might take longer. You could try to enforce some timeout, but that will increase the likelihood of the aforementioned scenario (mail is sent while the transaction is rolled back).
IMO the proper way to tackle this is to use a db-backed queue to send mails. My current clients have switched to using oban for that. Basically, inside a db transaction an Oban job is created. Once the transaction is committed, the send mail job will be started. If this job fails (e.g. mail server is not reachable), we’ll retry it (after some delay).
A nice consequence is that with such approach we don’t need to pay so much attention in the transaction code. If you’re doing an immediate send from the transaction, you need to make sure it’s the very last thing you do (otherwise you increase the chance of e-mail being sent but the transaction failing). With the persistent queue, such issue doesn’t exist, since we enqueue the job in the same db, so we can do it anywhere inside the transaction. The job will only be executed if the transaction is committed.
Here’s another thing that won’t work reliably with immediate send:
Repo.transaction(fn ->
do_stuff_with_db()
send_mail_to_user_1()
send_mail_to_user_2()
end)
If the second send fails, we’ll rollback the transaction, but one user will be notified about the success, and we can’t undo that. Using the queue approach, things will work as expected.
So tl;dr use the queue (managed in the same db) to enqueue the pending send jobs (or any other kinds of jobs which involve an external remote system).
benwilson512
This cannot be done. Postgres transactions can only provide guarantees between postgres operations, and sending an email isn’t a postgres operation. What you can do is put the user data in a “pending” state, and change that state to “complete” or something after the email successfully fires.
Last Post!
silverdr
@LostKobrakai - thank you for your clear explanations.
It’s Mail Transport Agent. In the case I mentioned it’s local sendmail (alternative). It has its own persistent queue, work schedule, failure notification, etc.). If that’s not available or an external mail dispatch service is to be employed, I’ve also been using other persistent queue/deferred job implementations, Sidekiq as an example.
Just to make sure - the way I mentioned does have persistent queue and does notify once it gives up trying (email bounces to appropriate address and is distributed among those who are supposed to care), it also keeps logs, and all. The main difference as I see it now that in my example a) the responsibility is delegated to the MTA, which is a good thing - less code in the application, and b) it is not really transactional, which is a bad thing as the queue insertion may fail once the record is saved. Yes, I handle the errors but it’s not guaranteed to cover all situations. Which brings me back to where (and why) I started the thread. While putting non-DB operations in a DB transaction block is possible, it has its own problems. From the discussion here I understand that maintaining own queue and using a transactional queue insertion with Oban is the way to go with this kind of problem.
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex










