augnustin
Delete my file before deleting the Ecto entry on association
So I have this Resume module, which stores infos about my attached files, stored in my object storage. This module belongs_to the User module.
In my admin, I let my team delete Users, but if there is a Resume attached, it will fail.
I know I could set the on_delete: :delete_all option, which would delete all Resume DB entries, but it won’t delete those files in my object storage.
How can I make sure a specific method is called when Repo.delete(%User{}) is called?
It seems like Ecto.Model.Callbacks used to exist, but doesn’t any more, as well as on_delete: :fetch_and_delete.
But I can’t believe there is no solution for my case.
Thanks
Marked As Solved
benwilson512
Ecto model callbacks were removed in no small part because people like to use them for this sort of situation, but they are woefully bad at handling this situation. Suppose a user has uploaded 100 files, and it takes ~200ms to delete each file on S3 (pretty normal S3 response time on a bad day). If you did the file deletion call to S3 in a callback deleting the user would take > 20 seconds which is a long time to hog a database connection and could easily end up timing out the transaction, resulting in inconsistencies between the database and S3.
What we do in these cases is:
Step 1) Mark the user as “deleted” via a deleted_at, allowing you to filter them out.
Step 2) Enqueue a job to actually go delete the user.
Step 3) This job goes through each file one at a time, deletes the record on S3, then deletes the database row. Doing this asynchronously allows for S3 outages or other issues along the way to be OK, since you can just re-run the job.
Step 4) At the end of the job, actually delete the user record.
This avoids hogging database connections, and takes on deleting external files in an idempotent way so that any errors along the way don’t result in problematic, inconsistent state.
As a final note:
But I can’t believe there is no solution for my case.
These sorts of remarks are… unhelpful. Everyone here is helping you for free, all of the code you’re using is free. Few things suck the fun out of helping people like exasperation that the solution didn’t fall into your lap.
Also Liked
al2o3cr
Timing issues aside, the key mental shift to building this in Ecto is to stop thinking of Repo.delete(%User{}) as the same thing as “deleting a user”.
That deletes a row from the database, but there is more involved in performing the logical / domain action “delete this user” than that.
Your context should implement that - so for instance you’d call Users.delete(%User{}) and that function would handle both removing the database entries and cleaning up the files.
tfwright
This is really well put, I agree one hundred percent.
It might feel like it is “extra work” at first, but it’s extra work you only have to do if your app has extra complexity, and it promotes a design that makes that complexity explicit, easy to see, test and debug instead of hiding it away as if a piece of core business logic is a mere side effect.
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance









