Best Practices for Cleaning up cloud stored resources associated with application Structures!

Hello Fellas,

In every Phoenix project that I have been involved with so far at somepoint we have database stored data structures that one way or another reference s3 stored binary objects.

So far my proceess for handling those kind of resources would envolve having an external process such as the live_file upload leveraging Javascript uploaders to store the binary files to s3 and then uppon deletion I am using ex_aws_s3, to delete files.

So far I am deleting s3 resources in the same function call that I am deleting the database stored data.

For example in a live view that might be something like the following:

def handle_event("delete", %{"id" => id}, socket) do

I have 2 problems with this approach

  1. There is no way for my to know that a database stored Struct can be deleted before deleting the s3 stored resources. For example if I have a structure that is referenced by other structures in some scenario I don’t want to delete the resource but instead warn the user so in that case I would not want to delete the s3 resousce either is there any approach that can create something like a “transaction” for those actions ?. (For obvious reason reversing the order has other drawbacks if the s3 resource deleting for some reason fails)

  2. Another problem that I am dealing with, using this approach is that I can’t make sure that a s3 cleaning up is called everytime the delete functionality is invoked on a particulat structure, are there any best practices for this scenario ?

2 Likes