benonymus

benonymus

Delete file from aws(arc) fom postgres trigger

Hey there,

I am using arc for file processing with aws, and I am trying to delete images from aws once an item is deleted
from my db.

I am using postgres triggers(https://blog.lelonek.me/listen-and-notify-postgresql-commands-in-elixir-187c49597851), so I am getting the item form the database, arc for the files only saves a string to the database such as:

test_document_fEfRY-pR2JX3jIF9CrXPC.pdf?63754075895

My problem is, since my trigger is upon deletion I cannot get the item by id from Repo to have the file taken care of by arc.
The attachment inside my item instead of a map is just a string.
I am usually using arc’s delete from the avatar with the image and entity but without having the item going through ecto it just won’t delete by that string that I mentioned above.

Any thoughts how could I get the attachment deleted with the string or how to load that attachment manually in order to be able to delete it?

Most Liked

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

I would probably invert this whole process. Don’t delete the record from the database first. Mark the database record as deleted_at pending_deletion or something, and then enqueue a job in something like Oban which can try to hit AWS to delete the record. Only after that has succeeded do you delete the record in the DB. The flag lets you filter it out from UI lists immediately, but you want the record around to reason about the file until it’s actually gone.

The job part here is critical. AWS S3 has good uptime, but not perfect up time. Requests to delete files will fail periodically and will need to be retried.

baldwindavid

baldwindavid

If you like Arc, you might take a look at Waffle (I've forked Arc to Waffle and merged all open PRs). Arc seems to have gone unmaintained and Waffle is a fork with fixes. I prefer the transparency and control of working directly with ex_aws_s3 and orchestrating uploads and deletions via Ecto.Multi, but understand the allure of an opinionated package.

dimitarvp

dimitarvp

Hm, I see. In this case I recommend you switch approaches. You’re better off putting a “please delete attachmend with ID 123” command on a message queue (RabbitMQ, Kafka, you can also use your DB as well) and then have a background worker that periodically works through those and just have that delete the files from AWS.

Where Next?

Popular in Questions Top

greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
nobody
Hi! In PHP: $SERVER['SERVERADDR'] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New

Other popular topics Top

vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 53578 245
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I fore...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
PeterCarter
There are pre-rolled solutions for other frameworks that do work. However, Phoenix does not seem to have these. Have people had good expe...
New
sergio
Kind of like when jquery came out, it was super necessary. Existing drag and drop libraries have a bunch of baggage to support old browse...
New

We're in Beta

About us Mission Statement