rm-rf-etc

rm-rf-etc

Replacing stored procedures with Elixir processes?

Some of the apps where I work frequently suffer outages as a result of our stored procedures, and someone suggested that maybe we could remove some of this load on the database by bringing the data into elixir. Has anyone here seen something like this where they work?

The query-by-id parts would likely get translated into phoenix topics. What would be a good way to backup the in-memory data in case a process crashes? I’m imagining that elixir could generate the needed processes based on demand, so that any processes representing objects that haven’t gotten any attention in a while, could be put to sleep and left in the database, to be warmed up the next time someone asks for those objects. Our apps stall when specific records get a lot of attention, so the reads and writes tend to be concentrated around a specific object ID and all the records associated with it.

Most Liked

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Ah! I finally get it. Which database are you on? This stored procedure is basically a home grown version of postgres’s INSERT ON CONFLICT so if you’re on postgres you may be able to refactor this to just use that and move on.

Beyond that though, I was referring to EXCEPTION WHEN unique_violation THEN. The whole thing is weird though because if the id is a primary key, indexing on (id, month) shouldn’t actually help because querying on just id should already be indexed. Where does arg_id come from in your stored procedure?

mindok

mindok

There’s a discussion on caching to reduce DB load here (particularly for heavy reads that are frequently hit). It is probably a bad idea to roll your own caching system using raw processes - it gets hard quite quickly (e.g. handling timeouts, knowing when to invalidate etc).

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Is data loss acceptable?

This is exactly the issue. If your database is slow and inefficient, batching data in memory before inserts just adds another layer of failure. It isn’t clear how this would even help with the stored procedure issue, cause those will ultimately fire when inserted anyway.

If you have valuable data, and you can’t tolerate data loss, then you can’t afford to store that data in memory, it must be written to a disk somewhere. If reads are your issue and not writes then you can look into caching.

Last Post!

rm-rf-etc

rm-rf-etc

Yup, I see it. I shared this link with the team.

I think I was confused what the IF found does. I think I understand now.

Where Next?

Popular in Discussions Top

scouten
I’m looking for a host for the server part of a small (personal) side project that I’m working on. It’s currently written in Node.js and ...
New
AstonJ
I’ve just started the Phoenix part of the utterly brilliant online course by @pragdave. On generating the Phoenix app he uses the --no-ec...
New
MarioFlach
Hello, I want to share a project I’ve been working on for a while: https://github.com/almightycouch/gitgud Background Some time ago I ...
New
PragTob
Hey everyone, this has been brewing in my head some time and it came up again while reading Adopting Elixir. GenServers, supervisors et...
New
saif
Hello everyone, Long time lurker first time poster here. I’ve recently begun working on Elixir full-time again! :raised_hands: It’s been...
New
pillaiindu
I want to convert a Phoenix LiveView CRUD website to a CRUD mobile app. What do you think is the easiest way to do so?
New
AstonJ
Can you believe the first professionally published Elixir book was published just 8 years ago? Since then I think we’ve seen more books f...
New

Other popular topics Top

JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New

We're in Beta

About us Mission Statement