rm-rf-etc

rm-rf-etc

Replacing stored procedures with Elixir processes?

Some of the apps where I work frequently suffer outages as a result of our stored procedures, and someone suggested that maybe we could remove some of this load on the database by bringing the data into elixir. Has anyone here seen something like this where they work?

The query-by-id parts would likely get translated into phoenix topics. What would be a good way to backup the in-memory data in case a process crashes? I’m imagining that elixir could generate the needed processes based on demand, so that any processes representing objects that haven’t gotten any attention in a while, could be put to sleep and left in the database, to be warmed up the next time someone asks for those objects. Our apps stall when specific records get a lot of attention, so the reads and writes tend to be concentrated around a specific object ID and all the records associated with it.

Most Liked

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Ah! I finally get it. Which database are you on? This stored procedure is basically a home grown version of postgres’s INSERT ON CONFLICT so if you’re on postgres you may be able to refactor this to just use that and move on.

Beyond that though, I was referring to EXCEPTION WHEN unique_violation THEN. The whole thing is weird though because if the id is a primary key, indexing on (id, month) shouldn’t actually help because querying on just id should already be indexed. Where does arg_id come from in your stored procedure?

mindok

mindok

There’s a discussion on caching to reduce DB load here (particularly for heavy reads that are frequently hit). It is probably a bad idea to roll your own caching system using raw processes - it gets hard quite quickly (e.g. handling timeouts, knowing when to invalidate etc).

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Is data loss acceptable?

This is exactly the issue. If your database is slow and inefficient, batching data in memory before inserts just adds another layer of failure. It isn’t clear how this would even help with the stored procedure issue, cause those will ultimately fire when inserted anyway.

If you have valuable data, and you can’t tolerate data loss, then you can’t afford to store that data in memory, it must be written to a disk somewhere. If reads are your issue and not writes then you can look into caching.

Last Post!

rm-rf-etc

rm-rf-etc

Yup, I see it. I shared this link with the team.

I think I was confused what the IF found does. I think I understand now.

Where Next?

Popular in Discussions Top

New
jesse
Hi everyone, I hesitated to post this here because I don’t want you to think I’m spamming, but I’ve been working on a Platform-as-a-Serv...
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 54996 245
New
wmnnd
The Go vs Elixir thread got me thinking: Would it be too hard to implement a simple mechanism for creating Go-style static app binaries f...
New
Fl4m3Ph03n1x
Background A few days ago I was listening to The future of Elixir from Elixir Talks, with Dave Thomas (@pragdave ) and Brian Mitchell. I...
New
owaisqayum
I have a sample string sentence = "Hello, world ... 123 *** ^%&*())^% %%:>" From this string, I want to only keep the integers, ...
New
Rustixir
Hi everyone, im working on find best language/framework/system for high concurrency, high performance and stable performance after wor...
New

Other popular topics Top

hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" => #BSON.ObjectId<58eb1a7a9ad169198c3dXXXX>, "email" => ...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New

We're in Beta

About us Mission Statement