Sanjibukai

Sanjibukai

Hello everybody,

It’s not really a Phoenix problem (and might even be a devops concern) but I’m asking in the context of a Phoenix App.
Until now I never really thought about database redundancy or backup (besides what it’s already done in the file system level by the VPS provider in case of a disk failure).

In my current project, payments will be involved and I’m wondering how I can make the payments data redundant so that if something bad happen I can start fresh with at least the data of who made payments. (In fact this particular scenario is already covered by the payment provider itself e.g. stripe which provide the history of the transactions).

I don’t really know what are the best practices for this kind of use-case and wanted to know what are the best practices?

  • Is it simply managed at the OS/FS level for disk failure handling? But what if it’s the VPS itself which is broken?
  • Do people use two (or more) databases at once (for each write/update/delete requests) which will run ideally on different provider?
  • Does another solution might be to still use only one DB and perform regular dump? (it’s an easy task when doing it manually from time to time but I saw that quickly becoming complex if it’s needed to do it more often and ideally by transferring the dumped data into another location)
  • What about an “event store” or some kind of dedicated DB/queue/pubsub system which will only store the payments events (with the associated data like email of the payer, etc.)?
  • Maybe people are using DB systems which have built-in support for write replicas? (IIRC mongodb have this kind of feature)

I’m really interested to hear how you guys are handling that or if you have some resources explaining what are the best practices.

Thank you!

Showing Posts 1 to 3

hauleth

hauleth

VPS probably has some kind of RAID setup for preventing disk failures, you do not need to worry much there. But remember - RAID is not backup.

There is concept of hot/warm/cold-replica that replicates the data from the master and sometimes is used for reads as well.

You can use dumps or if your DB supports it, you can also use streaming of write-ahead-log (if your DB supports this). The latter is a little bit more complex to set up, but will provide you with point-of-time recovery (so you can recover state of the DB at any point you like).

It is log that will contain events as they happen in your application. That is it.

Multi-master replication is quite hard to get right, but most of the DBs have support for such (either built in or 3rd party). But if all you want is redundancy, then master-replica is simpler and easier approach.

danielzfranklin

danielzfranklin

Consider using an external service like Datadog or Papertrail or Logly for monitoring and log management. It’s useful to get alerts about the status of your server anyway.

Then, if you log everything needed to recover (like user x bought plan y) you can download the logs and write a little script to restore the db if you need to.

shanesveller

shanesveller

Please don’t ever make this your “plan A”. That is not a DR strategy, that’s what you do under duress to save a capsizing business because you have no other choice. Logs are not a system of record, as many many things have to go just right for them to even get collected and retained.

— All posts loaded —

Where Next? Top

Trending in Discussions Top

AstonJ
As the title says, please share what you’ve been up to with Elixir. Whether that’s been learning it, looking into it, making stuff with i...
2977 92995 915
New
caslu
I want to open this thread for you all to discuss and help those who really like Ash but are still hesitant to use it in a real project. ...
New
mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
GES233
I’m posting this in response to Jose’s recent tweet (Cr. link) : People are sleeping on Elixir for a coding harness: Hot-code swappi...
New
_mfierro
Hello, I wrote Stop My Hand, a Scattergories-like web application using Phoenix/LiveView as my learning project for Elixir (after readin...
New
marciol
It would be helpful to have a list of companies worldwide that hire engineers without prior experience in Elixir. Often, it can be quite ...
New
durvia
Anyone running long-lived stateful processes on BEAM? We’re building an AI agent runtime and would love to compare notes. We’re a small ...
New

Other Trending Topics Top

garrison
Hobbes is a low-level distributed database for the Elixir programming language. Hobbes provides a simple, safe, and scalable storage lay...
New
marciok
Hi there! We created Gust: A task orchestrator inspired by Airflow. For those who have never heard about Aiflow, it’s a Python-based wor...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
mcass19
ExRatatui lets you cook up rich terminal UIs in Elixir, powered by Rust’s ratatui via Rustler NIFs. Build interactive terminal applicatio...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve. They are GUI (Emerge) and State management (S...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews