landric

landric

What's the cost of DB as a service?

Some background: I’m a solopreneur paying rent with what’s currently a Rails app on Heroku, and I’ve been about two months away from deploying a rewrite in Elixir/Phoenix for the past year. :+1:t2:

I’ve got Edeliver deploying to some DigitalOcean droplets, and I’ve been toying with some different options for where/how to host my Postgres.

Options I’ve considered:

  1. Rolling my own Postgres on another DO droplet
  2. Continuing to use Heroku Postgres (or some other Postgres as a service, Compose looks like a thing?)
  3. Just Heroku-ing the whole thing, even though I’d lose some BEAM-y goodness by doing so

I specifically wanted to test #2 (continuing to use Heroku Postgres) to see how much latency “cost” I’d be incurring by having my app servers and DBs not in the same “house”.

My test was seed script that adds about 60,000 rows of test data (roughly representative of one year’s worth of data for a customer of my company). Here are the results:

DigitalOcean droplet (NYC-3) to a DB on localhost
4 minutes and 43 seconds

DigitalOcean droplet to another droplet in the same region
6 minutes and 40 seconds

Digital Ocean droplet to Heroku Postgres
51 minutes and 57 seconds :man_facepalming:t2:

I wasn’t quite prepared for the magnitude of the difference. If the cost of DBAAS convenience is a tenfold increase in latency for my users, my inclination is to invest my time in running my own database. Would love to hear what choices others have made…

Most Liked

roblally

roblally

Cost isn’t simple, it varies with a number of factors, including risk.

You’re talking about never having had a failure, which is fine, and, if you’re considering a single node then banking on it can totally make sense. I’m considering 100K+ servers, 1K+ applications and 1K+ databases deployed across 1K+ physical locations. With a mean-time to failure of various components in the 10 years range, that means I need to think in terms of continuous partial failure as the norm. Servers will fail every day, probably in cascades/clusters because that’s how **** happens.

You’re talking about being able to flip things over manually. I’m in a world where a critical failure at the wrong time can cost tens of thousands of dollars per second. How many seconds will it take you to solve the problem? I’m in a world where an increase in latency can cost millions of dollars as people abandon an app, or a site, or a shopping cart in a store.

Say I’m in a highly seasonal business, where our hardware utilization is high for a small part of the year and very low for the rest: *aaS services let me scale and pay for what I need, when I need it. That makes it cheaper in real terms.

Say I have processes that only run 5 * 8. I can turn off my databases for the rest of the week and pay only for storage. That makes them much cheaper.

Combine scaling only when I need it and scheduled shutdowns and I can reduce my costs from *aaS offerings by a really high percentage.

Can you build something that solves those problems yourself? Absolutely! Is it easy or cheap? No, it isn’t. It also isn’t easy to test it and make sure that it will actually work when you need it to. Cloud/*aaS offerings are much more capable of investing the time and energy necessary for that.

Not all money is created equal, either: in general purchasing hardware is a capital expenditure which means I need to carry the value of that hardware for years, pay taxes on the value of it and I need to either have the money or borrow it. SaaS services are all operational expenditure, which makes it more attractive from a taxation and net-present-value of capital perspective.

Say I’m in San Francisco, where the full cost of an employee is somewhere in the region of $250K per year - salary, pension, healthcare, training, insurance, holidays, office-space, travel, etc. etc. etc. If I’m considering 2000 staff, what percentage of our total productive output should we spend on database administration?

Overall, I want to minimize the total cost of ownership of our databases whilst creating an acceptable risk profile and the right performance characteristics. *aaS is sometimes the right option for that, and sometimes it isn’t.

I toss out these figures, not because I, or the company I work for are special, but precisely because we’re absolutely not. There are thousands of companies like us around the world. This is the reality of enterprise software; we’re not optimizing for cost of individual servers, we’re optimizing for organizational level concerns and we’re architecting for risk because - in a big enough corpus of possibilities - ten thousand to one risks happen dozens of times every day.

Your concerns are different from mine, I totally respect that your choices works for you, and I’m not suggesting you should do anything different. It doesn’t work for me and it certainly isn’t always cheaper, in real terms.

sanswork

sanswork

The time spent on optimising your postgres config for the hosts they are on, setting up a backup system and regularly testing your recoveries, monitoring the postgres lists for security announcements, etc. To me that is too much hassle to save a few bucks a month. At least until you get up into the $XXXX level on a DBaaS which most people will never hit.

sanswork

sanswork

Yeah these are things that aren’t problems until they are and when they are I’m real happy they are someone elses problems. Also if you aren’t spending time optimising your config you’re probably leaving a lot of performance on the table. Which admittedly for most of our projects probably doesn’t matter but it is nice to have it.

The backups gem only does pg_dump style backups so if you have a crash or someone makes a mistake you will almost certainly still lose data. Possibly a lot depending on how often you do snapshots.

I use to run all my stuff on dedicated servers but having each system split off into its own virtual server is just so much nicer for me. I don’t have to worry about a single point of failure, I can easily compartmentalise tools on the server, etc. Now with stuff like PaaS and DBaaS my life is even easier. I’m not paid to be a fulltime admin so the less of it I can do the better.

Where Next?

Popular in Discussions Top

thojanssens1
It would be nice to be able to define a redirect from one route to another from the router.ex file. E.g.: redirect "/", UserController, ...
New
AstonJ
I’ve just started the Phoenix part of the utterly brilliant online course by @pragdave. On generating the Phoenix app he uses the --no-ec...
New
Ankhers
Just a little information upfront. Generally speaking, if I feel like I need to either break a pipe chain or use an anonymous function in...
New
rower687
Hi all, I’ve been reading a lot about the “let it crash” term and how supervising processes and the whole messaging passing make an elixi...
New
jer
I’ve been using umbrellas for a while, and generally started off (on greenfield projects at least) by isolating subapps based on clearly ...
New
PragTob
Hey everyone, this has been on my mind for some time and I’d love your input on it! TLDR: I feel like maps are superioer for storing and...
New
eteeselink
Hi all, In the last days, two things happened: A blog post titled “They might never tell you it’s broken” made the rounds. It’s about ...
New
MarioFlach
Hello, I want to share a project I’ve been working on for a while: https://github.com/almightycouch/gitgud Background Some time ago I ...
New
jesse
Hi everyone, I hesitated to post this here because I don’t want you to think I’m spamming, but I’ve been working on a Platform-as-a-Serv...
New
axelson
Decided against including more info in the title, but the gist is that Plataformatec sponsored projects will continue with the assets bei...
New

Other popular topics Top

sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 29603 241
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
boundedvariable
I am going through the kafka architecture. All the features what the kafka is providing are already in Erlang. I would like hear your opi...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New

Latest on Elixir Forum

Elixir Forum

We're in Beta

About us Mission Statement