landric

landric

What's the cost of DB as a service?

Some background: I’m a solopreneur paying rent with what’s currently a Rails app on Heroku, and I’ve been about two months away from deploying a rewrite in Elixir/Phoenix for the past year. :+1:t2:

I’ve got Edeliver deploying to some DigitalOcean droplets, and I’ve been toying with some different options for where/how to host my Postgres.

Options I’ve considered:

  1. Rolling my own Postgres on another DO droplet
  2. Continuing to use Heroku Postgres (or some other Postgres as a service, Compose looks like a thing?)
  3. Just Heroku-ing the whole thing, even though I’d lose some BEAM-y goodness by doing so

I specifically wanted to test #2 (continuing to use Heroku Postgres) to see how much latency “cost” I’d be incurring by having my app servers and DBs not in the same “house”.

My test was seed script that adds about 60,000 rows of test data (roughly representative of one year’s worth of data for a customer of my company). Here are the results:

DigitalOcean droplet (NYC-3) to a DB on localhost
4 minutes and 43 seconds

DigitalOcean droplet to another droplet in the same region
6 minutes and 40 seconds

Digital Ocean droplet to Heroku Postgres
51 minutes and 57 seconds :man_facepalming:t2:

I wasn’t quite prepared for the magnitude of the difference. If the cost of DBAAS convenience is a tenfold increase in latency for my users, my inclination is to invest my time in running my own database. Would love to hear what choices others have made…

Most Liked

roblally

roblally

Cost isn’t simple, it varies with a number of factors, including risk.

You’re talking about never having had a failure, which is fine, and, if you’re considering a single node then banking on it can totally make sense. I’m considering 100K+ servers, 1K+ applications and 1K+ databases deployed across 1K+ physical locations. With a mean-time to failure of various components in the 10 years range, that means I need to think in terms of continuous partial failure as the norm. Servers will fail every day, probably in cascades/clusters because that’s how **** happens.

You’re talking about being able to flip things over manually. I’m in a world where a critical failure at the wrong time can cost tens of thousands of dollars per second. How many seconds will it take you to solve the problem? I’m in a world where an increase in latency can cost millions of dollars as people abandon an app, or a site, or a shopping cart in a store.

Say I’m in a highly seasonal business, where our hardware utilization is high for a small part of the year and very low for the rest: *aaS services let me scale and pay for what I need, when I need it. That makes it cheaper in real terms.

Say I have processes that only run 5 * 8. I can turn off my databases for the rest of the week and pay only for storage. That makes them much cheaper.

Combine scaling only when I need it and scheduled shutdowns and I can reduce my costs from *aaS offerings by a really high percentage.

Can you build something that solves those problems yourself? Absolutely! Is it easy or cheap? No, it isn’t. It also isn’t easy to test it and make sure that it will actually work when you need it to. Cloud/*aaS offerings are much more capable of investing the time and energy necessary for that.

Not all money is created equal, either: in general purchasing hardware is a capital expenditure which means I need to carry the value of that hardware for years, pay taxes on the value of it and I need to either have the money or borrow it. SaaS services are all operational expenditure, which makes it more attractive from a taxation and net-present-value of capital perspective.

Say I’m in San Francisco, where the full cost of an employee is somewhere in the region of $250K per year - salary, pension, healthcare, training, insurance, holidays, office-space, travel, etc. etc. etc. If I’m considering 2000 staff, what percentage of our total productive output should we spend on database administration?

Overall, I want to minimize the total cost of ownership of our databases whilst creating an acceptable risk profile and the right performance characteristics. *aaS is sometimes the right option for that, and sometimes it isn’t.

I toss out these figures, not because I, or the company I work for are special, but precisely because we’re absolutely not. There are thousands of companies like us around the world. This is the reality of enterprise software; we’re not optimizing for cost of individual servers, we’re optimizing for organizational level concerns and we’re architecting for risk because - in a big enough corpus of possibilities - ten thousand to one risks happen dozens of times every day.

Your concerns are different from mine, I totally respect that your choices works for you, and I’m not suggesting you should do anything different. It doesn’t work for me and it certainly isn’t always cheaper, in real terms.

sanswork

sanswork

The time spent on optimising your postgres config for the hosts they are on, setting up a backup system and regularly testing your recoveries, monitoring the postgres lists for security announcements, etc. To me that is too much hassle to save a few bucks a month. At least until you get up into the $XXXX level on a DBaaS which most people will never hit.

sanswork

sanswork

Yeah these are things that aren’t problems until they are and when they are I’m real happy they are someone elses problems. Also if you aren’t spending time optimising your config you’re probably leaving a lot of performance on the table. Which admittedly for most of our projects probably doesn’t matter but it is nice to have it.

The backups gem only does pg_dump style backups so if you have a crash or someone makes a mistake you will almost certainly still lose data. Possibly a lot depending on how often you do snapshots.

I use to run all my stuff on dedicated servers but having each system split off into its own virtual server is just so much nicer for me. I don’t have to worry about a single point of failure, I can easily compartmentalise tools on the server, etc. Now with stuff like PaaS and DBaaS my life is even easier. I’m not paid to be a fulltime admin so the less of it I can do the better.

Where Next?

Popular in Discussions Top

matthias_toepp
I’d love to hear what people think about Wisp, the new Gleam web framework started by Gleam’s primary creator Louis Pilfold. Gleam, alon...
New
cvkmohan
The upcoming Phoenix 1.6 release looks very interesting. Became a habit to watch the commits - and - what they are bringing in. phx.gen...
New
gausby
I asked this very same question on twitter and got some interesting feedback, but I thought it would be a good question to ask here as we...
1207 39467 209
New
ricklove
I was just introduced to Elixir and Phoenix. I was told about the 2 million websocket test that was done 2 years ago. From my research, t...
New
New
und0ck3d
Hello everyone! A few days ago I’ve created a topic here about how people were creating CMSs with Elixir and Phoenix. I’ve been studying...
New
ben-pr-p
In general I’ve been sticking to this community style guide GitHub - christopheradams/elixir_style_guide: A community driven style guide ...
New
100phlecs
Are there any downsides, like perf issues, to putting all functional components in CoreComponents as long as you prefix it with the conte...
New
acrolink
How does the two languages compare when it comes to server side application development? Any experiences or ideas? Thank you.
New
opsb
We’re considering our architecture from a viewpoint of scaling our traffic heavily over the next 6 months. Our current deployment is runn...
New

Other popular topics Top

WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
274 41989 114
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
johnnyicon
Hi all, I’ve just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I’m trying to use Postgres...
New
chrismccord
This release brings a number of exciting features, including integration with the new Phoenix LiveDashboard and Phoenix LiveView. There h...
New
AngeloChecked
What learn first? Rust or Elixir Hi Elixir community! I’m here because i want learn a new language. I’m a junior developer and mainly i ...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
RisingFromAshes
I’ve read in another post that it may be possible with a router helper - but I couldn’t find an appropriate one, and tbh, I’m still just ...
New
PeterCarter
There are pre-rolled solutions for other frameworks that do work. However, Phoenix does not seem to have these. Have people had good expe...
New
svb
Hi! Currently I want to submit a form by pressing the Enter key. However, since my input field is of type “textarea” this is just adds a...
New

Latest on Elixir Forum

We're in Beta

About us Mission Statement