What's the cost of DB as a service?

[lleger]

I actually gave a talk recently on our hosting journey for Elixir. The TL;DR is that we started on Heroku, went to Gigalixir, and then finally landed on AWS. I’d definitely recommend using a combination of ECS for running the Elixir service and then using RDS for the DB. It’s a great set up and we’re super happy with it.

Here’s all the code from that talk if you want to spin up your own, it should be plug and play: https://github.com/newaperio/revista

[sanswork]

Is the talk online anywhere?

[lleger]

Yup, here you go:

[sanswork]

Thanks will check it out over lunch

[AstonJ]

@sanswork

It depends entirely on your application. For many though yes definitely even when they’re small the difference can be noticeable.

I run PG in several sites and as of yet, none of them have required me to tweak my PG configuration. I’ve also not heard people complaining that PG is poorly optimised out of the box, just the opposite - that it’s so stable and efficient coming from DBs like MySQL

Perhaps if you’re running on tight hardware it might need tweaking, but on a dedicated server, I just can’t see it being an issue tbh.

@sanswork

If you’re ok with losing up to an hour of your customers data then they are fine. If that isn’t ok then no. DBaaS don’t do anything I can’t but they can do it so I don’t have to. Stuff like setting up a separate barman server at another location so I can have point in time backups for example makes running my own just as expensive if not more(up to a point, at the higher end DBaaS get really pricey). Again this is just for me and my use cases. If some data loss isn’t critical(like for a blog where updates are infrequent, or a forum where losing an hours posts isn’t that big a deal) then it doesn’t really matter what you do and in those use cases I tend to just stick a postgres server on the same box.

I agree that it depends on what’s on the server - though even then you can do things like Raid arrays for contingency, off-server backups, etc. If you felt you really had to set up something like barman there are lots of online guides that would help.

99% of the things I have learned about servers is via such guides and forum posts

@sanswork

If you enjoy the process then I envy you because it would certainly make parts of my day a lot more fun if I did too.

I think that’s a large part of it too - if you loathe it then don’t do it I don’t mind it, I quite like it because I feel as though I’m more in control of what I can do and don’t feel as restricted.

[sanswork]

I run PG in several sites and as of yet, none of them have required me to tweak my PG configuration.

The fact you haven’t done it doesn’t mean your systems wouldn’t run better with time spent doing it though it just means they don’t run poorly enough for you to care.

Perhaps if you’re running on tight hardware it might need tweaking, but on a dedicated server, I just can’t see it being an issue tbh.

It’s actually the opposite. The default config is targeted at non-dedicated setups with limited memory availability. Most of the changes I make on a new postgres server involve calculating the optimal settings for the available memory(upwards), and modifying linux settings to improve handling of stuff like swap space(db throughput death).

though even then you can do things like Raid arrays for contingency, off-server backups, etc. If you felt you really had to set up something like barman there are lots of online guides that would help.

Yup, but now you’re managing multiple servers, or testing that your contingency stuff actually works regularly when instead you could pay $50/month and have someone else do that for you.

[frigidcode]

Also there is the security aspect of doing it yourself. Encryption at rest /transit, SSL connections between client / server, firewalling / vpn, etc.

[AstonJ]

@sanswork

The fact you haven’t done it doesn’t mean your systems wouldn’t run better with time spent doing it though it just means they don’t run poorly enough for you to care.

I wouldn’t say they run poorly at all. They’re lightening fast

@sanswork

It’s actually the opposite. The default config is targeted at non-dedicated setups with limited memory availability. Most of the changes I make on a new postgres server involve calculating the optimal settings for the available memory(upwards), and modifying linux settings to improve handling of stuff like swap space(db throughput death).

Those are not hugely difficult adjustments to make though

@sanswork

Yup, but now you’re managing multiple servers, or testing that your contingency stuff actually works regularly when instead you could pay $50/month and have someone else do that for you.

Again it boils down to what’s important to you. I think dedicated servers are the way to go because of the freedom and power you get with them.

@frigidcode

Also there is the security aspect of doing it yourself. Encryption at rest /transit, SSL connections between client / server, firewalling / vpn, etc.

There are loads of guides out there that you can follow for all of those things

[roblally]

There are three types of costs here, and different people, in different circumstances can, quite rightly, value them differently:

• The cost of paying for a dedicated server v’s the cost of a DBaaS service.
• The cost of failure/recovery/downtime/data-loss and the relative likelihood on a dedicated server v’s a DBaaS service.
• The opportunity cost of managing a dedicated server; what else could you be doing with that time?

In general, DBaaS will probably:

• be more expensive
• be more reliable
• be faster to recover from failures
• have less chance of data-loss
• let you spend your time on other things

Pick your poison based on your needs.

[mitkins]

I like getting my hands dirty with my OSs/services. But for my side project, I was looking out for the cheapest solution. It’s currently residing with Gigalixir while in development (on the free tier). One thing that has been nice is being able to concentrate my efforts on my code, rather than think (too much) about servers, databases and backups.

Now that things are moving on (read: I’m reaching the 10,000 row limit) I’m thinking about my next move. Currently, I plan to use Cloud SQL on Google Cloud directly. As far as I understand, Gigalixir and Cloud SQL can run in the same zone, so the latency will be similar (if not the same) as my current solution.

Last time I checked, that’ll increase my costs from 0 to roughly $US9 per month. If my side project ever grows (in number of users) I intend to move to a paid plan for my app server on Gigalixir

Having said all that, I think if the above option wasn’t available to me, I might roll my own server (to reduce cost)

[gcauchon]

3. Just Heroku-ing the whole thing, even though I’d lose some BEAM-y goodness by doing so

@landric Would a distillery/docker deployment on Heroku be an option?

My guess is your 3rd option is a mix-based deployment using Heroku’s buildpacks, which is not how Elixir code should be deployed imho… You would have a 100% BEAM compatible binary, plus the peace of mind not to have to deal with managing instances!

[sanswork]

which is not how Elixir code should be deployed imho

What is the difference when you’re targeting heroku with both?

[gcauchon]

mix is a development tool, it was not meant to run code in production! Building releases directly from mix is actually an ongoing effort by the Elixir core team…

It takes at max 1h to setup a release with Distillery when you have a little knowledge of Docker.

https://hexdocs.pm/distillery/guides/phoenix_walkthrough.html

Add a Mix Config Provider to the recipe and you get a real Twelve-Factor binary that can run anywhere, thus on Heroku since it supports Docker deployment.

https://hexdocs.pm/distillery/config/runtime.html#mix-config-provider

[sanswork]

Right but what is the practical difference? If heroku is your only deployment target then what difference does it actually make?

[landric]

@landric Would a distillery/docker deployment on Heroku be an option?

mix is a development tool, it was not meant to run code in production! Building releases directly from mix is actually an ongoing effort by the Elixir core team…

Theoretically? But Docker is just a whole other thing for me to learn. Could I take the time? Sure. And I’d be interested to hear more about the practical (as opposed to… philosophical? Not trying to be perjorative, jut not sure I have the right word) benefits of Distillery → Docker → Heroku vs. just using the Elixir Buildpack to run the app with mix.

[gcauchon]

There are a few documented limitations in Phoenix’s Deploying on Heroku guide, which might not be relevant to your project. Running an Elixir project in production using the language development tool is clever, but not what was meant to be.

This thread from 2017 actually explain it better than I could ever do :

Always use Releases

Correct if me I’m wrong, as best I can tell there aren’t any reasons to use mix run --no-halt in production vs releases. The marginal value always outweighs the marginal cost. Marginal cost: I’ve seen a lot of folks think that using releases means you MUST have a separate build vs application server/image/whatever. Heroku is a good example. You have a buildpack that contains Elixir and you compile your app with mix. What is the marginal cost of using releases on heroku? Just one extra command!…

[gcauchon]

Especially this part from @sasajuric

But above all, I like the self-contained, isolated nature of the release. You won’t pickup some unexpected module from somewhere else (e.g. mix app is not included in OTP release, so less chances of invoking Mix.env at runtime), and you can easily run multiple different systems (services) with different versions of Elixir/Erlang.

I also agree that the cost is marginal. But even if it takes longer to properly setup your own build and deployment, it’s a one-off cost. Pay the price once, reap the benefits many times

Keep in mind this thread is almost 2 years old! Distillery 2.0 was officially released last August and simplified the setup by a full order of magnitude. Especially the ENV handling that is now almost as simple as System.get_env/2 with mix configurations.

[sanswork]

Thanks, that is a great thread to read.

[gcauchon]

A friend pointed me to another great reference on the subject in Distillery’s repository :

Distillery, Understanding releases

Unlike source code, which requires that you have the build tools, fetch dependencies, and compile a new version (which may differ from the last time it was compiled, due to different tools or dependency resolution resolving newer versions of your deps), OTP releases disconnect development and deployment in such a way that you only need to build the artifact once, and can then deploy it many times; rather than build the artifact each time.

The more you know

[Udi]

You may want to check out Gigalixir for a heroku-like experience (more heavy on command line) but tailored for elixir. I’ve had good luck with it so far. Based on google cloud by default but I believe that’s flexible.