Extremely beginner questions about using Postgres/Ecto

Hello! I feel a bit embarassed posting here because I’m an amateur getting into web development as a hobby so my knowledge is very shallow (I feel like this kind of thing is more common with game development since more people are interested in game dev as a hobby, whereas with web dev I feel like everyone is a professional…)

I just went through all of Pragmatic Studio’s Elixir & OTP course and it was very easy to follow. I’ve started their Phoenix LiveView course but so far it feels different from the Elixir one, where everything was explained from the very beginning; it feels more something for people who were already using Phoenix and want to learn LiveView specifically.

I have a hobby project I want to develop with Phoenix, but my main problem is I’ve never worked with databases before. I’m confused about how working with the database locally and then deploying it would work. Because I know that my Postgres database is in the /usr/local/var/postgres folder, which isn’t my project folder. When I define my schema with Ecto and it creates the tables in that dev/local database, how do I then deploy that to production?

I see that the config/dev.exs file has the database info for the local database, so I suppose I’ll have a separate config for the production database. Does that mean I need to first create and host a database somewhere, then put the database connection info on that prod config somewhere, and then when I deploy Ecto will configure that database for me? But how would this work for future deploys after the first one, when the prod database already has data in it? If I change my schema and do a migration in dev, when/how would that migration happen in prod? It’s very confusing to me.

I’ve been looking for tutorials/courses on databases to understand things better, but I seem to find stuff aimed at people who want to be DBAs. I don’t want to be an expert, I just want to know the basics to get a small hobbyist project working. Does anyone have course/tutorial/book recommendations for a complete beginner to get started?

3 Likes

First, welcome!!!

Now, this question is a mother-load! (joking but there is some truth to that).

When I define my schema with Ecto and it creates the tables in that dev/local database, how do I then deploy that to production?

You have diff environment configs, as you mentioned. So the config will be there. However, I need to mention that this question involve DevOps tasks (how to release, when, process before releasing and so on). The deployment process to production is its own art. For that and for starters (unless you already did that), refer to Releasing.

Does that mean I need to first create and host a database somewhere, then put the database connection info on that prod config somewhere, and then when I deploy Ecto will configure that database for me?

In a way, yes. But again, things are never simple, right? They are simple but not simpler. So, yeah, as long as you have the right config for prod/release, you are good to go.

But how would this work for future deploys after the first one, when the prod database already has data in it?

There are mechanism already well thought but many genius at ecto, phoenix and so on. If you want to know the nuts and bolts, I recommend you to dive into the actual code of ecto!!! (HUGE THANKS TO ALL THE GODS AND GODDESSES OF OPEN SOURCE!)

Let us know if you have more questions :slight_smile:

2 Likes

This may be a historical side-effect: LiveView is much newer, relative to Phoenix. It’s also built on top of mechanisms in Phoenix, so you still need to understand those.

I’d recommend working through at least part of a “plain Phoenix” book/course first, to fill in those details that LiveView material is going to assume you know. For instance, “Programming Phoenix” from PragProg walks through setting up a migration and a schema, changesets, etc

4 Likes

Hey @meroron , welcome to the forum and to web development! As a fellow hobbyist, I thought I should provide some help. I’ve been a hobbyist web dev for 6 years, using Elixir for 3 years, and have deployed a lot of small personal and work projects in Ruby and Elixir on different platforms. Getting your hobby project off the ground with a working database is not as hard as it seems.

It might help you to read the Programming Phoenix book and just play around with Phoenix a bit first on a throw-away test project and not worry about deployment right away. That will give you some time and experience to learn best practices, and to make some mistakes and learn from them too. Once you are working on your passion project, I recommend deploying early and often, especially when you’re getting started. If you run into deployment issues after your app is already somewhat complex, then it will be harder for you to debug. Deploying early and often will allow you to catch each potential issue as it comes up.

When you deploy I recommend reading through the entire Phoenix Deployment Guides. You should be able to get your app and database deployed with any of the three PaaS they have guides for: Fly.io, Heroku, or Gigalixr. Lately I’ve been preferring Fly, for which you’ll need to read through the Intro to Deployment, Deploying with Releases, and Deploying on Fly.io, in that order. On your first deployment ($ fly launch), Fly will provision a Postgres database for you and automatically set the DATABASE_URL environment variable so your app will connect with the default config. Your prod database config actually lies in the config/runtime.exs file, so you should look that over to see what it contains.

By default, Fly will also run a release_command on each deployment. By default this release command runs any necessary migrations so your database schema will be up to date each time you deploy with no extra steps required. This is in the fly.toml file that the $ fly launch command generates for you, under the [deploy] release_command = "/app/bin/migrate" configuration.

Happy to help if you have any further questions, good luck!

2 Likes

Thank you all so much for the help and the warm welcome!!

I was interested in the Programming Phoenix book, but was a bit scared of the fact that it’s from 2019 / on version 1.4; like I see a lot of Phoenix tutorials going through how to set up npm and I know that’s no longer needed as of 1.6, and I get scared of getting tripped up by all these little changes. But I guess that’s silly when I’m still trying just to get an overview of the framework as a whole. I’ll try reading the book before getting back to Pragmatic Studio’s LiveView course so I can do it with a more solid base :slight_smile: I’ll dig deeper into the Release/Deployment documentation you guys linked, too.

@msimonborg I’m glad to know it’s not as daunting as it seems! And thanks for the hint on deploying early/often. It would be pretty sad to get far into the project only for deployment not to work for some reason, haha.

I’ll definitely have a lot more doubts as I go along, but I’m feeling more confident knowing I can ask the community for help. Thanks again!! :heart:

3 Likes

The Phoenix guides are also very helpful and always up to date, you could start there and see if that’s enough to provide you with background for the LiveView course

3 Likes

I don’t know if you already used Docker before, but when I learned Phoenix and struggled with the Database and environment changes (from dev to prod specifically), learning Docker is what solved my problems.

If you want, I wrote a guide a few weeks ago to have a compose stack running smoothly in production.
Link : Proposition for an official Docker Compose Guide

It is down to the fundamentals which make it imperfect (cf comments), but it works, and may be helpful to you, who should be taking it slowly.

1 Like