I am having a bit of ecto-phobia… if this term catches on, please send royalties my way…
I have used rails in the past and played with ActiveRecord and now moving into Ecto for my Elixir projects. I have no doubts that the Ecto API is solid but I have always feel like I might be painting myself into a corner when I use Ecto and never sure about how locked down my database design needs to be before I venture into ecto-land… I have my design down on paper, but I am always worried about changing a column name later on and what impacts that will have on my code.
Is it possible to get to a point of no return with Ecto and similar tools?
Maybe I just need to be told that everythings “gonna be alright”! Happy new year all…
Less about Ecto but database design of a complex systems is a tough task overall. Modifying tables (e.g. adding, renaming, indexing columns) can get messy when you’re in the millions of rows I also always have the anxiety, that little 10% of my mind thinking “what if I had designed this all wrong?”
IMO there’s nothing you can do about it other than to always be ready for change and perhaps consult to a domain expert to make sure you get the terms right (therefore less likely to rename columns; I like to think DB column names are like an API; you’re expected to do some work if it changes).
I think the issue with “ORMs” in general is that they will always be less flexible vs underline store so you always end up with a mix of direct queries and ORM code. When it is up to me I’d rather use DB directly
Database schemas evolve as applications evolve, new tables, new columns, new constraints, drops, renames…
That is normal and to be expected. But think that if the schema changes, you’ll need to edit the code no matter what. Either Ecto, or hard-coded SQL, or stored procedures that depend on it. None of these things auto-update. Renaming a column is not just a SQL statement, it needs maintenance and coordination wherever the column is being used. It may even require multiple deploys if you may have old and new code cooexisting for a moment in a rolling start.
To me, something like Ecto (or AR) is a catalyzer. You know the SQL you need, you could write it by hand. But you instead leverage these tools to have the SQL robustly generated on your behalf, using Elixir code that is more understandable most of the times because of the higher-level API it is expressed in.
Then, occasionally the SQL you need is just too complicated, then you resort to raw SQL.
And just come at the point from a different direction: renaming columns is a good habit to get out of. As your application grows that type of operation becomes one that is difficult to deploy with zero downtime since the application code will All have to change the instant the column name is changed. It is more difficult to roll back in the event of a deployment issue as well.
There is a great ruby gem called https://github.com/ankane/strong_migrations/blob/master/README.md Strong Migrations that tries to protect against these things. I don’t know if there is an Elixir equivalent yet, but that readme is a short read that’s worth looking over regardless of language.
No disagreement here, but the original question seemed to ask if something about Ecto made doing so harder than it otherwise might be. I don’t think Ecto does.
It’s a subproduct of relying on ORMs vs RDBMS features if you rely on views or sps your application would not care about changes to to underlying schema
I just want to say, that there is the other side to that. If you learn how to read SQL and write performant SQL, then you can use it anywhere. But if you rely on ORM’s and the like, you have to learn new ORM and it’s DSL’s almost with every new language you use. Beside that if your application get complex enough, no tool like AR, or Ecto will help you, there is a time where you just have to dive into pure SQL. And if you need to write queries by hand in one place, you can IMHO just write it in whole app, this way you can clearly see, and extract to proper “layer” all the DB access stuff.
I’m not against Ecto or any orm just for the sake of it. I actually work mostly with them, and Ecto is really great. But SQL is really neat and very expressive language (compared to, fo example some nosql query languages ) and for me so far it was always way more readable than any ORM or similar tool.
In my job, when sometimes things need to be checked by hand, people tend to open rails console, and use AR to query database. I quickly came to conclusion, that doing raw SQL queries is so much faster and easier.
And in conclusion I find that, when it comes to more complex queries, SQL is actually way more readable than Elixir, or Ruby or any other language I’ve been writing in.
I prefer Ecto due to the type-safeness it brings to double-check me compared to raw SQL->Language conversions, otherwise I’d prefer straight SQL. If only the language understood SQL itself. ^.^;
Need at least TWO computers to make a non-stop
system
TWO computer is concurrent and distributed
It follows that you need more than one database to maintain high availability. So when it comes to database updates you take one down at a time. If database refactoring is a hurdle you are crippling yourself.
Granted choosing to change the database clearly has consequences and risks but so does choosingnot to change it - and those consequences tend to be cumulative.
Years ago people thought updating systems every two weeks was good enough, now daily or more frequent updates are not uncommon.
There are other ways to approach the same problem even with a single instance, without risking the downtime.
Instead of renaming a column, you create a new one with the new name.
Allow your new code to read from the old field if the new isn’t populated.
This ensures that in the event of a rollback on your deploy both the old and new code will work with the new state of your database. This is critical for a zero-downtime deployment.
If the deploy is successful:
Kick off a background job to populate the new field after a successful deployment.
On next deploy, remove the old field.
If you’ve got an application with a steady amount of traffic, that is the only safe way to avoid downtime. Two databases doesn’t actually do you any good because no matter what the old and new state of your application code have to work with the new state of the database. If you want to do that at the database level with a view that’s fine too…as long as the precaution is taken. The instant you rename a column, the old state of your application code no longer functions.
If you have two databases you can change it on one…but as soon as you make that change they are no longer in sync. As soon as their states no longer line up, you’re going to run into the exact same issue if new data is getting written to the different name.
You can do it, you just have to do it more steps than just using “rename”. In the early days of a project it’s not a big deal. When a project gets larger and people begin to rely on it, it’s reckless without proper precautions.
Thanks for the replies… this took a direction I wasn’t expecting… looks like I have some reading to do!
BTW - I have overcome my fear of commitment and started with Ecto. I think I was throwing myself off by using phx.gen.html when I should have been building my migrations and schemas from the ground up so that I understood it better… lesson learnt.
I also always have the anxiety, that little 10% of my mind thinking “what if I had designed this all wrong?”
) and for me so far it was always way more readable than any ORM or similar tool.
