The issue isn’t necessarily databases, they’re an implementation detail for persistence and query requirements. Databases solve the real problem of making sure your data is still there throughout restarts of the system. Databases do a lot and they’re hard. Want to learn more? The issue might instead be an architecture that couples directly to the database through misuse of CRUD semantics. So maybe it would help to discuss the misuse of CRUD?
Have a problem? Let’s add a field! Have a bigger problem? Let’s make a new table! Have a huge problem? Let’s make another service with it’s own database so it can have the same problems but somewhere else but with more coordination costs! Ha ha! Microservices.
The issue isn’t CRUD, but usually exposure of CRUD semantics as the interface rather than domain semantics.
With CRUD we have an entity and it gets created, updated, deleted or read. If that’s our interface what does the business logic? Maybe the CRUD is exposed over JSON via HTTP or even…GraphQL. What? I thought GraphQL was fancy and new and modern and solved all our problems? Well not if you just do CRUD over GraphQL… same issues can still surface because instead of exposing a service that does business things we’re exposing a service that does database things. Because our business logic isn’t involved in the interface to the consumers it means our consumers have to do the business logic! As a result, the business logic: the whole thing you built the app for, tends to get scattered all over the stack.
This isn’t always a bad thing. Sometimes exposing the CRUD is a great way to get moving fast when the real requirements of the problem are unclear. A reasonably designed table can operate like a slightly better spreadsheet. You monitor post-deployment the repeated efforts of some user to discover the real “business logic”. However this benefit of CRUD is a double-edged sword - what if you never notice that real requirement, or the developer leaves and finds a new job? Business has to business and changes will still be made, but will they be good changes?
So real issue is change. More fields? New picklist options? New tables? Migrations? For the database or the existing data? Both?! Good luck migrating that unvalidated text field you tossed in there because it was the quickest option. You’ll just need to write an ETL script that covers every single possible string your sales team could’ve thought up on the spot over the last year. Maybe we should’ve just used a spreadsheet…
So why does everyone build CRUD systems? Why do we have Phoenix generators for CRUD? The reason is because it’s extremely time efficient and if we know what we’re doing we can minimize most of the main issues. Instead of
create_user we make
register_user. Now our interface to potential downstream consumers is domain-driven rather than CRUD-driven! Problem solved. Mostly. We still have to handle change and data migrations over time. But, if the database model is well defined and we’re validating our data, that change doesn’t have to be that bad.
Finally, just think of all the engineering hours into frameworks, Postgres, etc over the decades. At this point we can run those Phoenix generators and do what would’ve taken teams of database administrators years ago. We’ve taken something that was hard and difficult, thrown hundreds of thousands of engineering hours at it, and now we have
Can we do the same for other architectures than CRUD?
From a technical perspective we could argue that CQRS and Event-sourcing is “simply better”. It scales! We get an audit log! We can torch that poorly designed read-model and make a new one. That’s awesome. But if it takes more developer time to implement a given CQRS/ES feature it will almost always lose out from the business perspective until it burns through significantly less hours than CRUD. We can run a large profitable business on a $10 / month server using free open source software. Hooray! Infrastructure is cheap! But the developer hours are going to cost you thousands per week. Have a fancy new architecture? Now you have to pay for fancy developers and they cost even more.
So this is a long winded way of saying that instead of investing engineering hours into the diminishing returns of improving CRUD we should be looking at making new and improved architectures as or more productive than CRUD is currently.