Well, Phoenix itself is totally dbless, just remove Ecto and .
The automagical state backup would be a nice thing to have though!! But I would prefer it at least a little bit explicit and not embedded to a framework, a new lib for that would be really interesting.
I didn’t try it, and I suspect it might be in early stage. But such “framework” would allow us to avoid many design issues such as persistence etc. This should be rock-solid base to build in, that will allow us to focus on building the application and not do infrastructure tasks like saving/restoring state, which is also error-prone.
Yeah! I share your thoughts. Unfortunately, a lot of the products people value nowadays are a stateful solution to problems that could be solved by stateless ones. I keep forcing myself to find stateless solutions to the problems I have, but unfortunately I mostly fail. Sometimes I even think state is really an inevitable evil for almost every solution.
IMO a lot of the RDBMS inertia also comes from the added value that SQL can bring – namely reports. Business people, legal teams and finance departments just LOVE reports.
Yeah but how will you do the various aggregations, means etc. that a professional SQL dev can do? GraphQL is just an interface; something below it has to actually create the report.
We are talking about state here. In my opinion if you have business data that needs to be persisted you should be using a SQL database. I also think that you should design and manage this outside of an application context. I.e use SQL scripts for DDL and migrations.
The business data is likely to outlive your application by a big margin. Having this completely separate from the app will make it much easier to maintain in the long run. Perhaps after 5 years you want to use a completely different framework. Your business data though is still the same and by having it managed outside of the application makes this transition much easier. You can also add multiple different applications to your data without having the design choices of one framework affect any other.
On the other hand. If the data is only there to help run the application and not a thing in itself then the database-less approach is interesting. This auxiliary data can be stored in many different ways and can be quite tightly coupled with your application as it is more there to help the application than being data you care about in the long run.
A third type of data would be ephemeral data which can be recreated at any time. This is usually stored in memory can can be recreated when needed.
Likely your application will contain all three and there may some overlap between the three. What I think is important is that business data and the auxiliary data is loosely coupled. On is the core of your business and the other is just there to help.
Object databases are great when you know the shape of every query you will ever run on your database. Otherwise, it can be very slow and laborious to write those reports, and many reports won’t be worth writing at all, when you could have done the same thing in 5 minutes in SQL. You can also run into major performance challenges when you simply need to group your data differently from how it is stored in an object database.
In most domains I’ve worked in RDBMS is the right choice for these reasons. Even specialized use cases where you can make a strong case for a different model, can turn out to be more trouble than it is worth in the long run.
I am not disputing the three kinds of data you are talking about but it’s my experience that most project members opt for a singular storage to avoid potential complexity. That’s not an invalid concern.
I guess we are starting to tread into the configuration storage again as a by-product of this discussion, though.
and that is fine. As long as they are somewhat logically separated and not coupled with the business data (i.e many foreign key constraints and other relationship between tables).
Yes, configuration also comes in many multiple layers. I’d say that the same separation works here too. I.e business configuration, auxiliary configuration needed to just run the software (ports, number of processes, etc), and ephemeral configuration (temporary log levels, tracing)
The reality is most commercial apps need some sorts of database, not files. Files-based storage were there way before relational database, files-based storage didn’t work for most business companies and corporations. That’s why folks like Boyce and Codd invented relational database, and C.J. Date preached about it. That’s why Oracle, Microsoft and IBM makes billions of dollars on it, because they help companies and corporations to do their business.
Yes, relational database is slow… We know it really well. Guys with 50 years experience know it, fresh-grads know it. That’s our job as engineer to tackle with it. Plenty of way to do performance optimisations.
In fact, the reality is that most sizable apps need more than one storage mechanism. Keeping everything in database may lead to disaster, keeping everything in files as well, not to mention keeping everything in memory.
If the commercial app is big and complicated, it usually requires some sort of relational database, some (possibly remote) file storage, something to record events quickly and something analytical, and possibly a persisted caching layer…
That’s something to remember when you try to build “database-less” application: you most likely will need database anyway at some point, do not exclude the possibility that it may be beneficial to save some stuff in database, and query it as well. I guess we have to be pragmatic about what we use.
Okay, well not always. Try implementing in-memory joins / mapreduce on your virtual actors to filter them out to some subset matching given criteria, and then compare the speed to your SQL query that’d do the same. Or implement a full text search and compare it with elasticsearch.
Generally SQL/database will likely to be quicker when you need to deal with querying large data sets, and in-memory solution will be quicker to access to individual ones.
I have thought that a database-less framework would be a good thing to have and for a lot of cases just having enough in memory replicas would be fine. you could perhaps have a follower process backing data up to a db to deal with catastrophic failure but in normal operation the DB simply gets no writes at all.
I am currently working on a project that experiments with such a solution. temporary erlang processes that die when a machine dies are replaced with entities that are transparently moved between nodes when their host node dies. These can still be thought of as actors but there are a few interesting difference between normal processes. i.e. don’t use self()
That is certainly a consideration for many cases and probably I’m first thinking about tackling the state that doesn’t live that long. In game state that needs to live for perhaps days but after the game has completed only results are needed
.
