Is it a good idea to create new database engine as Elixir library?

Yes, I know, but I need more knowledge to start real work on it. If someone have it and is interested in to implement core methods then feel free to send pw to me. I will help us much as I can. :smile:

Iā€™m a little confused about what youā€™re going for here. The topic is creating a new database engine, but the discussion seems to focus around the API and not any of the things that actually make building a database hard.

By way of analogy, if you were trying to invent a new kind of car, the focus would not be on the paint color and wheel size.

I think itā€™s awesome that youā€™re interested in this stuff. However I think that interest should be channeled into understanding what makes existing databases tick before even considering rolling your own.

DISCLAIMER: If you have an application where just serializing some data to binary and loading it on boot works, fine. That isnā€™t what I mean by ā€œrolling your own databaseā€.

If I were you, I would first try to read up some papers on database design. Then try to build some basic key value stores and probably look at the source code of the many open source databases (many popular databases are open source).

If you are planning to build a database which is better than the current databases in every way, you may be under estimating the effort required to build any decent database. You should set the right expectations on what is possible by a single person to build. However, if you want to build a niche database which does only a few things well it is entirely possible.

P.S Checkout pouchdb or lunr which are implemented in javascript, these should be easier to understand.

1 Like

@benwilson512 and @minhajuddin: Thx for tips and info about projects. As I wrote already I will read some books and source of other projects before start real work on it.

At start I want to just ask: What you think about this idea?
I expected that someone links to a Erlang database engine project in first response and question would be quickly solved, but I see @andre1sk is interested, so itā€™s not so bad idea.

After it I just want to ask for good info/sources - a good point to start. I think about looking a book that describes creating a database engine from scratch (like Linux from Scratch :smile:) .
Of course not with finally new database code, but with lots of descriptions about how it should work and the ways to do it + example small demo code (write only demo row to file or only perform binary search on that one row) - book for developers new in this topic (I mean the algorithms to read, save and binary search in database files). I want to understand how that engine works and after that I will at least try to make small database demo.

Simple:
KV: leveldb
SQL: Sqlite

Actually leveldb would make a great backend for a database too.

I think for such a use case pure Erlang/Elixir store as a lib would be perfect

1 Like