What would you like to see from a native Elixir database?

[byu]

First, I preface this as I’m reading this thread as an intellectual exercise… I don’t give good odds that this may turn into a practical project since postgres (and other open source DBs) have so much momentum and world-wide support.

I do love to read garrison’s posts all DB related.

Random thoughts:

1. Intent?

Is the intent of just something better than mnesia? Lightweight (sqlite-like) native in BEAM? Or like a serious scaling RDBMS along the lines of FoundationDB?

2. Question of embedding

a writeup for Any way to embed riak_kv? · OpenRiak · Discussion #23 · GitHub would be enough for me

I can’t find the thread/link, but I think there was something where someone stated that they wouldn’t want to collocate Riak with the app because of “blast radius” boundary, and thus didn’t really build for the embedded case.

I personally would not want to have an app deployment (e.g. fly blue-green deployments) to elevate the risk of database issue, as deploying each to me are different concerns:

• DB deployment or upgrade is an infrastructure issue
• App deployment is application/businessFeatures level.

I still say that the idea of a BEAM-native DB (actually DX usable instead of mnesia) might be a good thing, especially if it takes advantage of BEAM’s uptime/fault capabilities for operational management (e.g. work towards zero-downtime upgrading/downgrading)?

3. Temporal Databases

The ability to “rewind” and see the state in the past.

I think that Datomic’s time travel works because it models (turtles all the way down) as statement tuples like RDF’s Subject-Predicate-Value-- They have Entity-Attribute-Value-Transaction. So, to me, Datomic’s unification of Domain Business Modeling with Data (Structural/Persistence) Modeling is what allows the application-developer to have something easily understandable/usable in the time travel.

Whereas, it appears to me that Relational DB implementations (e.g. Postgres) is all about the WAL that is persisting what I think CJ Date might have described as the “internal model”? An impedance mismatch between that and the conceptual level-- This is me (a non-db-dev or researcher) working on hazy memory from past reading.

Where I think I would find more use in Event Sourcing DB where I’m describing at the “Business/Domain” (not CRUD) changes, and then rebuild a custom business-driven view at a point in time IF I ever need to.

And if time travel were important in my use of RDBMS-- I have not regularly needed a native-db temporal time travel capability-- I would think we could model the time-components as distinct columns anyways, use SQL to give me point in time views?

4. SQL

Didn’t Andy Pavlo-- gangster by thy name? – say something to the effect of how so many people tried to reinvent or create alt-languages, but everything eventually will consolidate back into SQL, which he implied, just skip the pain and work with SQL?

---

love reading the above thread.

[garrison]

@AstonJ

Most importantly, stability/reliability and performance/speed

Stability and reliability (and correctness) are a given, totally agree there.

Performance is tricky because at the end of the day Elixir is slow. Competitive performance would definitely require some NIF abuse.

What level of performance would you deem necessary? Transactions/s (say 90/10 r/w split), read latency, and commit latency are good metrics.

@AstonJ

I think irrespective of which language a new DB was built on there really would need to be a compelling case to get people to switch from something like Postgres - what usp could an Elixir DB have

Well, for us, the convenience of Elixir integration. You could similarly ask what advantage Cachex has over Redis, and for me the answer is simply “I don’t have to deal with Redis”

But to echo what I said before, programming languages are turing complete. There is nothing that Elixir can do that is impossible with another language, so I don’t see the value in answering that question. Now from the perspective of the implementor, OTP and the BEAM give us a lot out of the box that many databases have to build from scratch, which can make things easier.

As for Postgres, well, it checks off almost none of the criteria I proposed. No native HA/replication (there is replication but not like what I mean), no scale-out, atrocious consistency model out of the box (though at least Postgres has serializable, except nobody uses it).

I have a lot of other problems with Postgres (and RDBMS in general) but if I get into it this thread will 100% get derailed and I’m trying to keep it focused on new ideas

[garrison]

@byu

First, I preface this as I’m reading this thread as an intellectual exercise… I don’t give good odds that this may turn into a practical project since postgres (and other open source DBs) have so much momentum and world-wide support

It was intended as an exercise, or at least a chance to do some idea farming!

With that said, I disagree. I think a lot of people are desperate to move past Postgres and its very (very) old architecture and associated limitations. Many were excited for Cockroach (like Postgres but distributed) until it was rugpulled. Likewise many were excited for Firebase (totally rethinking the database) until it was acquired and essentially killed. There are many such stories. The lesson being: Postgres survives not because it is good but because it is free (freedom).

(I mean, honestly, MongoDB was a raging dumpster fire that incinerated your data and had an atrocious hierarchical data model and it still achieved wide adoption.)

Now for me personally, I could not care less about momentum or adoption. I want something for me, so I’d be happy regardless! Still, I think there could be value. People are kinda starved for databases tbh.

@krisleech

The ability to “rewind” and see the state in the past.

Time travel is a fun one. You can implement time travel just using long-lived MVCC, and indeed this is necessary for long-lived read transactions anyway. But many databases have supported this (Postgres used to) and it never seemed to gain any serious adoption.

Datomic has a fascinating data model which I don’t know enough about to comment, but I’m glad it was brought up. I’ll throw it on the study pile and come back to it later. Would love to hear from anyone who has actually used it.

I know there is one company which is trying “firebase but it’s datomic’s model underneath” which is certainly an interesting bet.

There is also Dolt which is an RDBMS but underneath the entire thing is built on unicit btree variants (Prolly trees), which allows them not only to fork but to merge branches. They seem to have a very hard time explaining why this is actually useful, but at the same time they seem to have found many cases where it is actually very useful, and they’re just hard to explain.

[garrison]

@byu

Lightweight (sqlite-like) native in BEAM? Or like a serious scaling RDBMS along the lines of FoundationDB?

These are not mutually exclusive, and I would argue there is enormous benefit to being both. On the one hand, you could have a database which boots a new cluster in-memory for every unit test, and on the other hand you could have a database that scales out to 100 nodes. And they could be the same database, with the exact same API and implementation.

What’s cool is only on the BEAM does this proposition even make sense. Like I said, anywhere else “embedded+distributed” is an oxymoron.

@byu

personally would not want to have an app deployment (e.g. fly blue-green deployments) to elevate the risk of database issue

I have thought about this a lot. For a serious deployment you would certainly want a separate cluster with a security boundary (some sort of binary protocol, or maybe just HTTP/WebSockets?). Not only for upgrades (which is a great point) but also because as we all know the BEAM has no security boundaries at all. If you want multitenancy you will eventually need access controls.

But continuing the theme here, why can we not have both? There are absolutely times where you do want to deploy an embedded DB, for lightweight apps when you’re just starting out and so on. If you have one node it doesn’t matter if the database goes down! Tons of people use SQLite for this reason.

And then if your app hits it big you can scale out without changing anything other than your deployment topology. Pretty killer for adoption I think.

@byu

Didn’t Andy Pavlo-- gangster by thy name? – say something to the effect of how so many people tried to reinvent or create alt-languages, but everything eventually will consolidate back into SQL, which he implied, just skip the pain and work with SQL

Indeed, leading databaseologist Andy Pavlo has said this many times But what he means is not that SQL is good (Andy is after all a student of Stonebraker, who is, uh, not a fan), but that SQL is an inescapable industry standard and if you try to start a database company (particularly domain-specific) you will eventually be sherlocked by the RDBMS players that everyone is already using. In other words, the Vector DBs du jour are cooked. This is an entirely valid argument from a business perspective.

But, and I really cannot emphasize this enough, I Don’t Care.

Database startups are a joke anyway (see rugpulls).

[garrison]

See also this paper from Andy+Stonebraker and the original.

Both fantastic reads. But remember: just because something wins does not mean it’s good. I mean just look at JavaScript

[pawoc50825]

After reading this thread and posts #77 - #87 about gaps in the elixir ecosystem, (also ExESDB), I’m convinced that creation of a new state-of-the-art BEAM-native database is worth considering.

Not only it makes perfect sense from technical standpoint, but also I would be confident of it succeeding under @josevalim’s leadership after him being open to challenging projects like Gradual set-theoretic types, Tidewave, Numerical Elixir and Livebook.

You’ve done plenty of initial exploration. If I can, I suggest that you now focus on creating a proof of concept (if possible) and crafting a great technical presentation (for an IRL event) which could be a starting point for evaluating undertaking the most ambitious elixir project yet.

[garrison]

@pawoc50825

creation of a new state-of-the-art BEAM-native database

Nothing that has been discussed in this thread is state-of-the-art. The techniques needed to do these things (HA, scale-out, autoshard, strong consistency) have existed for about 20 years now and are widely deployed. The reason these things seem state-of-the-art to most developers is most developers are still using databases like Postgres which are, architecturally, 40-50 years old. Postgres in particular is literally over 40 years old!

Its storage engine is a bit newer, but still pre-dates the widespread availability of, ahem, multi-core processors. Which as you might imagine have had a rather strong influence on software architecture in the succeeding decades

@pawoc50825

but also I would be confident of it succeeding under @josevalim’s leadership

Lol I am definitely not trying to create more work for poor Jose.

I started this thread to collect my thoughts in one place and see if anyone else had any to share (and several have so far, thanks guys!). But it is no secret I have also been working on a database and I have indeed been sticking to the principles I laid out here. I would say I surpassed proof-of-concept several months ago.

I wouldn’t say “most ambitious”, though. Like I said, this is all old tech!

[pawoc50825]

Strong opinions, humbly held:

I take it as a given that BEAM is a great fit for a distributed database.
However, development model is as much, if not more, important.

I see this as all or nothing situation.

I envision this being not just a library but a crown jewel of elixir ecosystem. José providing leadership, erlef.org helping securing sponsors, Fly io donating engineering hours (they’re into local+distributed - LiteFS - Distributed SQLite · Fly Docs), etc.

I see the future where ElixirDB is known as the hottest distributed database that you can use standalone or like SpacetimeDB:

a database system that allows developers to build the entire application, including game logic and real-time state management, directly within the database itself

People joining the ecosystem at first for the db and gradually getting drawn into elixir programming.

First class database management/exploration panel in liveview for data scientists.

This is what I meant by ambitious.

[garrison]

@pawoc50825

However, development model is as much, if not more, important.

I see this as all or nothing situation.

Can you explain what you mean by this?

@pawoc50825

This is what I meant by ambitious.

I think it’s important to keep the scope in check here Ecosystems are built on abstractions, each layered over the last. If you tried to force coordination on all of the things you list here I think what you would end up with is, probably, nothing. These things evolve and grow naturally, and if you’re trying to plant a seed you have to remember that you’re starting from the bottom, not the top.

Which reminds me, there was something mentioned earlier in this thread that I wanted to highlight and never did.

@jstimps

The FDB team never got the chance to implement all the Data Layers on top of their storage engine because of the acquisition. It would be exciting to have an ecosystem where those ideas could carry on.

This exact realization is one which I had many months ago. See, the founders of FDB were obsessed with the concept of layers. Whenever I see any of the FDB guys talking about the database, even relatively recently, they always mention layers. It was a really good idea that never panned out.

The proposition is simple: provide a low-level distributed database “tool” and let people construct higher-level databases out of it. They solve the hard distributed problems, you solve the domain-specific problems. And eventually you have a sort of “hyper-database” which contains every other type of database within it (this is often referred to as a “multi-model” database) with atomic transactions across everything.

But it didn’t work out. Why? Well for one, FDB was a proprietary database. Who would waste their time building a database on top of something they don’t even own? I mean, Snowflake apparently, but they had the source in escrow. There was no opportunity for a community to build up around FDB because it was behind a paywall.

And then it was eventually acquired and (mercifully) open-sourced, but it was open source in name only. The documentation is poor and even now everything is quite arcane. There was again no real chance for a community to form. To this day they regularly dump enormous patches without so much as a paragraph in the PR explaining what they do (let alone actually documenting anything). And as a result, layers never “happened”, other than their own Record Layer.

The idea, though, is rock solid. With a good community (like this one!) maybe it could actually work out. Of course, somebody would have to plant that seed

[byu]

@garrison : After reading more of the thread, I agree that a native Elixir/Erlang/BEAM RDBMS could be a boon to the community (and open source in general), which would bolster the “Persistable Data” point in (Sasa Juric’s Server A vs Server B.

It could be a interesting value proposition (capability) to be able to:

1. Deploy a new app that is all Elixir/Erlang/BEAM without the need for any external dependency (ala postgres).
2. Then be able to just spin up new nodes (cluster) as data/traffic grows – app and data still colocated
3. And if things get really serious, then there is some sort of “Cell Division” process to separate into different clusters (e.g. separate database from web-serving) – whatever data/domain topology makes sense for the project – and do it smoothly.

Yes, I think we all agree that BEAM ecosystem has qualities that no-others have to even be able to support the above.

Side Note: Just mentioning Bryan Hunter + Waterpark as the project recently popped up again in my feeds, and it seems at least tangentially related to this thread.

@garrison

SQL is an inescapable industry standard

If DB is relational and I can use Ecto in the app, then I’m not as concerned about the flavor of human-usable query language (or dialect) is used to introspect the DB.

I would still want some sort of story about how devs/ops/dba can safely access the DB like I might do using pgAdmin (read-only) for ad-hoc queries to help debug/solve problems that may arise, or query the data for exploratory reporting-like queries. Whatever that may be. Ecto + LiveView? iex remote connection? Custom language?

@garrison

I think it’s important to keep the scope in check here Ecosystems are built on abstractions, each layered over the last

So is the idea to take FoundationDB DNA-- ala, logical-key-ordering, key-range partitioned and replicated (LSM Tree powered?) with dynamic range splitting and migrations–, clean/tidy design details for BEAM, and use it as the “direct storage engine” lowest level lego blocks, then to build relational on top of that?

So is the imagination that there would be two deliverables: (1) a distributed and embeddable BEAM-native K/V (not OpenRiak), that has modern improvements to better support the (2) RDBMS for our community?

And that to get adoption, in addition to Ecto integration, Oban support would be required by the large populace of phoenix projects.

The intangibles goals/constraints being:

• Improving the Elixir/Erlang/BEAM ecosystem.
• The go-to data storage for new-apps in our community, not necessarily trying to usurp postgres yet.
• Added benefit for freedom and protection against the DB company “rug pull” inevitability?
• Open the door for new tech to be built on top of this-- Maybe a BEAM Large Object Storage?

Could an initial approach be to (top-down) implement an RDBMS feasibility toy-prototype using FoundationDB as the underlying abstraction to validate the idea, then work to rebuild the abstraction (bottom up) replace FDB?

I say this is all “Cool Beans”, and I hope the seed gets planted.

[jstimps]

@byu

Could an initial approach be to (top-down) implement an RDBMS feasibility toy-prototype using FoundationDB as the underlying abstraction to validate the idea

I humbly submit the repos of foundationdb-beam to fit this need. Although I believe it’s more than a prototype and can support real applications even today.

The Ecto layer is called ecto_foundationdb. This implements some relational ideas on top of the KV. Not intending to go full RDBMS, though. We have indexes, migrations, large objects support, live queries (kinda sorta).

@byu

Then be able to just spin up new nodes (cluster) as data/traffic grows – app and data still colocated

This is the intention of ex_fdbmonitor: to use the BEAM dist to facilitate database clustering with minimal manual input.

@byu

some sort of story about how devs/ops/dba can safely access the DB like I might do using pgAdmin (read-only) for ad-hoc queries to help debug/solve problems that may arise, or query the data for exploratory reporting-like queries

efsql is meant to cover this, using @Schultzer 's SQL. efsql is very much a WIP.

Finally, it’s on top of a solid client library erlfdb which started as a fork from the Apache CouchDB project of the same name, so it has had more time to mature.

---

Weaknesses of this approach that stand to be solved by a new DB project as @garrison is discussing:

1. Not embedded
2. Non optimal value encoding
3. Difficult or impossible to use in escript (due to client NIF)

[pawoc50825]

@garrison

@pawoc50825

However, development model is as much, if not more, important.

I see this as all or nothing situation.

Can you explain what you mean by this?

@garrison

just because something wins does not mean it’s good. I mean just look at JavaScript

The inverse, just because something is good doesn’t mean it will succeed.

I agree with you that a new database wouldn’t succeed if the project is too centralized/closed. However, you may be overlooking the inverse, when most projects with great potential flame out without anyone noticing.

@garrison

These things evolve and grow naturally, and if you’re trying to plant a seed you have to remember that you’re starting from the bottom, not the top.

I believe that’s incorrect for a certain category of projects such like building a new internet browser engine, a new microkernel, and yes, a new database - can’t get to the top only by growing naturally from the bottom.

I argue that a first class support from elixir leadership would give long-term confidence for the community to seriously invest the time and effort required of such bold project.

This could even lead to innovation upstream in elixir-lang or erlang/OTP for the benefit of the new database, similarly how introducing set-theoretic types to elixir necessitated some brand new academic research in that field.

───────────

On the technical side, I don’t recall any real-time guarantees being discussed, something embedded hardware people would appreciate in an embedded database.
(btw. I wonder what’s the progress on Kry10)

[garrison]

@byu

I would still want some sort of story about how devs/ops/dba can safely access the DB like I might do using pgAdmin (read-only) for ad-hoc queries to help debug/solve problems that may arise, or query the data for exploratory reporting-like queries.

This is a good point, and is something I have not spent a lot of time thinking about yet. I’ll probably “cross that bridge when I come to it”, but it’s definitely something to keep in mind. There is a school of thought that an OLTP database should be “exported” to a larger OLAP data warehouse at regular intervals for the analysts to play with (in larger orgs, of course). I agree with that viewpoint to a degree, but for debugging or small use-cases something else will be needed. Maybe something in the direction of remote iex?

@byu

So is the idea to take FoundationDB DNA-- ala, logical-key-ordering, key-range partitioned and replicated (LSM Tree powered?) with dynamic range splitting and migrations–, clean/tidy design details for BEAM, and use it as the “direct storage engine” lowest level lego blocks, then to build relational on top of that?

I had originally intended to keep this thread more “general” but in hindsight that was somewhat silly. But yes, this is pretty much exactly my intention.

I do want to be clear that what I am working on is intended primarily for my own use and carries my own biases (as you may have noticed I am quite opinionated). I will share freely (soon!) as that’s what I believe in, but I will never be able to please everyone. The layer concept can help with that, though!

@byu

And that to get adoption, in addition to Ecto integration, Oban support would be required by the large populace of phoenix projects.

I had discussed this a bit with Jesse in the EctoFDB thread, but I think Ecto probably cannot be “saved” here. SQL is way too entrenched in Ecto - I know they tried to design it to be more general, but momentum simply dragged it to where it is now. I absolutely love Ecto, don’t get me wrong, but I will be starting from scratch.

At the end of the day ORM-like libraries like Ecto exist to make up for the inadequacies of SQL databases. The functionality Ecto provides should be designed into the database. I can get into this further if you want but I promise you I’ve thought about this a lot and there is simply no way. Huge swaths of Ecto would have to be redesigned to make what I want work. It’s just not possible.

I don’t know enough about Oban to know how coupled it is to RDBMS, but I bet it could be made to work if there was demand. It would be on them, though. Maybe some day! Way too early to speculate on stuff like that.

@byu

Maybe a BEAM Large Object Storage

Actually it’s funny that you mention this as I will probably end up doing this first. Postgres is okay but there are basically no object/blob stores that I actually like. I have some (app) features that I am holding off on until I have a solution in place. I’ve already got a good idea of the design, just need to finish the underlying KV first!

[garrison]

@byu

Could an initial approach be to (top-down) implement an RDBMS feasibility toy-prototype using FoundationDB as the underlying abstraction to validate the idea

Well first, as Jesse mentioned he has already done significant work in this area and is well beyond the toy stage (EctoFDB and co are seriously cool). Also, there is not so much need to “validate” my ideas here as few of them are actually novel. A lot of this has already been done by FDB and the record layer. There are things I want to change, but they’re not fundamental.

But there is actually a deeper reason I explicitly chose not to do this (build on FDB or another existing DB).

@jstimps

Weaknesses of this approach that stand to be solved by a new DB project

There is unfortunately a more fundamental weakness when you build on someone else’s abstraction. If for any reason you need to punch through that abstraction to make something work, you cannot. As it turns out there are a few things I want to do (incrementalization/live queries in particular) which cannot actually be done correctly with FDB.

So what do I do then, fork FDB? Well, actually I know the FDB codebase fairly well at this point (I have been studying it for a while now) and there is absolutely no way I could maintain it.

Fundamentally I need the freedom to make radical design decisions without being restrained by the underlying abstraction. If I own the KV layer then I can punch whatever holes in that abstraction that I please. A good example is predicate pushdown, for which FDB has fairly poor support. But if I want it, I can have it, because I own that code. That is very important to me, which is why I chose this path.

There is also one other big problem with FDB. The entire database was very cleverly designed for deterministic testing from the beginning, but they failed to design an abstraction which allowed layers to be simulation tested as well. This was a pretty big oversight, actually, and it cannot be easily corrected.

[garrison]

@pawoc50825

However, you may be overlooking the inverse, when most projects with great potential flame out without anyone noticing.

This is definitely not lost on me, which is why I have been careful to define my own success criterion as “build something which is useful to me”. This makes it considerably harder to fail

But with that said, the success of free software can be somewhat harder to predict. Unlike commercial projects which generally have a “window” of success (before they run out of money), free software simply continues to exist and deliver value as-is. Take Postgres, for example, which was written in the early 80s, eventually ended up being open-sourced in the 90s, and then lingered in relative obscurity as MySQL rocketed to the top before coming out of nowhere in the late 2010s to become the de facto hot database of the 2020s, 40 years later. It’s weird!

@pawoc50825

I believe that’s incorrect for a certain category of projects such like building a new internet browser engine

I can’t speak to microkernels, but I do know a fair amount about databases and web browsers and I don’t feel these are things which belong to the same category. A web browser is, fundamentally, something which has to meet an enormous spec in order to reach a baseline level of usefulness, where anything less is essentially worthless.

A database, by contrast, is a somewhat open-ended construct. A database can be a lot of different things, and the “success criteria” are something which can be arbitrarily defined for some use-cases.

I am fairly confident the criteria I defined in this thread can be achieved in <50kloc of code, maybe <100kloc down the road. By contrast, for a browser engine 7 if not 8 figures are table stakes. A browser is essentially a space program compared to what I want to do here

Anyway, if in the future when this stuff is more fleshed out others believe in the vision I would be very happy to see that! But I am absolutely not trying to push anything on anyone (or make anyone do work!).

@pawoc50825

On the technical side, I don’t recall any real-time guarantees being discussed, something embedded hardware people would appreciate in an embedded database.

That’s an interesting direction. The BEAM is not hard real-time but I know Nerves is a thing and many here are interested in that (I don’t know much about it). I also seem to recall that Nerves users were interested in CubDB (the pure-Elixir KV store) when it was announced, so maybe there is something here.

The key difference though is that what I’m interested in is a concurrent database, that is to say one with concurrency control and interactive transactions. This is something which is more useful in a highly concurrent environment (a web backend, say) rather than an embedded environment which I imagine to be mostly single-threaded. But again, Nerves exists, so maybe I don’t know what I’m talking about?

[Schultzer]

Are you using DETS under the hood?

[posilva]

Riak was what came to mind, used before (in Erlang) it has some very good properties to be built with a Beam language ( that’s why it was built). I really not see why we could have an Elixir database for community completeness. Unless it fixes a really problem, it’s just an hobby, a toy to have some fun.

[garrison]

@Schultzer

Are you using DETS under the hood?

No, dets is not suitable for something like this.

Firstly an ordered KV model is critical for implementing a relational DB in practice as you need to be able to do range scans on indexes. Of course there are some indexes which cannot be served by btrees either (spatial stuff, vector stuff) but it covers the vast majority. An un-ordered KV model (a hash table like dets) is only suitable for doing point lookups and would not provide enough functionality to be useful in practice for the things I’m interested in (application backends).

Dets also has a 2GB limit which is not near enough (1TB would be a more reasonable upper-bound). And I think it’s log-structured on disk in such a way that it has very slow recovery times, which is not acceptable here. Also the docs say it uses a lot of memory sometimes, whatever that means (I have never tested it).

Finally I probably wouldn’t trust it for correctness. Basically every embedded DB except SQLite loses data. Actually SQLite loses data too but I think the only database which is resilient to that one is Tigerbeetle, which is obviously not suitable here. So I will probably start with SQLite as a storage engine as it’s quite resilient and has mature Elixir bindings and then, down the road, write my own.

For the record, RocksDB is so widely deployed nowadays that I bet it’s fairly resilient as well, but I don’t know of any hard data on the topic. There are also several Rust embedded DBs in various stages of completion (ReDB, sled, and Fjall to name a few). Fjall is an LSM and probably the most promising (that is, it looks to be the most stable).

[garrison]

@posilva

Riak was what came to mind

Riak is architecturally a generation or two behind what I have in mind here. It comes from the Bigtable/Dynamo era (Riak is afaik a Dynamo clone (not DynamoDB)) when everyone was still throwing consistency guarantees to the wind in order to “scale”.

For reference, BigTable came to be in around 2004/5, Dynamo 2007, HBase 2009, Riak 2009, and then Percolator obsoleted that architecture in 2010. FoundationDB started with a Percolator-style timestamp architecture around that time (2010-ish) as well, though it was closed then.

Then Spanner and Calvin both came out around 2012 and sealed the deal, and by the mid-to-late 2010s Spanner derivatives like Cockroach (degenerate, lacking precise clocks) began to proliferate.

In other words, there are some seriously important developments which are entirely absent from Riak. Transactions, strong consistency, etc. Also, Riak does not have an ordered model from what I understand, which is very important (see my previous reply).

@posilva

Unless it fixes a really problem, it’s just an hobby, a toy to have some fun.

Sounds good to me!