jallum
Bedrock - a scaleable, distributed key-value database with better-than-ACID guarantees
Hey folks ![]()
I’ve been building a distributed key-value store in Elixir/OTP called Bedrock. It implements FoundationDB’s architecture but with a twist - instead of running as separate database servers, the components run embedded within your application’s supervision tree.
It’s a key-value store with:
- ACID transactions with strict serializability
- MVCC with snapshot isolation (repeatable-read + read-your-writes)
- Write-ahead logs for durability + storage servers for serving reads
- Components that run as OTP processes inside your app
- Recovery orchestration using supervision trees
- Consensus over RAFT
The architecture follows FoundationDB’s model closely - resolvers, log servers, storage servers, commit proxies, sequencers, etc. The design is modular, so you can swap out or experiment with individual components without reworking the whole system. The interesting part is exploring what happens when these components run embedded in your application rather than as external services. Local reads can potentially happen at memory speed while still maintaining these strong consistency guarantees.
Current state: The core transaction pipeline is working. You can spin it up in a Livebook for experimentation or run it across a full cluster for real distributed transactions. It’s designed to scale from in-memory test instances, to local development through to production clusters. Recovery works when components fail, though there are still rough edges and optimization opportunities.
If anyone’s interested in distributed systems or has experience with FoundationDB, I’d love to get your thoughts on the approach. The modular design makes it pretty easy to experiment with different implementations if you want to try out ideas.
GitHub:
https://github.com/jallum/bedrock
Happy to answer questions or hear what you’d like to use it for. PRs are always welcome!
Most Liked
garrison
Hey, someone else has been doing this?! Looks like beautiful work! I’ll have to spend some time looking through the code later.
So glad to see FDB’s architectural influence growing. The design is just so good, and it maps really well onto the BEAM; probably because they FDB guys were somewhat inspired by Erlang/Actor model.
jallum
Bedrock 0.2.0
Major Features:
- KeySelector API: Added comprehensive key selector support across storage, gateway, and repo layers with range query capabilities
- New Olivine Storage Engine: New B+ tree(ish) storage implementation with advanced indexing and persistence
- Range Reads: Full range-read support with key selectors, conflict detection and read-your-writes within transactions
- Enhanced Transaction Processing: Refactored commit proxy finalization with synchronous sequencer notification and exactly-once reply guarantees
Architecture Improvements:
- Reworked transaction tree balancing and processing
- Improved resolver capabilities with enhanced telemetry and tracing
- Director recovery system with capability-based retry mechanisms
- Shale log consistency fixes and transaction stream enhancements
Developer Experience:
- Updated documentation with transaction format guides and architecture deep-dives
- Subspaces and fast tuple packing/unpacking
- Enhanced example bank livebook with range query demonstrations
- Comprehensive test coverage for new features
This release represents a significant evolution of Bedrock’s transactional capabilities with improved performance, reliability, and developer ergonomics.
jallum
Hey folks ![]()
Quick update on Bedrock! Just released v0.3.0-rc4 with some major API improvements based on community feedback.
What changed:
The transaction API got much cleaner. Instead of threading a transaction handle everywhere:
# Before - explicit transaction threading
transaction(fn tx ->
put(tx, "user:123", user_data)
get(tx, "config:app")
end)
# After - implicit context + Ecto-style returns
{:ok, config} = transact(fn ->
put("user:123", user_data)
config = get("config:app")
{:ok, config}
end)
Subspaces are no more! The new Keyspace system replaces the old Subspace concept and features built-in encoding support:
# Works great with the directory layer
{:ok, app_dir} = Directory.create_or_open(["app"])
users = Keyspace.partition(app_dir, "users", key_encoding: Tuple, value_encoding: BERT)
# Familiar things are still here
alice_key = users |> Keyspace.pack({"alice", 42}) # still makes keys, but you get more control over *how*
# gets / puts / etc. use the Keyspace as context for understanding how to encode and decode your data
%{name: "John", age: 23} = Repo.get(users, 42)
:ok = Repo.put(users, 42, %{name: "John", age: 23})
# Range operations will decode and values for you, too, if you want:
Repo.get_range(users) |> Enum.to_list()
[
{42, %{name: "John", age: 23}}
{63, %{name: "Jane", age: 25}}
]
Also simplified conflict management, improved error handling, and streamlined the range operations.
Why this matters:
These changes make Bedrock much more ergonomic for daily use while keeping all the other goodness intact. The API now feels more “Elixir-native” rather than being a direct port of FoundationDB patterns.
The encoding system is particularly nice - you can plug in custom serialization for keys/values while the keyspace handles encoding automatically. Great for when you want structured data but it gets out of your way if want to manage binary keys and values directly. It’s made the class scheduling example very succinct.
This version focuses on the concrete improvements users will see, shows before/after code examples, and explains why the changes matter for real applications. The suggestions here have been phenomenal, and I’ve tried my best to integrate them all.
Always interested in more feedback, especially if you’ve worked with FoundationDB or other distributed stores!
Popular in Announcing
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #advent-of-code
- #elixirconf-us
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #performance









