Game Programming & Databases: Lessons from SpaceTimeDB
Hey Elixeers!
I recently came across SpaceTimeDB, a fascinating distributed database being developed by Clockwork Labs for their upcoming MMORPG, BitCraft. What caught my eye is how it tackles familiar data, scale, and concurrency challenges in a way that could offer valuable lessons for the broader software world, even outside of gaming.
SpaceTimeDB blurs the lines between database and server. It allows developers to upload application logic directly into the database via “modules,” effectively eliminating the need for separate web or game servers. Clients connect directly to the database, executing logic within it. This approach promises to streamline development and potentially simplify deployment, as the entire application could be packaged as a single binary.
Intriguingly, SpaceTimeDB prioritizes speed and low latency over batch processing or OLAP workloads, making it well-suited for real-time applications like games, chat, and collaboration tools. This emphasis on real-time performance is achieved by holding all application state in memory and using a write-ahead-log (WAL) for persistence and recovery.
Here’s where it gets interesting for the Elixir community: The creator of SpaceTimeDB draws parallels between his project and Erlang in terms of its approach to concurrency and distributed systems.
This skips directly to the Erlang comparison:
Now, I’m curious to hear your thoughts:
Could SpaceTimeDB’s approach to integrating application logic into the database influence how we think about database interactions in Elixir?
What are the potential benefits and drawbacks of this tight coupling between application and database?
How might Elixir’s strengths in concurrency and fault tolerance complement or contrast with the design principles of SpaceTimeDB?
Do you see any opportunities to apply SpaceTimeDB’s focus on real-time performance in Elixir-based applications, within and outside of the gaming domain?
What are the potential benefits and drawbacks of this tight coupling between application and database?
That’s a topic I’m really interested in! Since discovering Mnesia, I’ve been contemplating building a fully-fledged, modern database on the BEAM that could run directly alongside an application. It’s definitely a project I plan to tackle one day.
In my view, the biggest hurdle lies in adoption and operational complexity. The current infrastructure ecosystem is heavily geared toward stateless applications for good reason—stateful systems are significantly more challenging to manage and operate. So, as soon as you mention a stateful app, it often raises eyebrows.
That said, I think BEAM’s concurrency model has shown itself capable of handling live updates while safely managing state (Mnesia is a great example of this).
The potential benefits are compelling, especially in terms of performance—keeping data within the same memory space enables faster I/O operations. Plus, simplifying development by eliminating translation layers between database and application formats could improve both speed and ease of development.
I’ll dig into this SpaceTimeDB more in-depth when I have the time. Thanks for sharing! (:
No worries. I haven’t had time to look into Mnesia, could you please give me your assessment of it, and its current role in the Elixir ecosystem. Is the technology dated in anyway or does it hold up well and is being underused currently? Your thoughts would be most welcome.
I’m definitely not an Mnesia expert, as I haven’t had the chance to use it in production yet, but I can certainly share a bit about it.
Mnesia is a distributed database that you can run alongside your BEAM application. Built in the early days of Erlang, it’s a remarkable piece of technology that can handle distributed storage at scale and is definitely in use in some serious production cases. A quick YouTube search highlights some real-world applications:
Mnesia does have a few pain points that show its age. A major one for me is the persistent disk size limitation. If you can’t fit all your data into memory, it becomes quite challenging to use Mnesia effectively.
I’m only saying this half in jest and I haven’t seen the video but… AS/400 with COBOL and DB/2 running on the same machine sounds a lot like this programming model (minus the distributed aspects)
