I’d like to introduce IntegrateDB – https://hexdocs.pm/integratedb/intro.html – a database sharing toolkit project that I’ve made in Elixir.
I’d love for some feedback and any thoughts on it. The idea is inspired by projects like React, Tailwind and LiveView, in that it aims to boost developer productivity by deliberately coupling concerns that “received wisdom” holds should be kept separate.
Basically integrating services directly through the database is usually seen as a smell, so we all build APIs and loosely coupled SOAs. But in many cases this can be overkill and actually if you could integrate through the database you could eliminate layers and go much faster. Especially if you’re making LiveView apps where you don’t need an API and you have Ecto at your disposal …
The problem is that having multiple services using the same data is a recipe for disaster – unless you have some way of declaring and controlling data dependencies whilst enabling schema evolution. Which (as you may have guessed!) is exactly what IntegrateDB does
It’s a Phoenix-based web application that sits alongside a Postgres database. You interact with a JSON API to declare data dependencies and generate scoped db user credentials (which you then give to your applications for them to connect directly to the database — they don’t talk to IntegrateDB at all, it’s not a proxy or an extension).
It then adds a user defined function to your database which you can call from your migrations to validate that they don’t break any of your declared data dependencies. It also sets up and consumes a logical replication feed (hence being Postgres only for now), with the idea being to convert events that you register for into higher level notifications for your apps.
All of which is designed to allow you to break the rules and integrate straight through the database without shooting yourself in the foot.
I’d love to get some feedback on the project.