Separating the DB in an Umbrella project

[JoeZMar]

I am making my first umbrella project right now and I want to separate the database into it’s own application. I am trying to understand how to appropriately break up an umbrella project.

Here is what I am thinking so far:

My app has two web interfaces: user_web & admin_web

In addition I have an app to handle the backend logic: app

Right now my two web interfaces talk to the one backend logic app that I plan on connecting the rest of the micro-service apps to.

Here is where things get dicey for me. I’ve seen a few videos where the app was broken down even further into several micro-services. I haven’t found a decent/modern example that shows how small these micro-services should be. If I was adding authentication to my app I could create a micro-service for Authorizing users and have that as a separate app. I feel like I kind of understand this concept (or can at least figure it out) until I started to introduce the concept of separating the database.

From what I’ve read it seems like creating a separate application to serve the database logic is the way to go and have that app connected to the only database in the umbrella project. Does that mean when I introduce the ability to create users within my umbrella project the schema is stored within the database logic app or the app responsible for implementing the users?

I think if I understood the layer that manipulates the database application and how it i should implement the db api I can figure out how each piece of my app should fall together.

My app is eventually going to be very large including several services for different agencies and a few different native mobile applications so I really like the idea of having well thought out applications that are small and functional so they can be easily plugged into our future apps we create.

[maz]

Take a look at this thread:

New Series: Kubernetes-Native Phoenix Apps

Thank you! Yes, and as a matter of fact this is how mix phx.new --umbrella will construct your application for you. The super-brief explanation is that you want to have only one umbrella application directly responsible for your Ecto interactions with a given Repo. You have that umbrella app provide an abstraction layer used by your other applications within the umbrella, and they depend on that first one in their mix.exs. This is mostly just a slight variation on Phoenix Contexts which were …

I suggest just start with that. It’s easy to get into ‘analysis paralysis’

[cohawk]

The CaptainFact API might be a good example project for what you are trying to accomplish.

[mbuhot]

One of the great features of Ecto is that the schema structs don’t need to map directly to tables. This allows each umbrella app to manage its own schemas, and the shared DB app just defines the repo module, migrations and configuration.

[JoeZMar]

This was my original thought was that each app would hold its own schema (a great example is my users and admins have two separate schemas, but are used for the same table). If I created a Db app, what responsibilities should it have?

Right now here is how I am thinking about structuring my umbrella project:

user_interface ----                 |---- Auth
                   |- app_name -----|
admin_interface ---                 |--- (Additional Micro-services that create the app)
                                    |---- Elixir App hooked up with Ecto and a DB

Does the database application only contain functions within its Context to help the other microservice applications in the umbrella create records?

For example within the Database application a function in the context might look something like:

def create_record(table, schema, attrs) do
   ...
end

and each application has it’s own schemas and makes the request to the database app this way.

[mbuhot]

Not sure if create_record is a great example, since there’s usually Changeset validations associated with record creation.

But other generic helpers like adding Pagination clauses to a Query, or multi-tenancy concerns might belong in a DB application.

I’d avoid wrapping the Repo API, since a Repo is already a fine DB abstraction layer.

[JoeZMar]

Okay thanks for the help. This answers my question because I hadn’t thought of the possible helper functions that a db app could hold.

[zubairshokh]

Did you go with the single Repo or multiple Repo(each app with its own repo) structure.

@mbuhot what is better in this situation. a single repo or the multiple repo? if the structure is decoupled already.

[mbuhot]

I’d start with one Repo then split it into multiple when you want to isolate access to the tables using postgresql Schemas (aka namespaces) and separate roles/ permissions for each Repo.