Questions about Umbrella apps

I put them in the repo/db app. But I don’t generally put a business logic in them either.

I take that to mean that you’re changeset functions also live in the app or context?

I am brand new to Elixir and haven’t even started learning Phoenix yet.
I just read about Umbrella Apps yesterday and like the idea of dividing an App by ‘context’ to prevent creating one big, messy, monolithic app.

Simple question: Can one ‘sub-app’ easily inter-communicate with another ‘sub-app’ (i.e. sub-app A gets information from sub-app B and stores information to sub-app C)?

Yes. See in Dependencies and umbrella projects:

defp deps do
  [{:kv, in_umbrella: true}]
end

…in mix.exs should be enough.

Yes but note that should only be calling in one direction. So B can depend on and call methods from C, but C cannot depend on or call methods from B. There are ways around it (such as registering a listener), but generally you want to have just one-way dependencies.

Thanks dimitarvp and axelson! Much appreciated

1 Like

The suggestion to store schemas and migrations in a Repo app is still not very clear to me. This means centralising validations and business logic in the Repo app, or moving the changesets and validation rules down to the context, or app, in this case. Is that what usually happens?

The migrations and Ecto schemas are (hopelessly) tightly coupled with the underlying datamodel of the RDBMS. So as such they will always belong to the Ecto repo.

This means centralising validations and business logic in the Repo app

No. Most of the confusion arises from the desire to leverage the Ecto.Changeset functionality for validation within the context (app) in concert with the Ecto schema structs. From a DDD purist perspective that is an anti-pattern.

A DDD repository would typically unmarshal the RDBMS data into data types defined by the context - rather than the context depending on data types defined by the repository. You are free to implement a repository this way but at the cost of easily using Ecto.Changeset for validation within the context. (Inside the repository you are free to use Ecto.Changeset as much as you like - but any data entering or leaving the repository is supposed to be valid domain data).

I suspect that is one of the beefs that Dave Thomas may have with Ecto.

2 Likes

I don’t think we should go all the way to the biggest heights of absurdity while chasing DDD purism as the Java’s “enterprise” stacks often do, though.

A couple of points about Ecto:

  1. The maintainers are considering splintering Ecto in two: core and SQL. Core would handle changesets, validations and overall managing changes in structures per operation / transaction. That’s not bound to SQL at all. You can do it in hobby projects while exploring append-only immutable DBs, for example. Ecto’s SQL-neutral parts will work just fine regardless.
  2. To me, Ecto is a fantastic glue and manager when working with pieces of data that change over time. No need to wrap that in an even higher-level abstraction which is gonna be specific for your project only and will only increase confusion while onboarding, and force people into practices which might or might not help their career prospects in the future.

I see nothing wrong in accepting Ecto as a hard dependency. In commercial projects we have to prioritize readability, predictability and minimal WTFs per minute, and most of the times making the rational and practical calls is quite enough.

EDIT: All that being said, I still don’t use Ecto outside of my data app (it lives inside an umbrella which handles all storage and validation; that’s how I usually structure my commercial Elixir apps). But I don’t see anything wrong to use Ecto’s SQL-neutral parts outside these bounds.

2 Likes

Dogmatic pursuit of “purity” for it’s own sake is rarely beneficial. However the idea behind Ecto’s schemas is that the shape of your data will likely closely mirror the shapes that already exist in your RDBMS’s datamodel - an idea that is rooted in the realm of CRUD applications.

However, as explored in the references in this post, in some “enterprise-y situations” the shape of the data used in the business context(s) may be some strange projection of the data as it actually exists within the RDBMS - at which point DB schema based structures and generic changeset functionality will be much less relevant - so the context defines the shape of the data that it needs to work with (not the repository).

Now if your business context has autonomy over its persisted data (i.e. data is not shared on the level of persistent storage) then there should be fewer reasons for divergent data shapes in persistent storage and the context.

4 Likes

That’s a problem only in legacy apps – like some heavily denormalized databases in 10+ year old systems where data duplication and the impossibility to maintain the stored procedures that held the whole thing together are a real threat. If your RDBMS model makes total sense for your business and code then the only problem would be possible worries about future-proofing (which are admittedly always 100% legitimate since none of us can accurately predict the future).

Oh, absolutely. That’s why I have User.get(user_id) and Accounts.get_customer(user_id) and Accounts.get_billable_profile(user_id) etc.; the contextual functions either transform the data structure or just return a subset of the struct’s keys. If you have the slight extra bandwidth you should IMO do that because it forces you to work with limited information and only include keys which are truly needed for the particular subsystem of the bigger business app. This is not about privacy or personally-identifiable information; this is about not leaking info where it should not be needed (the functions that operate on the billable profile should not care about the encrypted password or the last signed in IP or date, for example).

This however needs to be not overdone; instead of using other structs, most of the times I just use maps and pattern-match the required keys in the related functions. (Although it might still be a better idea to not only have a User struct but Customer and BillableProfile as well.)

1 Like

@dimitarvp @peerreynders can’t we create a “Repo” context?

As it stands, I’m not sure I see the value added of umbrella apps when I can split my app into contexts and namespace my web layer (as well as write a router for each namespace). In Phoenix 1.4, you have to import the router into the controller anyway.

What would be the gain of going a further step and, instead of a Repo context, split up the app into multiple ones?

Consider this app dependency graph: web -> domain -> storage.

The umbrella is a soft limiter in terms of dependencies and gets the message across that your web app shouldn’t touch your storage app; it should only go through the domain app modules and functions. At least for the several Elixir devs I worked with so far, it conveyed the message clear.

It’s obvious that there’s nothing stopping you – but when your web app has {:domain, in_umbrella: true} but does not have the same for the storage app, your teammates (and you) should understand not to cross the boundary and thus preserve some sanity and simpler dependency graphs in the project.

Additionally, I find umbrellas refreshing because they allow you to have several piles of code in different directories as opposed to one huge pile of unrelated code in the same directory. But that’s highly subjective of course and I am not trying to convince anyone that it’s “the one true way”.

1 Like

And to expand a little on what @dimitarvp said. The idea is that you should be able to swap web with something like console and have console -> domain -> storage and not have to touch any of the code in domain as web or console is “just an interface” to the business logic within domain – i.e. “(web|console) is not your application.”

1 Like

Right, but you have that with contexts.

Users.register(params)

In this case, you can call that from the web, the console, or whatever. It doesn’t matter here if the Users module is a context module or in an umbrella app. I understand the philosophical separation. But contexts seems to be “umbrella light,” without some of the added complexity. Unless I’m mistaken, in hard technical terms, I’m not sure I see a hard delimiter. Of course, the granular code organisation is nice, but I rather enjoy working with contexts.

Newbie here (and to Elixir/Phoenix), so take with a grain of salt.

Unless I’m mistaken, in hard technical terms, I’m not sure I see a hard delimiter.

I don’t think anyone is saying there is a hard delimiter? Communication of intent is a huge part of software engineering. An umbrella project communicates (and reifies) the concept of “these are all part of the thing we are building, but they have been separated at the application level for reasons x, y, and z”.

So yes, granular code organization is nice, but the complexity is [edit: sometimes] warranted for this and reasons mentioned by others above.

3 Likes

Except that it doesn’t “reify” anything. It provides no mechanism to do that. So if all we want to do is communicate intent, why not use module structure to do that? And leave apps for what they were originally designed for: packaging dependencies that have a separate deployment lifecycle from the main application.

Good point. I was thinking in terms of the umbrella giving you said module structure. In that sense, the utility is perhaps more pedagogical than prescriptive, with the additional benefit of being able to run mix tasks from the root (umbrella) project—which is very practical.

And leave apps for what they were originally designed for

Well, the umbrella is not really an Elixir app per se, if that’s what you meant. I’m also not sure in which situations it would make sense to add dependencies to the umbrella mix project itself, since those deps aren’t available to the inner apps.

Or are you suggesting an alternative to creating an umbrella project is to create an actual umbrella-like app?

In any case, as the Elixir guide says, you don’t have to drink the Kool-aid. Your mileage may vary, it depends, you do you, etc.

Right. If you are actually building a single application, such as a web application - I would use the module hierarchy to delimit separate logical concerns and give umbrellas a miss completely. I believe umbrellas have a place, but it is for managing multiple libraries in a mono-repo. If you actually are developing and releasing (even if its a private release) your own libraries as part of your application, then umbrellas are a great fit. But they don’t provide anything beyond the module system in terms of segmenting concerns in a single application.

1 Like

But in an umbrella, if you make circular dependencies or use modules that are not correctly required, it will bail out on you when building the release (although not when running it in mix), and this forces you to not have circular dependencies and having the correct deps tree structure in your app? So it might provide you with a better insight into the decomposability of your app down the line?

2 Likes

If thats the case then I’d agree, it is providing a real mechanism as long as you use distillery. Thanks for finally cluing me into that.

1 Like