Emily

Emily

Umbrella projects vs. stand alone apps as a dependency

I’m going through Dave Thomas’s video course, “Elixir For Programmers”. In the course he talks about decoupling & maintaining that decoupling.

He recommends seperate apps over an umbrella project.

What he says makes a lot of sense, but I believe there must have been a lot of thought going into the umbrella project design.

I don’t have enough programming experience to comment & would like to hear the counter arguments.

(For more context, a “rant” quote from Dave:)

A Personal Rant About Kitchen Sink Web Apps
When you create a new Phoenix application, you end up with over 7,000 files spread over more than 1,000 directories—a total of about 45Mb.

Now most of these are not part of Phoenix itself—they are third party dependencies. In fact, over 90% of this is JavaScript.

But, ignoring all this, we’re still faced with the potential for another kind of bloat. Although Phoenix 1.3 is better in this regard than its predecessors, it still tempts developers to include their application code alongside the Phoenix code, either directly or as a so-called umbrella app.

Part of this temptation is the directory structure itself. Why is there a separate web subdirectory under gallows if we aren’t supposed to put non-web code somewhere?

Part of the temptation is also the fact that the ecto database interface code is included by default in every new Phoenix app. A pure web interface has no need of a database—all persistence work should be performed in services.

I strongly suggest you fight the temptation to stick everything in the Phoenix app. As we’ve been building our hangman game, we’ve been working hard to build code into separate, well defined applications. We’ll continue to do this as we add Phoenix integration.

If, in some future app, you fall victim to the pressure to put all your code into a single Phoenix basket, then it isn’t a disaster. But, when you finish, look at the result and ask yourself if it might be better structured as a set of isolated components.

First 10 of 14 Posts! Switch mode

hubertlepicki

hubertlepicki

I think in most cases it’s a good starting point. And that’s it. There is no big philosophy behind it. I was inspecting the way umbrella projects work recently because we were investigating how to deploy stuff separately to different nodes and still maintain one repository. The umbrella project is actually getting into the way in such set up.

But for most apps, simpler cases, it provides a good abstraction on the project structure, and convenient way to start the whole system in development, compile, run tests and other mix tasks.

It’s a starting point that can be later on migrated to more custom project structure. We don’t have to always agree with @pragdave, although he’s usually right. I like to think we can think on our own :wink:

mkaszubowski

mkaszubowski

@hubertlepicki Can you elaborate a bit on how umbrella project is making it harder to deploy your applications separately?

In my experience there’s no much difference between keeping the applications as an umbrella or as completely different projects. If you keep respect the boundaries between the applications, I think the only difference in that umbrella project lets you run mix tasks for everything at once which is really convenient for running tests.

One problem we had with umbrellas is that because they are compiled into one directory, they allow you to call functions from other applications without explicitly declaring dependencies in the mix.exs file. This is a big problem because you may even create a circular dependencies if you aren’t careful. To prevent this, we decided to build and test every application separately. Fortunately, we can do this on CI.

I guess the key is to keep the applications separated and decoupled. The way you do it - may it be an umbrella project or a set of independent applications in separate repositories - is not much of a difference.

hubertlepicki

hubertlepicki

Funny enough, it’s not the deployment at all. Deployment is fine, it’s super flexible with distillery and ansible set up. Again, my favourite abstraction of OTP application comes in as a savoir, I can decide where to push and start what stuff easily.

It’s development that I’m concerned with. I’d like to start a local cluster, that somehow resembles the production cluster, maybe with less nodes but similar topology. And while in production I don’t have all code loading onto all nodes, I don’t want to load all code onto all my nodes in development. So that I avoid bugs where I rely on something being loaded / started while on production it won’t even be available.

This set up requires custom scripting for local development, and you lose some nice helpers from umbrella project, but in turn you get the flexibility.

mchean

mchean

anthonator

anthonator

I’m also interested in this topic. I’ve gone down the umbrella route and I think it’s great but I see it’s limitations. I’m curious how you would coordinate the applications using the standalone method. Seems like you’d need some sort of central registry you pass to each application.

I’d love to see an example repo using the standalone app approach.

AstonJ

AstonJ

There is a good discussion related to this topic here:

Personally I am a big fan of Dave’s approach - it just ‘makes sense’ to me.

I think Dave’s point is that if you go with an Umbrella app, it is still coupled, whereas his approach of totally independent apps, are, totally independent.

I’ve not heard him call it this, but I like to refer to it (or class it as) The Replaceable Component Architecture.

I can’t wait for his next course, which I hope ties in Ecto, deployment and goes more into sharing of or interacting with various data/components etc i.e. something more typical of a bigger real life app.

anthonator

anthonator

I agree I think Dave’s approach makes sense. I always separate my web and data layers using an umbrella app.

I’m more curious about the actual implementation. Umbrellas provide the ability to include apps as dependencies using the in_umbrella flag. I’m assuming with the standalone approach you’d have to do this yourself. I’m just not sure how you’d hook everything up.

kokolegorille

kokolegorille

Dave Thomas use a path dependency instead.

anthonator

anthonator

Let’s say you have two applications (A and B) in the web layer. You have a third application ( C ) in the data layer. If A and B both need to depend on C wouldn’t that start two instances of C? I would think it would make more sense to pass in a single instance of C rather than depend on it in that scenario. Seems like some sort of integration layer would be necessary. Am I thinking about this correctly?

kokolegorille

kokolegorille

The problem remains the same wether You choose in_umbrella, or path. You might also have cross dependencies.

If You follow Dave Thomas’s advice, that would be no ecto, no context…

I do use ecto and umbrella. But I wanted to mention how he does the integration, in his course, with the help of the path option.

Last Post!

michalmuskala

michalmuskala

Yup, config handling is different because of how it is generated. It’s nothing inherent, though, if you look at the config files (and the config_path: "../../config" setting in mix.exs), you’ll see you can easily change how it works to have a per-app config.

Where Next?

Trending in Discussions Top

AstonJ
As the title says, please share what you’ve been up to with Elixir. Whether that’s been learning it, looking into it, making stuff with i...
2976 91332 914
New
f0rest8
Hi everyone :waving_hand: Posting here to showcase and announce that Metamorphic is now officially live on a public-facing domain at htt...
New
byu
@chrismccord : I just saw the Extract AGENTS.md from Phoenix.new into phx.new generator commit to the phoenix project. My initial shotgu...
New
arcanemachine
I was working on an Ecto migration and I needed a timestamp. So, for the nth time, I looked up the different data types for timestamps, a...
New
AstonJ
Just a general thread to post chat/news/info relating to AI/ML stuff that may be relevant for Nx now or in the future. Got anything to sh...
New
type1fool
I just stumbled on a newly redesigned elixir-lang.org. :tada: It looks like @Software_Mansion did the work, and I think it is generally a...
New
juhalehtonen
There has been a thread to discuss the Stack Overflow Developer Survey on this forum every year since 2018, so here’s yet another one for...
New

Other Trending Topics Top

JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve. They are GUI (Emerge) and State management (S...
New
ausimian
Emily is an Elixir library that runs Nx computations on Apple’s MLX. Install it as the default Nx backend and Nx, defn, Axon, Nx.Serving,...
New
akoutmos
@hugobarauna and I (Alex Koutmos) have been hard at work on writing a book on Nerves that takes you from simply blinking LEDs to building...
New
zachdaniel
Introducing AshStorage! Attachment and file management that slots directly into your resources :smiling_face_with_sunglasses: I had hope...
New

We're in Beta

About us Mission Statement