I run on 1.11, and that doesn’t solve the same problem, application boundaries are not the same as domain boundaries, and I believe that when most people reach for umbrellas, they want to solve the latter, not the former. There’s no reason to use umbrellas when just defining top level namespaces do the same, and don’t break tools like mix xref.
At least it’s not a problem the language itself is introducing, cross compatibility is a known problem in most software even nowadays.
There are also some other issues I’ve ran into, like having to force-compile the whole project because the order in which the sub applications were compiled somehow broke the whole umbrella, but I’d have to dig more into that bug again to provide a reproducible example(it had to do with running the project sometimes from root, sometimes from a sub app). I know for sure that wouldn’t have happened if the project wasn’t an umbrella, because it was precisely the “this module isn’t available because it’s in a different app” error.
That’s cool but doesn’t address the problem that running mix xref at the root of the umbrella basically doesn’t help you with anything that you would want when you normally run that command.
To me umbrellas are kind-of sort-of like the central Docker Compose / Kubernetes config file for a project that spans several microservices.
From what I’ve seen while working in Elixir companies, umbrellas are often liberally utilized as a stylistic / personal choice (usually by the CTO’s preferences) and are rarely used as they are technically meant to, leaving some of us to deal with the aftermath. Which is, again, utilizing mix xref to understand dependencies in a project better.
Having mix xref not work well in umbrellas leaves me with one less tool to enforce the single responsibility principle. Happily boundaries came along. It and mix xref – outside of umbrellas – have measurably improved the quality of my work (IMO).
Not saying everyone should be like me – but I personally will go out of my way to make a dynamic language a tad more predictable and easy to reason about upfront. Umbrellas stand directly in the way of that goal and that’s why I avoid them.
I have this non-umbrella app with two Phoenix endpoints, “api” and “web”: lib · trunk · Mikko Ahlroth / minisome · GitLab – And it really wasn’t any hassle at all. They’re both completely separate, yet in the same project. Now only one of them has frontend stuff, so setting up esbuild/webpack for two endpoints would have to be a custom job, but the Elixir side was easy to have separately.
A few months ago I started consulting on an umbrella project (about 10 apps). The first thing I did was deumbrellization of the project, which brought some benefits:
about 2k LOC less, due to removal of the repetitive boilerplate across subprojects
faster test and build times
simplified folder structure
better tooling support for simple projects
As an example of the last point, various paths that are printed by mix tasks (e.g. test IIRC) aren’t “clickable” in vscode (I couldn’t click to open the file in the editor), because the printed paths are missing the apps/myapp/ prefix. I found this extremely annoying and disruptive. This alone is for me the reason to avoid umbrella as much as possible.
Generally, my sentiment is that if the subapps are not deployed separately, the umbrella app doesn’t bring anything useful to the table, compared to single project + boundary. Even if boundary is not used, I find the benefits of umbrella don’t justify the issues mentioned above.
Thank you for sharing your settings. I will probably steal some ideas from it for my future projects. ^^
My concern is that if I use customized folder structure within a Phoenix app, I couldn’t take full adavantage of Phoenix generator anymore. The generators I think expect a given structure.
For example I don’t know how I can tell mix phx.gen.html to put generated controller-view-templates files sometimes in admin_ui folder and the other times in the default_ui folder.
Another concern is for example I would like to have different priv folder per UI for gettext translations instead of using a single folder with domain translations. Consider that my UIs don’t support the same set of locales and have their own domains of translation as well.
If there are a number of generators that don’t support umbrella project, I also feel like many can work out of the box without even being aware that the project is an umbrella. After all an umbrella app is just an app like another one in the first place.
I think it is a mater of preference but I’m really happy with umbrella as long as I keep the number of apps small and of course make sure all my apps depend only on the core app.
Thanks everyone for the discussion and sharing experiences. As I say, all the large projects I’ve worked on have been flat and have only experienced umbrellas on small (mostly personal) projects.
Much food for thought. I think the most significant issue with umbrellas reported here is around tooling, which is also an issue that I’ve also encountered. (There’s also that example and tutorials are written for flat structures and need to be tweaked for umbrellas). I really get how that could be a deal breaker.
I’m also inspired to take another look at boundary, which looks pretty useful. (I think the last time I looked at it things were in a very early state and I didn’t get round to investigating it further).
I don’t have the numbers anymore, but the difference was visible. IIRC, the main reason was that various mix tasks (e.g. compile, test) processed apps sequentially. OTOH, when everything is in the same project, compilation of modules and tests can run concurrently on the entire codebase.
Whether that’s a bug is debatable, and I’m also unsure where should this be tackled (in the editor or in mix). Coupled with the fact that I personally don’t find umbrellas particularly useful, I didn’t spend any additional time on this. Instead I focused my effort on removing the umbrella, which brought various other benefits, as I mentioned earlier
I remember this definitely being an issue. I’ve just tried it out as I happened to be picking up an old project again, and the cmd-click now works from the root of the umbrella. ¯\(ツ)/¯ so I think it’s something that’s somehow got fixed in VSCode (or Elixir LS).
Our xrefs sections became huge, so we had to ask why??
We used umbrella apps to determine what should be started as a part of Kubernetes deployments. But then I collapsed the umbrella app into a single app and used environment variables to determine what processes should be started. It was much more simple and we didn’t have all those dependencies (xref) problems.
Umbrellas can’t separate out your mix deps anyway… so they really just become about making separate releases. But disk space is cheap these days… why not have a single release and use env vars to determine what’s started?
After thinking more about this I realise that I could use multiple gettext backend modules in the same app and define different locations for their priv folder.
# Look for translations in my_app/admin_ui/priv/gettext instead of
use Gettext, otp_app: :my_app, priv: "admin_ui/priv/gettext"
I knew it is possible to define the location for the translations files but I didn’t think I can just define multiple modules such as MyAppWebGettext, MyAppAdminUiGettext etc.
I guess I could think of a way to have also different locations for assets.
And after reading all the posts in this thread, I guess I can be happier without my umbrella setup. Let’s embrace simplicity. ^^
For generators such as phx.gen.html I would maybe create a sample project with the good namespace and generate the files inside that project, then I would copy and paste them in the desired folders of the project that I customized the folder structure.
Definitely I will give a try to thoses ideas in my next project to see how easier the things would become.
There’s downsides to that as well though, as (only) runtime conditional starting of things still means you ship all the code everywhere. That might be a tradeoff, which can be made or might not. But which applications are put into releases can be customized in plain mix apps just as much as that can be done for umbrellas.
I also like umbrella projects, and have used it for nearly all production projects. My working environment is startups with very rapid prototyping. I haven’t had issues with keeping the projects DRY and the projects have always felt fast to build and deploy.
This is one example of an umbrella structure based on real world umbrella projects I’ve helped build:
This is the meaty main app containing all business logic
Phoenix app for API
Handling the API communication over HTTP/WebSockets and manages sessions
Phoenix app for landing pages
Phoenix app for internal admin
Plug app acting as reverse proxy
Infrastructure required traffic had to be served on a single port in the docker container
Also handles requests to non-elixir web apps running in the container
The reason I always went with umbrella was that it seemed to give me the best balance between monolith and microservices. It helped me abstracting interfaces and concerns in a way that felt very natural and easy to understand.
I’ve actually went the other way of what @sasajuric described. I used umbrella structure to help clean up a messy Phoenix app. It helped remove odd code paths, clean up dependencies, and assets.
Also very recently showcasing the umbrella structure helped me convince a microservice heavy (AWS lambda) team to give Elixir a try and to keep it monolithic for faster development
The comments here are thought provoking. Thanks for initiating this discussion @paulanthonywilson! I’ll have to try some different approaches as well.