Forcing generated directories into Umbrella apps (split thread)

They might be but you keep forgetting people want drop-in works-out-of-the-box tools and not an engineering PhD exam when working.

If I am to embark on such an adventure I’ll make doubly and triply sure that the experience is near-seamless, easy and quick. I have no intention of introducing a spaghetti monster that makes people’s lives harder. If that were to ever happen I’ll just freeze the GitHub repo and put a huge banner on it: “WARNING! DO NOT USE!”.

Not sure I agree with your opinion on umbrellas though – or with anybody who opposes them really. Umbrellas, much like the proposed future private functions, are a communication-of-intent mechanisms and a way to separate bunch of source files from each other. Nobody claims they are magic. If you have this dependency graph: business_logic >> business_data >> data_app_using_postgresql then no technology in the world can stop a dedicated programmer from cutting out the middleman and thus not respect semantic boundaries.

Example from almost the start of my career: people have been known to reverse-engineer compiled C++ multiple-inheritance classes which were specially crafted to hide the implementation – and they did so quite successfully. What can you do? If they don’t get the message of “here be dragons, don’t touch” then let them tinker. But I ain’t ever accepting a single “bug report” from such people.

So to this day I can’t understand the objections against umbrellas. They simply give a small peace of mind and address a part of the single-responsibility-principe / separation-of-concerns patterns. Nobody says they do something more.

The thing is that they don’t do that, rather they combine lots of code into a single repository that shouldn’t be touching. Its far easier to just have each part be its own testable library then merge them at the end via the normal built-in elixir dependency system, which is all of a single line added to the mix.exs file each. o.O

Is it though? How about big corporate gardens? Also some CTOs cannot be convinced to self-host a private Hex repo. And they don’t want their stuff on the public Hex.

I believe the idea of a mono-repo being easier to work with – at least when you use GIT – is quite proven but am open to the other point of view.

Oh absolutely not, I don’t host any of my work dependencies on hex.pm, it would make no sense. They can either be git or path dependencies (I use git as I have a little hosted git server at work) or other options as well.

I find them quite significantly more difficult to work with overall. They encourage touching code across sections that should be touching code (this is why the dependency graph makes things wonderfully explicit and clear), it’s more code to go through at once (easier to ignore little things when there is just sooooo much to deal with, compared to nice fully compartmentalized libraries), it’s easier to PR in to and have the CI run tests on it, etc… etc… Mono-repos are an absolute horror after having to deal with far too many at my old jobs at such ‘big corporate’ places, the amount of time changing something innocuous and private breaking ‘others’ stuff was utterly astounding! o.O

This is not an argument against mono-repos though. It’s an argument against incompetent programmers. Shall we ban selling bread because people can eat so much of it as to get diabetes?

I guess we can agree to disagree here since our experiences seem to differ drastically. In the past, when having to modify a corporate private library and then 4 other internal GIT repos that depend on it, it’s not exactly an atomic operation. I warned two other team leads, they were like “got you, sure, I’ll migrate the code” and then… take a guess. Tomorrow everybody is angry even though the entire process has been communicated crystal clear, with migration guide and timelines, and has been known for weeks.

If I was given the power to do the entire migration by myself, the whole thing would be over in 2 hours. Instead it dragged on for 12 or so days and everybody was mad at me except the CTO and the business leaders (who were quite aware why the migration had to be done and supported it). I was really bitter back then – what are they mad about? That they can’t be bothered to read and can’t plan if their life depended on it? Anyway.

We all have our filter bubbles. In my experience mono-repos bring dangers to the table but with proper code reviews, pre-commit hooks and CI with added value (not only checking if tests pass but also linting) the usage of a mono-repo can and has been tamed and used to reduce friction when bigger changes need done.

It’s more of like how it encourages a bad form. Like when someone programs in OOP they see the solution to everything as an object, a monorepo encourages cross-coding because all the code is ‘right there’ so “why not, it’s all together anyway”. Stringent coders can code well in any language at any place, but not everyone is, I’m certainly not, and to date I’ve not seen anyone that is in any corporate setting, so the few that are seem very very few from my point of view. By keeping things as distinct, small, easy to run and test libraries then you don’t have to worry about that as the boundaries are already set for you. Sure it may be an artificial limitation, but those aren’t bad, like static typing can help make average coders into good coders, proper boundaries can do the same, but monorepo’s encourage the opposite.

That is a perfect example of what I saw happen at my old job endlessly (it’s one of the reasons I didn’t want to stick around for a ninth year), except it was via a monorepo. Having distinct repo’s with static versions (I outlaw depending on master for example at my current work, the CI enforces that) prevents such breakages, and the main program that depends on all the parts ensures that everything is running a compatible version or it just won’t build, and thus can’t be deployed. It has removed soo much headache from the whole process. ^.^

Heh, I relate all too well. I so love my current job, the pay may be low but they let me do things right, and to date the 3 years here have not seen a single production breakage. ^.^

If the code reviewers are letting code in a single commit that cross an umbrella apps boundaries then they have failed in their job, and that is all that I saw happen (very rarely was there a commit that stayed within its bounds), although this was in Java at my last job rather than elixir/erlang.

The thing about bigger changes is that they aren’t versioned well, like an API change you may think you catch all instances of its usage everywhere but missing just one means you can suddenly get random production failures, and this all goes back to the scope of the code having to be looked at. By using a proper dependency tree then you know the specific libraries that call in to your changes, and you only have to look at those comparatively tiny codebases to fix and update then update the requisite version of the library it depends on in its dependency list (mix.exs/ant/etc…), with the CI running its tests and verification’s and all manner of things on each and every individual step with each commit being absolutely tiny and trivial in comparison to a suddenly massive PR of sweeping changes among a ton of parts.

In general I have learned, and this very much includes me, that code changes need to be kept minimal and segmented, even across the definitions and the users of it, otherwise it becomes amazingly simple to miss things, even by very well trained and knowledgeable people.

Not sure we’re arguing the same thing because in my eyes we agree on almost every point. It’s just that we disagree if a mono-repo is a good idea or not.

What you are arguing for looks to me like the discussion of “microservices make everything easy” when in fact they only invite additional complexity and make maintainers’ lives a nightmare.

I have noticed the same pattern when using several GIT repos for things that semantically belong together. There are CI configurations that can disallow people from one team to ever touch apps/business_logic or apps/data_app_using_postgresql as well.

But I believe our main disconnect is the team size. For years now I am not at all interested in working in big teams due to the horrific amount of red tape and drama this involves; to me that’s a distraction from the craft I love so I fight really hard to stay on small and focused teams. There, the policies that make working with a mono-repo a pleasure are very easy to communicate and use.

Ahh, I bet that’s it! I’ve had to work with team sizes at my last job that are way way larger than they ever should have been. My current job has like 4 people that have access, and they still put everything through me regardless. ^.^;

I can see mono-repo’s being fine for very very few programmers on the project, but honestly just keep them as different namespaces in the same thing then, no need for the umbrella overhead. Even for projects by myself I still like to pull the pieces out a lot because it forces me to keep the bounds that I know belong there so I don’t accidentally (and I will otherwise) cross those bounds unless they are enforced.