We eventually migrated off the umbrella app, sticking to the officially recommended app structure, e.g. by having lib
and test
folders at the project root.
The whole process took under 2 weeks of work for a single person. During this time, I had to move around 3500 files. Probably 90% of this migration was automated, done via a bash script that moved the files around and did many replacements in the code using a combination of rg
and sed
commands.
rg
/ sed
were largely used to find and replace occurrences of various forms of Application.get_env()
, such as the following, for example:
Application.get_env(:my_app_1, :my_var)
:my_app_1 |> Application.get_env(:my_var)
:my_app_1
|> Application.get_env(:my_var)
But since it couldn’t catch all things, manual work was still required. And having a lot of tests helped too.
The most problematic part of migration, I think, was moving things related to the priv
folder. I initially decided the new priv
folder will have a structure like this:
priv/
app_1/
# stuff that previously lived in apps/app1/priv
app_2/
# stuff that previously lived in apps/app2/priv
app_3/
# stuff that previously lived in apps/app3/priv
Things mostly worked after this, but when they didn’t - it required a really hard manual searching of what got broken and why. Somehow this was much harder than when dealing with Application.get_env()
.
One of the bigger downsides of this migration is that git history is now polluted with a handful of “tectonic shift” commits, so searching how to make an original change in code using git blame
became harder (although not impossible, of course). The script I mentioned above was also making commits after significant changes, so we ended up with over 150+ commits for various groups of changes in the code, to make it at least a little bit easier to review.
Having the migration scripted also allowed this work to be done without interrupting or “freezing” the main branch: people could still commit their code to the main branch, and I could rebase my branch against main & every time re-run the script “from scratch”. My manual changes were extracted into patch files, so they could be rolled on top of changes done automatically every time too (although setting this up turned out a bit problematic and didn’t work smoothly).
Our team is quite large & works on many things at once. Due to this fact, after umbrella app removal was merged, folks needed to rebase all of their open PRs against this mammoth change. Surprisingly, rebasing went quite smoothly: nowadays, git ships with good defaults that notice file movements & know how to apply changes that were made to file back when it was in its original location.
When git
wasn’t able to figure out what to do, we used a few options:
-
find-renames
, more on it is here, and
-
merge.directoryRenames
to make certain kinds of file movements automatically,
So, git rebase
became “tuned” like this:
git -c merge.directoryRenames=true rebase staging --strategy-option=find-renames=70
But again, this tuning wasn’t needed in 99% of cases.
Overall, everyone is quite happy with this transition. I could only wish we’d done it much earlier.