Umbrellas are not more than a common folder structure for handling (starting/stoping) multiple otp apps as a single unit. A single otp app is still the most high level organizational structure within the erlang vm. If you want to depend on all your umbrella children either depend on all of them manually or create another child, which you depend on in new projects and which does then depend on the other parts of the umbrella.
This is actually why I don’t use umbrella’s. I just make everything as standalone dependencies (maybe just file: "../blah" dependencies in many cases) but then I can compose and recompose as I wish.