I’m working on a well-established Phoenix application that has grown quite a bit, and I now need to create a separate Phoenix app to act as a back office for managing the data and operations of the main app. The existing app has a lot of schemas and helper functions that I’d like to reuse in this new back office application to avoid duplicating code.
What I’m trying to achieve:
• Set up a new Phoenix app for back office functionality.
• Reuse existing schema files (and some helper functions) from the main app in this new app.
• I don’t want to create an umbrella app or move things into a standalone library.
• I’m considering using git sparse-checkout to manage only the parts of the codebase I need in the back office app, but I’m not sure if this is the best approach. I know that mix has support for it, however while testing my back office app wasn’t able to find modules defined in the other app.
We’ve done someting like this at work with a Ruby app. We used a git submodule for the common parts, but eventually realized that we were basically reimplementing a package manager using git.
If I was to do this again, I would first seriously consider keeping everything as one application. If that got ruled out, I would probably make a regular library from the common code, so that it would be managed the same way as all other libraries.
But then it’s important to be very careful about what this common code depends on. Our apps share some dependencies that are really only needed by one of them, but the submodule was pulled out ad hoc without anyone puting in the effort to really make it reusable, so it’s kind of a mess.
I don’t know if you want to introduce complexity from git into your codebase, I personally wouldn’t consider this as a viable option.
Depending on how your projects are organized, I would recommend to have the shared code separated in a library, but in the same repo (a monorepo that hosts both the projects and the library). In this way you can use the option of local dependency and not worry about maintaining separate variants for the library codebase and version them:
defp deps do
[
{:local_dependency, path: "path/to/local_dependency"}
]
end
If you must split it off, then +1 to path dependencies in a monorepo. You would end up with applications like app, app_back_office, app_common and app_web_common. Check out [workspace] for working with a monorepo.
You are more regular here than me, you are telling me you don’t remember the objective arguments against them?
TBF I don’t remember them but I’ve tried umbrellas, the whole package, from ideation to deployment and they had drawbacks I was not willing to live with. And other people expressed the same and made lists. I did not save those lists but again, there were good reasons.
Though in this case of this thread in particular, I don’t get the anti-library sentiment more.
From the experience I’ve had, umbrella projects bring in more problems than they solve. Starting from configuration handling to OOM issues when fetching dependencies for multiple umbrella projects.
The idea of umbrella projects is to have separated OTP apps and that is a hard requirement only for a very few specific projects.
More like “didn’t particularly pay attention to”. You can find folks who’ll decry the use of ANYTHING - even if statements! - and I wasted waaaaaaay too much time arguing with that sort in Rails-land.
I’ve seen an umbrella solve OP’s specific problem very efficiently - the system was one that coordinated delivery companies (+ drivers) and stores. The key problem the umbrella solved was keeping UIs for different audiences entirely separated; apps were roughly like:
core: giant ball of DB schemas + business logic
store_web: UI for stores to use to submit & track delivery requests
driver_web: UI for drivers to use when making deliveries. Mobile-first, designed to tolerate drivers losing network connectivity intermittenttly
dispatcher_web: UI for the dispatchers to manage their driver’s work and check status
We didn’t do anything fancy for deployment like splitting subparts (for instance, deploying core + store_web separately) but the built-in boundary checking means we could.
This setup really showed its benefit when Bigco asked us for a new interface that would let Bigco management admin all of their stores. It found a straightforward home as a new bigco_admin_web, with a separate asset pipeline so BigCo could have their logo plastered on everything.
Another nice momement was when we needed to add SAML as a login option, but only for stores. This involved “splitting” the login form to ask for email first, and then either prompt for password or redirect to the IdP. Since each audience has its own _web, it was straightforward to modify only the one for stores.
None of these are strictly “only umbrella could do this”; a team with sufficient organization and discipline could accomplish the same thing all wedged into one Phoenix app, the same way a team like that could make a well-factored Rails monolith. Here in the real world, IMO the umbrella helped us keep things organized despite the hectic schedule of a startup.
I think I need to provide more information. My goal was to find a quick and easy way to test if a back office app is feasible. I planned to launch it on fly.io without a public IP, making it accessible only via Wireguard. In the past, I’ve worked with various solutions you mentioned: monorepos, umbrella apps, and shared libraries. However, none of these solutions are simple enough for a quick Sunday afternoon hackathon. They all require structuring the code in a way that feels like premature optimization. And it is mind-boggling to me that something so simple is this hard in Elixir.
I currently have a proof of concept running with this in my mix.exs: {:app, github: "repo/app", sparse: "lib/shared", app: false, compile: false}. Then, I run Code.compile_file("deps/app/lib/shared/file.ex") to make it available.
If you want a quick and dirty way, can’t you just add an option to your main app to not start the application tree and just use it as a library from the thing you are currently building?
That I completely agree with, and that’s exactly the reason I stopped keeping records of my argumentation for or against X or Y; I trust my judgement and experience and if that means some semi-irrational bias here and there then that’s a deal I am willing to take because it saves me a lot of brain energy. But yeah, I can accept that in this case the bias might be unjustified. Open to hear arguments in the other direction – and you provided them. Thanks.
Maybe it’s worth I dug into the very limited set of former work code that I was allowed to keep by contract and for self-educational purposes only and see if I can remind myself why I hated umbrellas so much at one point.
If it’s for a sunday afternoon hackathron then copy/paste will likely just work.
The unit of composition for dependencies is not modules, but it’s (otp) applications. Therefore the tools (like mix) for working with dependencies are built for that model. This is not done for lolz as well, given that has always been the unit of how dependencies are resolved within the beam VM and releases (see e.g. *.app files in _build or start scripts in releases). Each app declares it’s dependencies, so if ecto e.g. depends on decimal the vm makes sure to start decimal before ecto.
E.g. if your shared/file.ex depends on ecto, but your new app doesn’t have ecto (or decimal) around things will just blow up. These kinds of dependencies are not tracked on a per module level.
As you’ve figured out – or by using my suggestion of copy/paste – you can always forego proper dependency tracking if you want to. I however wouldn’t call that a good idea nor something that should be encouraged or be made simpler to do. All the primitives to do that exist and can be used, but you want to actively opt in to the fact that you’re sideloading modules and not depending on a third party (otp) application.
After reading the answers I would suggest the same as @LostKobrakai
If this is just a hacky afternoon project, copy all necessary files and hack whatever you want. Just make sure to throw it away as soon as the project is signed off. I have made this mistake too often and continued to work on those projects.
Get the commitment, make sure you understand the requirements, plan accordingly and investigate feasible solutions. And only then start coding on a fresh new project or extend the main repo.
Thanks for the suggestions! I was able to reuse the schemas with my approach, but I’m facing issues with components since they’re more tied to the web layer. I’ve considered using umbrella apps as some of you suggested, and I’ve seen that Mix.Release supports releasing multiple apps, but I’m struggling to set it up properly. Do I need to move my app to an umbrella structure, or is it possible to have two Phoenix apps in one directory without it? I saw some very old example build by @wojtekmach, however he included a proxy app which I’m not so sure about.
You can have a Phoenix app that has multiple endpoints. I don’t think there’s anything “special” that you need to do that, just a bunch of configuration that you need to duplicate for the new endpoint.
If you have two endpoints then they’ll be running on two different ports. If you want to access them through the same port instead then you’d want to use something like main_proxy (like @cblavier mentioned).
