Would parallel applications still run in the same BEAM, and if not, would you then create a client module and a server module, where by the client modules simply make :rpc calles to known servers? And the client and server code live in the same application, but when you need to use only the client you do not “start” the server?
Thanks,
Troy
I agree with this. This is how I develop my apps which are a bit larger. For smaller apps which are more web only I just put everything in one app (but of course still try to use the correct interfaces between my modules)
I tend to create an umbrella application with my application as one app and the phoenix web layer as another app.
I think it is great that Phoenix 1.3 is pushing towards this direction. For me it feels natural but I guess if you are used to Rails or Django which doesn’t usually have as clean interfaces perhaps it may look odd. I really like erlang’s -export directive as you really need to think on what functions you are exporting and it also gives a quick overview when you just open the files. Technically it is the same with def/defp but quickly look at a module to see what the interface is.
The cons to this approach I think is the same as the pros and cons of strong type systems (haskell, ocaml, rust). You need to think about your problem and problem domain in advance to get the right abstraction. This takes more time up front but saves you lots of time later. With dynamic languages you can get visible results much quicker but may end up with messy code that gets harder to refactor later.
I feel contexts are in the same problem domain as this. If you think first and get your contexts sorted out you wont be the first one out of the box but you will win the race
Could you provide a link? I remember reading about it or watching a video about it somewhere but I can’t find it.
Thank you everyone for taking part in the conversation 
Discussions like this are incredibly useful for developers like myself who are transitioning from the old ‘monolith’ way of doing things and are on a quest for something better.
I believe this (and finding it in the Elixir/Phoenix world) is actually a fantastic selling point - while many people may come for the speed, distribution, concurrency and fault tolerance I reckon they might stay because the eco-system pushes them towards good solid ‘modern’ approaches to building software that is resilient, less frustrating and scalable in every sense of the word. All the while making it fun (easy?) and interesting 
With that said, back to the topic. Although it’s very early for me to judge at present as I’ve still got 20% of the course to go through and we sadly won’t be using Ecto in the course (so I can’t yet fully picture a typical app) I am very excited by what I’ve seen so far. Much of what Dave has said seems to answer the burning question I often had when creating apps in other languages (that surely there is a better way of doing things).
Having said that, I can see a few challenges that might need to be overcome. For instance, if everything is created as separate services, and Phoenix should only be used for communicating via the web, then we might lose functionality or conveniences that Phoenix provides that we might want to use in the non-Phoenix part of our (overall) app - so those might need to be somehow moved from Phoenix or reworked in some way. The obvious one is validations (though I believe these are in Ecto - so maybe not an issue after all).
There’s probably more that I’m not aware of. José mentioned operational concerns and I would love to read about those too.
With regards to optimising for on-boarding new folks, I wonder if this could be worked around by having a --simple flag on app creation and where the generators reflect that (monolith?) ‘style’. This would explicitly let people know that it’s a learning tool, or one for quick prototyping or for creating the simplest of sites/apps. But perhaps more importantly it will spearhead a brave new direction, ‘the Elixir way’ of doing things. (I would actually be super excited by this! As if I could get any more excited about Elixir and Phoenix that is!)
The reason I think this is worth investigating was actually commented by Chris himself in his keynote, when he referred to @sasajuric’s feedback of where an expert team were somewhat led down a not ideal path because they thought the Phoenix way was what it was telling you to do via generators.
One final thing I would like to say is that it’s a very exciting time to be a developer! And an Elixir developer at that! I’m extremely happy to see that José, Chris and everyone in the Elixir, Phoenix and Ecto teams have this pursuit of reflection > perfection - and equally heartened to see that experienced members of the community are part of that transition, by feeling able to have conversations and inspire discussions like this.
But yeah, overall I agree with pragdave. And honestly, as I’ve mentioned before, I’m not even a fan of umbrella apps. Umbrella’s encourage a lot more interconnections between sections then what should really exist. I instead make my parts as lots of little dependencies that I then bring into a main app that does all the configuring and more. If I need a couple sections communicate between then I setup a channel between them at this point (not like a phoenix channel, but a set of behaviours in a sub project that both depend on, then setup their communication modules in the config). I find it keeps it very very clean and maintainable.
This precisely, this is what I do.
And this, 1.3 became less coupled to ecto, but I still do not like the monolithic apps that hold many (to use phoenix terms) contexts when a context should be its own standalone application/dependency.
I pass the Repo one of my dependencies should access into its config, which it then bakes in to their calls at build time. ^.^
That is really how it should be done I’d say, usually I pass the Repo module around as an argument, but currently I’m just using things like @Repo unquote(Application.get_env(:this_app, :repo) || throw ":this_app does not have :repo defined") in more places than I probably should. I should probably make a library to simplify a lot of this repetition too… >.>
Ditto, make lots of little applications and bring them in as dependencies.
Precisely this! They should be standalone dependencies as I’ve stated in that huge Contexts thread. ^.^;
They all run on the same beam. Even right now your beam instance is running probably a few dozen or more applications.
It’s possible that we are overthinking this and if I’ve got to choose between one of these approaches, I think Phoenix is getting it right by encouraging and enabling separation rather than forcing it.
Completely separate applications come with a lot of hidden complexity that doesn’t generally need to be introduced until you really require it. The whole microservice vs monolith discussion generally boils down to isolation of dependency trees and resource consumption at its root. If a monolith is built with an object oriented language, you end up with this horrifying blur of inheritance chaining that’s incredibly messy to untangle. If you go microservices from the start it’s good long term but you pay for it up front with potentially unwarranted complexity.
With the approach from Phoenix and Umbrellas you don’t have a tangled mess, thanks to the way Elixir is designed. You’ve got smaller parts in their own zones that are significantly easier to separate when you finally NEED to.
Learning curve and on-boarding are a huge deal for a language. Determining trade-offs that get people moving quickly in the right direction is a key and it’s one of those things that is hard because you’re always sacrificing something. It’s something programmers debate about constantly because business factors like turnover, training, learning curve and time to market are real factors that matter beyond ideal architecture. Ideal architecture seems a lot easier when those other factors aren’t included IMO.
In that regard and from a community standpoint, I really do think that Phoenix 1.3 has nailed that balance. Like anything else in programming, it’s always a question of trade-offs and hidden costs.
If we aim for architectural purity instead of that balance, we aren’t going to be much better off than the “OMG Benchmarks!” crowd. If productivity, learning curve, training time, etc aren’t a factor then why aren’t we just using Erlang?
Just to mention: When you designing contexts as separate umbrella app it forces you to think about deps and configs.
My comment which Chris cited should be regarded in the context of Contexts
In particular, I’m not at all convinced that we should enforce separate OTP apps. Contexts basically nudge us to move web-unrelated concerns to another function in another module. That’s a simple separation of concerns with a very little overhead. Separate apps lead to much more operational and mental overhead, and I’m not convinced this is a good starting option. Personally, I feel that with contexts, it should be fairly straightforward to split an app later on, if needed, as it should mostly boil down to moving some modules and some parts of config. So IMO contexts are a very good trade-off between approachability and some sane default decoupling.
I think the question is, WHY. Why do database calls need to be broken into other services. What is gained? What are the concrete advantages from a development, performance, and operational perspective? How does that weigh against the additional complexity introduced?
If the answer is “well, because. Because it’s not Rails”, then I don’t think that’s a sufficient answer. If the impetus is architectural purity (“that’s not how it should be done in Elixir!”), without regard for the implications in real projects and real development, I’m unconvinced.
I’m not saying there aren’t advantages to breaking everything into microservices. But most apps that have a data layer can have a data model that is by necessity a series of highly coupled dependencies. And efficiently querying that model often requires that those dependencies be maintained. Breaking that into microservices loses a lot, and again, I’d have to be very clearly convinced that what is gained makes up for it. I think there is so much talk about “X is good and Y is bad” without real world examples of why, and what the implications are then you lose what X gives you because Y is so great.
As others have said, I think that contexts do a good job of at least a first pass of separation of concerns (even though there may be data dependencies between contexts). I’m not sure that there’s much need to make all applications, even basic applications, require much more than that in terms of complexity.
At the end of the day, architectural purity loses to real world concerns every time, in my book. Unless that purity has real world advantages.
This is very good question, and the only question that matters. The answer is a cliche, but it’s “to build better, more maintainable software”.
I think contexts were introduced because some people (me included) complained about inadequate separation of business logic from the infrastructure provided by Phoenix. It is awesome that the team tried to solve the problem, I am not on the side of these that think it’s a good enough solution, however.
Like @pragdave, I strongly believe that the “web” or “ui” app should be merely an interface to the underlying application - or applications most likely. Now it’s the question of how thick or thin this layer should be.
Database access layer, in my humble opinion, does not belong to the web UI interface layer for sure. It is an implementation detail, that the business logic application would use internally and behind the scenes. Ideally, this can be done using Ecto schemas that are as dumb as possible. In my app, I do not put any validation-related logic there, the Ecto schemas define just schema and relationships between schemas. These Ecto schemas also would not leak to the UI directly.
The “core” application, that provides the business logic, most likely keeps the Ecto schemas internally. The responsibility is to respond to either queries or commands issued by user, and either return some stuff (for query) or update some stuff in database (command), sending e-mails, generating reports, updating external services in the process. You can have many “core” applications in fact, which is probably good practice in most cases except simple problem domains.
How does user interact with our “core” application? They submit forms. Or they click on a link that can be mapped to empty form submit. In general case they call some endpoint on the UI web app, passing 0 or more arguments along. This input needs to be validated, business logic checked against “core” application, and if such command is looking like something user can do, it’s been passed over to “core” application for execution. The question if the modules responsible for casting forms to commands, and initializing validation is responsibility of UI or back-end “core” app is open and will vary from case to case in my opinion.
So the above is the ideal breakdown in my humble opinion, for most real life cases. I do not believe, however, that all the work above has the same weight in terms of making the software maintainable, and predictable: the most important part is separating command/form validation from underlying schemas that persist / read things from database. By doing so you no longer think in terms of CRUD app, instead of commands issued by user. You can also avoid many permissions-related bugs in the code, such as mass-assignment related issues.
Now, how to achieve that? In Rails we used to import Virtus.model to our so called “form objects”. You can do exactly the same, with Ecto schemas that are not backed up by any table. This way you integrate well with default form builder, gettext, etc. If you have an API, that layer can be totally separated from Ecto library, and either use your own, or something like vex.
So while I appreciate effort to make separation of modules in Phoenix, I think it went slightly less important way. Separating form validation from actual schemas does seem like a more important task for me. And it’s true you can easily do it on your own, but it’s not what the generators teach users to do