There’s an extra pattern here where you can explicitly implement in-application transactions using GenServer. This really helps in situations where you need to do multiple things, in atomic way, that is not limited to the database operations. In your example you could spawn a GenServer for each order, make sure it charges user only once, generate one PDF invoice etc. Just something I find very useful from time to time - rather than relying on database transactions that do not provide such locking mechanism easily, btw.
4 Likes
Generators
Personally I don’t think we should get rid of generators - they are a convenience far more than they are a learning tool Imo. I used them all the time in Rails, particularly for generating Models, Controllers and Migrations. I think I stopped using them as a learning tool (i.e to generate a Resource) after a few weeks.
But why not have a system where you can decide what kind of application you want to build and the generators reflect that? So on app creation a --simple flag for one (which would also be ideal as a learning tool) and then either have a default to reflect the direction you want to push people, or just more flags: --microservices etc? I think this would be an industry first 
and a way to cater to the needs of the most common approaches to app development in Elixir… while at the same time allow you to spearhead what the community thinks is the smartest way to develop applications*. There could even be an --experimental flag 
*Something I think would elevate Elixir & Phoenix even higher above everything else out there.
Microservices - same database, or not?
With regards to Microservices using the same DB, from my understanding this is an option:
There are a few different ways to keep a service’s persistent data private. You do not need to provision a database server for each service. For example, if you are using a relational database then the options are:
- Private-tables-per-service – each service owns a set of tables that must only be accessed by that service
- Schema-per-service – each service has a database schema that’s private to that service
- Database-server-per-service – each service has it’s own database server.
With regards to why use separate DBs (which I actually like the sound of when I picture the kind of apps I want to create and the scale at which they might operate) here’s what doing that enforces:
Services must be loosely coupled so that they can be developed, deployed and scaled independently
Some business transactions must enforce invariants that span multiple services. For example, the Place Order use case must verify that a new Order will not exceed the customer’s credit limit. Other business transactions, must update data owned by multiple services.
Some business transactions need to query data that is owned by multiple services. For example, the View Available Credit use must query the Customer to find the creditLimit and Orders to calculate the total amount of the open orders.
Some queries must join data that is owned by multiple services. For example, finding customers in a particular region and their recent orders requires a join between customers and orders.
Databases must sometimes be replicated and sharded in order to scale. See the Scale Cube.
Different services have different data storage requirements. For some services, a relational database is the best choice. Other services might need a NoSQL database such as MongoDB, which is good at storing complex, unstructured data, or Neo4J, which is designed to efficiently store and query graph data.
Source: http://microservices.io/patterns/data/database-per-service.html
5 Likes
The problem here is that someone would have to maintain those. And, as with Phoenix 1.3 rcs, you can see that the generators get updated last. It’s extra work for the team.
There is actually nothing stopping you or me from creating extra generators, that would not be part of Phoenix core.
2 Likes
Someone thinks that is a problem to start with context modules as Phoenix 1.3 encourages and as the application grows giving a better understanding of what modules needs to became full OTP applications, split it later? I think that start small and growing the application design is the right way, instead of over engineering the solutions just from beginning.
6 Likes
Well said, in general, but I’d like to focus in on this part a bit. What you describe there is the “web” part of the overall system (aside: we really should start thinking more in systems than applications in general).
I would argue that iex represents another user interface into the system. The user in that case is the maintenance programmer (including the original developer in a few months). I want to be able to test out some behavior easily. For example, suppose I want to create a new user for this purpose. It should be something simple like user = Accounts.create_user(%{username: "testuser", password: "secret"}). I should not have to figure out the exact params and simulate a call into the Endpoint just to create a new user from iex. This can be generalized to any other core behavior of the system.
Aim to make it easy to use your system from iex, and contexts become natural. Whether the context lives in one beam application or another becomes more an issue of code organization and keeping deployment options open. Getting the logical separation in place is the primary goal.
11 Likes
That’s very true - but I don’t think Dave, or anyone else is suggesting we can’t do what we want in Elixir ourselves (on the contrary, that Elixir actually makes it easier to do these extremely cool things!) but more that we have an opportunity, as a community (and as language and framework creators in the case of José, Chris and the core developers) that we can spearhead outstanding ‘modern’ development practices.
I find that extremely appealing as a user of these technologies and I genuinely feel that it is something that can help Elixir and Phoenix stand out even more from the crowd. It’s also already in line with how José and Chris think too - because they already spearheaded this movement with their work on Contexts (etc)
Elixir and Phoenix could be the first language and framework where philosophy of and approaches towards software creation is as first class a citizen as is features.
1 Like
This is IMO one of the key points, not an aside I like to use the term “web-facing system” (instead of a web app or web server) to denote a system which just happens to provide its service over the web.
I would argue that iex represents another user interface into the system.
This is an excellent point! Yet another interface are tests. Having contexts should simplify testing of the core logic (because we don’t have to deal with conns, requests, sessions, etc).
+100
10 Likes
I am pretty sure Smalltalk people thought the same about it. Then Java people. Then Ruby people. Then JavaScript people (sic!)
1 Like
Did they though - in the same way? Frameworks probably yes but languages themselves? (Notable exception possibly Elm and I’d guess Java… but I don’t think I know anyone who likes Java 
)
Ruby and JS (I don’t know enough about the others) don’t push you towards system architecture with things like single responsibility or microservices etc.
I think one of the advantages we have in Elixir is that José is part of the Phoenix and Ecto core teams and I think this lends itself very nicely to the kind of cohesion that I feel would be needed to pull this off
Smalltalk. All the original GOF design patterns come from Smalltalk and it’s standard libraries. Ruby is basically Smalltalk so no difference here. JavaScript people think they invented reactive programming and what not. Erlang is a big one too. Lisp a great example too with powerful macros that allowed programmers to do crazy things with 5 lines of code (and a dozen of parentheses). Elixir while innovating is not the first to bring patterns and architecture on the table.
2 Likes
Do any of them push those architectures as part of their core? (I mean in the same way Elixir does with mix new)
What does mix new have to do with architecture?
So hmm. mix new does not do anything special except creating an established project structure in terms of files/directories etc. This is good to have standard like that but many other languages have something like that. Many others do not.
Compare that to Smalltalk, however, that does come with IDE/debugger/observer/image persistence/document browser/test runner thingy that does establish you way more infrastructure in place, and pushes way more conventions on you. Try Pharo Smalltalk, it is very educational experience.
I would disagree that it’s not doing anything special - it’s saying “Hey, look, this is a great way to structure your app - you don’t have to do it this way but we recommend it/think it would be easier/beneficial”. Hence my point that this could be capitalised on
I don’t know about Smalltalk, or Java or any of those other languages but Ruby and JS definitely do not have this ‘guidance’ built in (or if they do have no where near the same kind of acceptance or adoption).
I would argue Elixir is already doing some of what I was suggesting (championing development in certain directions) - demonstrated by concepts such as this, Umbrellas and Contexts.
In that case, let me attempt to make that argument
The vast majority of people, developers absolutely included, have a very low tolerance for diving into the deep end of a whole new world. People tend to do much better, and feel much better in the process, when they can incrementally move from A to Z rather than jumping over the other 24 letters in between. To the point that they will pass on methods that ask that of them.
No matter how much we would like this to be otherwise, it is a generalized truth about people.
The reason this matters comes from answering the question of why people come to Elixir and Phoenix. I don’t mean why they choose it over other options, but when they are already here … what are they trying to achieve?
I suggest that very very few are here to get a 5 year journey into a more ideal way of thinking about programming. I hope everyone does get that as a happy consequence of being here and using what are some phenomenal tools and getting to work with what is a great community of people. But that’s not what most show up for.
They want to achieve something very concrete, and often relatively trivial, today.
If we give them something that requires either too much learning (and let’s be realistic here: there’s already a lot to learn on the way in that door as it is!) or requires too much effort to put something useful together, they just won’t. We can point to Haskell and similar (great!) functional languages for the former, and Erlang (among others) for the latter as examples of these pitfalls.
We should also be realistic and realize that 90%+ of people using Phoenix will be making something small and simple. They don’t need complex or design perfection, they need a pragamatic tool to quickly get to their destination.
If they are successful with that, they may then move on to something more involved. This, I think, is a key point. How many times did the “we made a stupid prototype in half a day and basically got where we needed to be to make a full project out of it” meme occur in the Elixir Conference talks? I noticed it as a repeating story.
So it is key to have something simple, easy and “batteries included” that can get people walking (again, having to learn to walk in the land of FP, actors, and the rest in the first place). And that is why having something that provides all your pieces in one place is important to get started. Without it, people won’t start and few people will be there for the 5 year journey.
That said … most people don’t know they should be on that 5 year journey … perhaps until they are a year into it. (At least, that’s been my personal experience … ) But it is 100% worthwhile, and honestly the world of software development needs more of us to be on that journey.
So I do agree with you that there should be something better, something of a longer view. For bigger projects it is a requirement, for our personal development as writers of software it would be a blessing. But perhaps it could be delivered as a journey rather than an educational course.
What I mean is that we should perhaps be happy to have people walk in the door even if the anteroom isn’t where we want them to end up. Once they are there, there should be a nice and sensible path from that front door (i.e. Phoenix with Brunch and Ecto and all the other bits thrown together) to a better architecture, something like what you are advocating for.
And to me that is where the tools fall a bit short still: they give a great starting point, and you can go anywhere you want from there, but how amazing would it be if you could start with a simpler-to-grok all-in-one smallish application and evolve it from there, using the tools, to something increasingly more separated by concern and domain … where after some months (or years?) of working your project from a small little flower in the field to a giant rose garden, the tools have taken you down that path.
You can already do this on your own, but it is effort and you leave the tools behind as you go. To umbrella or not is a question we ask people to make at project creation time, and that’s probably not the best moment for most projects or people to make that decision. To separate data storage/retrieval from API access is a good idea, but it makes a hell of a lot more sense when you don’t have a small app (where it honestly makes very little sense: complexity for little realized benefit besides feeling good about your phyrric deisgn win) but now have something bigger. Understanding where the cut lines for smaller, focused applications (“microservices” or whatever other buzzwords one prefers ;)) is often easier for people to understand as they explore the problem space by doing.
Having tools that could help set up projects for later refactoring (and I do think Phoenix 1.3’s contexts could be a first step in that) and then help you automate those steps when they make sense for you could be absolutely empowering. It would be a way to take people on a 5 year journey of discovery, and pick for that given day in that given project where to land.
Put more simply: I don’t think we can realistically start most people down a 5 year path to glory. But we should be trying to lead them there. We can certainly start them somewhere simple, and if the tools are there for project refactoring they can accidentally stumble down the 5 year path.
15 Likes
I wonder if there’s a market for a scaffold/generator task or plug-in in mix that would delegate the last mile to generators in each lib/framework, so that @chrismccord does not need to write Ecto code inside Phoenix, leaving it to generators in Ecto.
I really resonate with this, and I feel that starting to learn Elixir/Phoenix has helped me realize that I want to be on this journey.
Big 
for this approach!
1 Like
I guess I sort of don’t like how this is framed. Elixir is not championing them, They are just using best practices.
Technically these things are not new and are available in many languages some of which I am sure does it even better.
Umbrellas are just a term which comes from (I think) on best practices how to develop many OTP applications in the same project.
Contexts - I can’t see this being anything other than giving good old fashioned module separation[1] a new name, perhaps to help and clarify . This is the way software has been develop in many languages over long time, probably with the exception of web frameworks (php | rails | django) which evolved from simpler scripts into the web world. I don’t know why the phoenix developers decided to give them a name.
When you say you want to capitalize on them it seems more from a marketing perspective and on boarding new developers. And it looks like a classic:
- Take old concepts and ideas
- Rename them
- Market them as a new and novel idea.
As demonstrated by real world these things work but from my point of view one potential problem with this is that it may appear dishonest unless it is very clear from where the ideas come.
- You should really try smalltalk, lisp and haskell if you haven’t already. They can change the way you look at software and will help you as a developer regardless which technology you end up with).
[1] = (i.e. modules should export as little as possible and have a clearly defined API, don’t leak implementation detail from API, the module should be loosely coupled and highly coherent)
2 Likes
Watch the first several minutes of my ElixirConf 2017 keynote where I address this
1 Like
