Thoughts on single context Phoenix apps

Phoenix Forum Chat / Discussions
,
1

Hey! I vaguely remember that someone (maybe @chrismccord?) mentioned that small Phoenix apps can get away with just a single context which is the app itself. This is exactly how I often build small experiments.

Here’s an example: https://github.com/wojtekmach/twittr/commits/master. Note, we have: Twittr.list_statuses(), Twittr.create_user() instead of traditional Twitter.Statuses.list_statuses() etc. Minor change, but I find it cleaner and more convenient to have everything in one file early on. When I don’t have a clear idea how to name the context or what put it there, I just put stuff into the “app-context” and treat it as a “staging” area. Over time, stuff emerges and I extract it out.

While it might be counter to spirit of “intentional design” (since there’s just one context there’s not much design there, other than I guess naming functions properly), it helps me from “design paranoia” where I’m blocked on naming things.

It works pretty well for me, I’m curious if this is something worth encouraging people to do more often, or seems counter-productive?

6 Likes
2

There’s no one size fits all. If this fits you, and your style of coding go and encourage that. But make others aware that it’s not the only true way. Damn, “this is the Rails way” made so much damage to Ruby projects that it’s painful to think of it.

Luckily, both Elixir and Erlang are more flexible. You cn use contexts, sure. I just call them namespaces / modules, and do not follow the convention at all. I’d normally namespace my services, sure, but I would not namespace either controllers (and templates, views) unless they’re nested in router. I would not namespace schemas either. Controllers and schemas map to routes and database tables, and does not make sense in my head to namespace them in contexts having some extra meaning.

I do like another abstraction that comes with OTP, which are the applications. And I tend to use them more heavily than the others I think. What I would normally do when starting a project would be to start with umbrella project and two apps in it. apps/ui and apps/backend. In ui I make another heresy of getting rid of the web subfolder and just putting everything straight to lib. I even skip the lib/ui. Everything in this project is scoped within Ui.* namespace, I don’t need a Ui.Web, nor extra directory. It’s hard to reach to the files with such long paths even with Ctrl+P. That’s okay, it’s my project and I decide.

Then I’d put stuff to app/backend that is generally a place for the business logic. If I feel like separating the some things more, I’d create apps/whatever in addition to backend. Usually I’d extract the schemas and abstract the database read/writes and queries. I’d make apps/storage and expose some interface to backend, which in turns exposes some interface to ui. It’s an uni-directional flow where ui has no idea of internal data structures and implementation of backend, which in turns has no idea about internal data structures (including schemas) of storage. Usually there’ll be a few more apps in apps/ folder. Logs handling, watchdogs, handling integration to external services and other infrastructure bits would find it’s own apps.

And then there’s the kicker in form of testing the layers in separation. Right now, I can test my Ui, stubbing out the backend's functions as needed with mox, and test my business logic, stubbing out the database read/writes.

So the above is the reason I like applications. I don’t mind if there’ll be dozen of them in apps/.

That’s fine. Sorry for lengthish reply and off topic, this been on my mind for some time and had to share :wink:

4 Likes
3

I don’t think so. You are simply supposed to use all of the information that you can have before starting to formulate your starting position. That means putting in some genuine effort beforehand into understanding the domain that your application is supposed to be solving some problem in. That being said you likely won’t have all the information that you need up front. Trying to divine all the necessary knowledge up front is what leads to analysis paralysis.

Small experiments by definition have a very limited domain and a very tight focus - so you don’t tend to separate things.

This means you are still in exploratory/discovery mode. There is nothing wrong with that. The problem is when people only explore things in code - possibly bypassing the (cheaper) opportunity to talk to people who are dealing with the problems today as that conversation may shed some light on the inherent structure and forces within the problem domain that may help inform your initial design.

And this is the part where many fail. Project hygiene requires that you don’t let separate concerns/responsibilities “cohabitate” in the same place for the sake of convenience as they will tend to “complect” leading to accidental complexity. Refactoring is essential to manage that complexity. Hence the mantra “refactor early, refactor often”.

The problem is that refactoring is real effort. During that time and effort there is no visible progress “from the outside”. So people put it off - taking on technical debt. The longer the refactoring is put off the greater the refactoring effort will be, especially as new features are built on top of the sub-optimal foundation accruing additional rework - and that is how the big ball of mud starts rolling and growing.

So it comes down to seeking out information that is available to you before you start and make a “plan” (the design). However this isn’t a “create a plan and stick to it” kind of situation but instead the planning process is about exploring (and being familiar with) all the available options and choosing the best path at that time.

When the plan (design) is implemented, typically more information comes to light, some options vanish and new ones pop up and the “optimal path” changes and the plan needs be adjusted accordingly (i.e. Refactoring time).

In my experience one of the hardest things to refactor are database schemas (especially when there is live data out there). So typically I like to keep “schema knowledge” out of the code so that it is easier to refactor in the future.

7 Likes
4

“While it might be counter to spirit of “intentional design” (since there’s just one context there’s not much design there, other than I guess naming functions properly), it helps me from “design paranoia” where I’m blocked on naming things.”

One good thng about contexts is that you are not forced into using them one way. I really get annoyed when programming purist try and pretend that their concerns and design patterns apply to every one and every situation. Its hubris. They don’t . You are probably fine. the better approach is to ask you some questions.

Are you developing an app that you know will be in service for years to come?. Then perhaps Perry above has a point. You salary is sure and the company is committed for years to it. They probably want you to take your time amd even obsess over seperation of concerns etc

If however you are a startup working on a proof of concept Perry’s point would not apply. You simply cannot afford to have “no visible progress from the outside”. When you obsess and lose investment or market opportunity then “hygiene” won’t matter a wit - the patient is dead.

I find myself in that category with my present projects . I am either working on my startups or working for companies and business who have tight deadlines that determine their future. I’ll let separate concerns/responsibilities “cohabitate” in the same place for conveience every day and the weekends if it gets my clients app to production where he can maximize an opportunity. design purity (whatever that is) be damned.

If like me you came to Elixir for its high concurrency thats even MORE the case if its a new idea startup. Sure Phoneix willperform great if you break out but the idea might just as well flop and fail to get the traffic you chose Elixir to handle. Keep your eye on the prize. get the app out the door and working because if it does take off there will be money to refactor till your hearts content but no investor or market is going to pour in the money because its design is “pure”.

If that app is secure, works and performs well and satisfies its users then do it with one context two contexts or whatever gets you there. You will be fine. Living projects can reorganize. dead ones can’t.

If its a hobby project then I’d probably explore contexts fully and try and build ones with more than one just for the learning experience

4 Likes
5

I also think that this might be a good idea, especially in the early phase of the project, when you are not sure how various parts of your system connect and interact with each other.

From what I’ve seen so far, starting the project with multiple contexts often leads to model-like contexts which all just have the same set of functions (list, get, create, update, delete) and correspond to the underlying schemas and database tables. Having one global context can help you understand what the application does instead of what the data looks like. It’s especially valid in Elixir when you can easily refactor by moving functions between the modules without having to worry about the state and side effects.

5 Likes
6

@MikeAnthony Cannot find anything to disagree with you. Practicism and current circumstances > formulas. Always.

That being said, I discovered that when I pursue even half purity while progressing even a time-constrained project, it often times unlocks my brain in ways I could not predict. If you use the wrong design then you are burdening your mind with the wrong idea and thinking about how to solve problems inside that becomes difficult with time.

But that only applies for mid-to-long term projects, namely from 4 months and on. If you do a lot of short-term contracting then the above definitely applies much less (if at all).

7

Yep, seen this in several languages, Elixir included. Then I am onboarded and I am just “WTF?” and ask the person who authored it what is the purpose this thing serves outside of just having a separate module with DB-specific code. Sadly often times people cannot explain because they simply follow recipes. :frowning:

Not sure if any programmer celebrity ever said this but ever since I work with FP languages, I religiously pursue minimum file count. Context modules, as everything else, must have a very good reason to exist before being created.

8

I personally like the idea of only using Phoenix as a communication interface and the actually logic/work is done in a separate application that is called from phoenix. I have an application I am working on currently that is a single phoenix application that has 7 “sub-applications” in its mix.ex file that actually handle all the logic/work.

1 Like
9

I am fully with @hubertlepicki here and it’s a shame we are not colleagues. We would work perfectly together.

  • Modules (or namespaces of them) capturing certain business purposes work perfectly for the human brain. It’s intuitive and it’s instantly recognizable if one is not hung up on a ton of conventions.
  • Library and framework artifacts make sense for namespacing only if it adds clarity, e.g. Ecto’s schemas should go in YourApp.Schema.User. It’s a trained brain trigger that this is a framework / gluing module. Depending on project complexity, naming like YourApp.Schema.RDBMS.User and YourApp.Schema.KV.Avatar could work as well.
  • App for every collection of modules that is semantically too different from the others. @hubertlepicki gave perfect examples: logging, monitoring, 3rd party APIs, database-specific code, infrastructure.
  • Entirely separately (in an app) should be the business logic. For example, cancelling an order entails changing its status, adding a timestamp when it was cancelled, marking it pending or cancelled in the fulfillment system, etc. So you sould have YourApp.Order.cancel_order/1 function that knows about the steps and calls other modules (most of the time storage-centric ones) but has no idea about the storage itself.

Additionally, minimalistic directory depth. Not sure if umbrellas allow that but I would shorten all paths of the kind apps/storage/lib/storage/module.ex to just apps/storage/lib/storage.ex. I don’t expect to have two apps inside an app that is already a part of an umbrella app so these default paths don’t make much sense to me. There could be a good reason for them though.

Another thing I would do is change lib/yourapp/application.ex to lib/yourapp_application.ex. I feel like the OTP wiring belongs on as a higher level as possible.

And finally, minimum amount of files. Deleting the default lib/yourapp.ex is one of the first things I always do when starting a project. If I really need such a central point in my code then the need will be already known in the future and the file will have good reasons to be recreated.

EDIT: Regarding the minimum amount of files, I also mix my GenServer wiring and the actual functions that do cast / call to it. Every GenServer is, generally speaking, a stateful helper and a gateway to side effects, so I see no reason to split it in two files.

10

That’s also exactly what I do. Be it a website (Phoenix), or a REST API, or GraphQL resovers, they all should just be wiring / boilerplate that connects to the real actions that reside elsewhere in your app forest. And those real actions should be storage-agnostic and endpoint-agnostic. They should not care if ther results are served through a JSON firehose, a total JSON request, XML objects, streamed to a GraphQL client, or whatever else.

1 Like
11

I love how Phoenix and Elixir offer you the flexibility to do things how you want. It’s awesome that we have at least three routes emerging as being so widely used that you can feel confident about using them, knowing that there will be others who can help you in making decisions or if you get stuck as the concepts won’t be alien to most people in the community.

I actually think this is one of Elixir’s killer features; there’s not just - like in some other languages or frameworks - a single “____'s way”, there are several! And several that have got the support of the community after having been discussed and identified as a great way to architecture and build certain kinds of apps.

Phoenix is so flexible that you can use it in several ways:

  • As a (albeit more majestic) monolith - so great from those coming from frameworks like Rails or for when you want to make a simple/small site or when you just want to get something up/a MVP, or even as @wojtekmach says in his OP, for use as a starting point.
  • The middle ground and the default Phoenix way - with Contexts. I would probably use this for sites that are not small or simple. This will probably turn out to be the most popular way to build Phoenix apps as most will be a great fit for them.
  • The Replaceable Component Architecture - either via Elixir’s Umbrellas or Dave’s way. I would do this for the largest/most ambitious or long-term apps. This basically takes Phoenix’s Contexts that little bit further.

It doesn’t stop there either - Phoenix is so flexible that you can even have multiple Phoenix layers in an Elixir app - imagine trying to do something like that in Ruby :lol:

Thank you Chris, José, Dave and everyone else who have helped make all this possible - I really do think it’s one of Elixir’s killer features :purple_heart::orange_heart::yellow_heart::hearts::green_heart::blue_heart::purple_heart:

1 Like
12

Agreed with @AstonJ . I have an aside to share.

This will never be either a positive nor a negative. It seriously depends on how this freedom is used. Take a look at Javascript’s supposed freedom – it led to a huge sprawl of libraries and frameworks which ultimately (and very logically) led to the NPM left-pad fiasco.

A sad fact of life is that most humans do not seem to be able to utilize their full freedom without introducing anarchy in the system, and without the ego being the first priority behind almost every action of any singular agent inside the system. This is why, if you ask any Eastern European, I could bet that 8 / 10 of us will tell you that some control and authoritarianism are actually good for the system as a whole, provided the “dictator” or “the ruling class” are benevolent (which is a stretch and has been historically proven as impossible in politics at least; seems to work well with the right people in tech though, the leading example being Linus Torvalds).

Back to the topic at hand:

Me too. But I want that flexibility only inside Phoenix and co. I don’t need that flexibility in the form of yet-another-framework. I personally would dislike seeing a lot of frameworks coming into being that try and do things that Phoenix already does quite well. We need to be productive. The eternal illusion of the “infinitely free choice” is holding the industry back. We want to ship and deliver, our employers’ systems are not playgrounds, we should stop pretending we are 19 and we should specifically stop pretending that programming is only a passion and not at all a job. I am not taking shots at any enthusiastic open-source maintainer here – sometimes the alternate tech serves really good to make you think about other possibilities and this cognitive ability expansion is priceless! – and I admire the people who use their personal time to try and move an area forward. But I for one will still not replace Phoenix or Plug and would rather struggle with them for a bit until next version fixes a problem, compared to fully jumping ship somewhere else and fragmenting the community’s efforts and energy.

The community needs unity. All tech communities need less fashion and ego. This is ultimately a job and we must pick not only the best tools for the job; sometimes we have to settle for picking the good enough tools for the job if it means we will make the job of the guy / girl after us easier and more pleasant.

Agreed! I will cheer them if I meet them in person (okay I met Jose once but it was 2 years ago) and will tell them they make the tech world a much better place.

And I prefer to stick to their work – Jose, Eric, Michal, Chris and everybody else who works on Elixir, Plug, Phoenix, Ecto, and even contributes to Erlang / OTP.

We need more unity.

1 Like
13

I genuinely feel it will be (a positive). Like you said, it depends on how we go about it :slight_smile:

If you look at the extremes - the ‘Rails’ way on the one hand, and the gazillion different ways in JS land - Elixir & Phoenix are opting for something far more sane, with the three I mentioned (and perhaps one or two other emerging ideas which seem worthy of exploration).

I don’t think we are going to see a lot of frameworks, purely because Chris, José and the Phoenix core team have been astute enough to make Phoenix flexible and suited to many different needs :smiley: It blew my mind when I discovered you can have more than one Phoenix layer in an Elixir app :icon_eek::103::101::007::003:

Waaaaaaaaat - we’ve got loads :lol:

I think part of the reason for this is how José and Chris solicit feedback and include the community in conversations and the decision making process. Some of the most popular threads on this forum are in the elixir-news and Phoenix News sections (hopefully soon in the Nerves News too!).

When I came to Elixir it was for the speed and scalability, but what I’ve stayed for is the ‘better way of doing things’ - a question I often asked in other languages. I was pleasantly surprised to find the closest thing to an answer here in Elixir :purple_heart:

1 Like
14

Oh I am not saying we have no unity. On the contrary, this is the most united and friendly community I have ever had to honor to belong in.

I am mostly saying that we must keep it that way. :049:

1 Like
15

Yes, and so far Elixir seems to be striking a good balance, with reasonable alternatives that deserve to exist, rather than a big pile of half-baked ideas that we have to wade through to find the best fit.

So very very true! (And they’re really smart, and they’ve built/attracted a good community…)

2 Likes