Contexts - a barrier too high for newbies?

OGod that sounds horrifying! I look forward to it! ^.^

Yeah this is exactly it, people assume Contexts are more than what they are when they are just normal separation of concerns.

Understatement of the century. ^.^;

/me often gets hobbled and frozen by just figuring out how to ‘name’ something, so usually ends up just giving things very generic names (look at overminddl1’s github for project names as an example)

I think that well sums up my feelings on why I did not like Concern yeah.

Hmm, Scope feels a lot better to me, would have to play with it to be sure or not, feels ‘incomplete’ still…

1 Like

I’d argue Rails became popular despite it’s complexity. It became popular because of strong community patient teachers showing a better way, not because of any sort of magic the generators provided. It became a go-to tool because we had the ability to point new developers to _Why, Ryan Bates, Saint Joe, José, Sandi Metz, and Avdi. (normally in that order).

It’s worth pointing out that my team was organizing code around things that looked a lot like contexts after a couple weeks working in Phoenix. It felt natural… especially once we started reading more on FP. Really that’s what I think is missing… we need our (non-pro-parts) RailsCasts and our POODR. I want the ability to point juniors to something that isn’t written in F#, Scala, or Scheme to learn FP concepts and architecture.

But nah - I reject the idea that contexts would be a stumbling block to learning the framework.

I wasn’t talking about the generators. Rails is magic.

To be fair, I don’t have much love for Phoenix approach to “contexts”. Personally I do not see this as something I will use, and I actually agree that I will not recommend this to be used by newbies either.

I do think that Elixir/Erlang/Phoenix already comes with better alternative in form of umbrella application / Elixir/Erlang applications. And, truth to be told, Phoenix 1.3 does improve support for umbrella applications in form of new generators phoenix.new.web or phoenix.new.ecto.

I currently use approach of having umbrella app containing of several applications: “base” or “core” where say Ecto is used, “web” or “user_interface” where user-facing UI sits, “admin_interface” or “backend” for UI to perform administrative tasks etc. It is already easy and convenient to create such applications, and they do have additional advantage over contexts that they can be easier built, deployed, restarted as units.

Contexts seem like a way to improve the code structure of application where you put everything into “web” application. Which is not something I want to do in first place :slight_smile:

2 Likes

//
Some posts regarding the OP’s intentions have been removed - generally we tend to give people the benefit of the doubt - but if you feel someone is a ‘bad actor’, please flag their post for a moderator to take a look (rather than tackle them yourselves) - this helps the thread from getting derailed. Thanks :023:
//

4 Likes

I also quite like to use umbrella applications instead.
But then again, for newbies just starting out with Elixir, that might actually be a little bit too much.

@josevalim, @chrismccord: Why did you decide to go with Contexts rather than umbrella applications by default? Was it the concern that going all-OTP right away might overwhelm new users?

To speak from my own experience, I frequently have multiple contexts that I want to be part of the same OTP supervision tree.

I have started Phoenix projects with as umbrella project in the past, and I think that for most projects it does not make a lot of sense to do this when starting out. Umbrellas add an extra layer of complexity (because of the separate OTP applications) that contexts do not have.

But this is definitely an interesting question, as I hadn’t consciously thought of why I did one or the other before! :smiley:

2 Likes

You should really watch the introductory talk by Chris - he explains you should really only consider separate OTP applications when the applications have independent storage. Contexts are intended to allow more cohesion / coupling than you’d want in those cases.

Speaking for myself I’m glad to see Contexts in the generators. I had found myself creating something similar on my own; I was a little dismayed when I first looked into Phoenix and saw all the database work done directly in a controller. Having it in the generator saves me a little work upfront.

4 Likes

That’s not quite true. You can use the Phoenix --umbrella flag and have two apps: one for web and one for your domain. So you can have all of your contexts in an app with no “web” parts.

Also note that, if all applications depend on the same data source, such as “base” or “core”, they cannot be deployed in isolation. Imagine you have new migrations and you deploy a new admin which requires those new migrations. Once the migrations run, you may break the “web interface”, which is not using the new data. So in this case umbrellas may segregate the code, but the operation is still coupled.

In any case, umbrellas definitely have their place and if you are comfortable with them, then surely go ahead. But remember they are somewhat orthogonal to contexts (which again, is just organizing code in modules) and definitely require more discipline.

3 Likes

To me different business applications in the same umbrella should have data isolation, otherwise they are still coupled at the operations level (see my previous comment). From that, it is easy to see that it would be overly complex if mix phx.gen.html generated a new app with its own database in an umbrella every time the generator is run with a new context argument.

Also, remember those concerns are quite orthogonal. Contexts are modules. And all applications will have multiple modules. If you want to use umbrellas, then I would have an app with all of your contexts and another for the web interface, which is exactly what the --umbrella flag does.

4 Likes

Of course they are still coupled, and of course in many cases you will deploy them at the same time. It’s nice to have functionality, however, to re-design a registration page as new umbrella app, deploy it to running system, feature toggle it or A/B test and get rid of it if works worse than previous one. It’s the stuff like this that makes this umbrealla-based architecture extremely appealing to me.

When it comes to contexts, it all seem to boil down to generators and what is namespaced in the context module and what not. I think good software developer would namespace their code in similar manner anyway. However, I do not see the reason to namespace modules that I consider part of application infrastructure and those are for me:

  1. Controllers, views and templates. I’m fine if these are flat layered under “Web” or whatever. It’s just the glue that takes parameters, forwards request to services layers and renders response.

  2. Database-mapped Ecto modules. I generally namespace it under “Db” or “QueryDb”, “WriteDb” etc. This is also just the glue that allows one to read and write data to database.

In this approach you have clear focus on what is core of your application and what not. UI is important but it’s not your business logic. Database is important but it’s not your application either.

I do not think that one should treat “Comment” as a database-mapped struct namespaced in context “BlogPosts”, because it’s external data, stored in table “comments”, in some external database “Db”. If we really kept that data internally, private to context - then yeah, it makes sense. But we don’t do that, most likely.

The reason for 1) not making sense for me is that UI will most likely diverge from and cross the boundaries of contexts anyway. The reason for 2) is actually the same, if you have single database, the queries to this database will cross contexts anyway.

2 Likes

It’s about the organization of business logic, and it’s a lot more about writes than reads. Consider the system we have at Cargosense. We track shipments of goods, with sensors on those goods to monitor environment data.

Our app started pre Phoenix 1.0, and so we naturally have a models directory with quite a lot of modules. Some of these could possibly be broken into an umbrella but even with that, our core application really does require quite a few tables.

Core shipment operations (start, stop, etc) need to touch a variety of tables that constitute what a shipment is (shipments, environments, containers, sensor_installations). Right now there isn’t a good place for that logic to sit. It doesn’t belong in a controller, and it doesn’t belong in any one schema module.

What I have right now is a set of use case modules, and the code sits in one of these. The issue is that the structure of the application code doesn’t really offer any clue that shipments, environments, containers, sensor_installations are part of some logical unit that should be generally manipulated via the business logic in my use case module.

Reorganizing my existing models directory into a pod structure under contexts makes it clear where the default location is for the business logic that is responsible for the underlying tables lives.

Reading data from multiple contexts isn’t really a big deal. I’m in the middle of finishing some GraphQL helpers that will make this particularly easy even without the associations.

To your points directly:

  1. UI will most likely diverge from and cross the boundaries of contexts anyway.

Of course! But you don’t organize your contexts on the basis of UI concerns. Pulling data from multiple contexts is completely fine. If you’re using something like GraphQL then it’s even clearer that you have internal organization, an external representation and then a mapping to go from one to the other.

if you have single database, the queries to this database will cross contexts anyway.

This is no longer considered problematic.

2 Likes

There are some documentation about how to write/customize your own generators on phoenix/elixir/mix projects? I have searching for that without success.

It seems to me that you have a very good argument that is also a good argument for contexts: there are several ways of achieving code organisation and each one of them is OK as LONG AS YOU KNOW what and why you are doing it.
Contexts make you think differently about it and it’s an evolution from pure “copy paste” code.
I like that.
So, I’m more convinced that Contexts are a great step in the good direction and that generators will evolve in the future based on real life experience.
Anyway, hearing from more advanced developers that don’t agree but bring very good arguments to the discussion pushes me to try some ideas myself, which is great.

1 Like

as a newbie who has “lurked” for some time :slight_smile: (and for the express purpose of learning) I think this response indicates you are too close to phoenix (understandably) to understand the issue and its causing you to be quite unfair in your response to Shrike here.

As newbie this is precisely how we come to a new framework. Looking at it, reading through docs and announcements and trying to get a gist for what it is and YES whether we wish to totally commit to learning it thoroughly. Your response here seems to be suggesting - learn my framework first and then come back and talk about specifics but thats not how the newbie world works. If the barriers to learn are too high and the rewards are not there they move on. IF Phoenix framework is just for those who work through the barriers thats fine but its not actually “Being accessible to newbies”

I am (maybe) beginning to get the hang of contexts but not their importance to me . The Op said it quite elegantly -as a newbie I am probably going to make a mess of a lot of things and I can see getting myself quite muddled using contexts maybe even more than without them. As someone else has astutely picked up one of the problems is the name. A few weeks ago when I read “Contexts” I said what in the world are those? As Cmkarlsson wrote

It is made into some big concept that you must learn and that scares people away.

Thats EXACTLY what it did to me and I put Phoenix on the back burner for those weeks (I however like elixir and for now Phoenix is THE web framework for it). Especially after reading, sorry, but just high minded conceptualizations sentences without much specified meaning (from others not you) without any code examples.

Now it may be represented as frivolous to say that I put something on the back burner because of one barrier but thats not true. the introduction of a somewhat nebulous concept of contexts was just a proverbial straw on the back after many straws (some of which a relatively new framework cannot escape)

relatively few code examples to other languages and frameworks
lack of diverse training material to grasp the fundamentals
wrapping your head around functional programming
dealing with concepts and words before that are also new OTP, BEAM etc
a lack of seeing features of the framework in large projects making a real difference on the web yet
and finally what might be controversial - a kind of Love - passive aggressive stance to Rails which brought it much of its early adopters.

I look at a framework as a newbie that pretty much refuses to give me authentication built in (leaving it to me to choose my own path or even roll my own) but then wants to bake in (whether in generators or not) “contexts” and as a newbie I have to ask the more barriers that come along - is this worth my time? Its something every newbie has to decide on…

In Django thers a sense of a “pythonic” guiding force in terms of how to approach things. In Ruby I guess its the simplicity and Developer Joy. My sense and I may be wrong is there is almost a guiding force in Phoenix now - to be anything else but Ruby and make a mark of separation so strong it can no longer be compared.

For me at least theres another barrier as a newbie. I really appreciate in modern programming how learning concepts and words in one framework and language helps me transfer over to other ones. If a framework is going to have unique head space words I have to decide if it has something earth shatterring in web development that the web just can’t live without.

There are more non newbies in this thread than newbies so hope you will appreciate an outside newbie looking in

It’s not unfair to ask folks, newcomers included, to at the very least use Phoenix before etching their stance in the ground. This doesn’t mean I expect folks to learn everything, by “use” I mean at the very least I think it’s fair to ask someone to go through the getting started guides and tinker a bit before coming here and piling on with negative feedback.

That is not at all what I’m suggesting. The post in question said they hadn’t even taken a look at Phoenix yet.

I always appreciate newcomer stances, since after all I’ve been helping to grow the community since the beginning. We are by definition community of newcomers right now :slight_smile: I’m only asking that folks who want to take a negative stance here at least have run mix phx.new before. I don’t think asking such is unfair.

3 Likes

Fair enough Chris. It is what he said. I did not take it as literally or he would not even have been here discussing contexts. I do think a lot of people read through docs before they decide to actually create a project. Then they weigh the complexity they see against the utility. Anyway I have fired up a few projects and I would have done the project I am now doing in Django with Phoenix except as a newbie I thought it would be a very bad idea for me to roll my own authentication or even think i had expertise to know which existing one was secure enough.

I guess for contexts to totally click for me I would have to see a project in a previous version compared against the latest version of phoenix with code to really see the overall benefit it is to me in my stage of development. I am at the dangerous point of thinking I get it but not sure what I got is right :slight_smile: . I’ll keep looking and reading.

This is exactly my main pain point with Phoenix. Although I use it I feel very insecure with authentication because I’ve used all the approaches:

  1. make my own following the Phoenix book (to learn and use it in a very simple use case);
  2. use Coherence/Openmaize/Phaux but none of them is stable or covers my needs (Oauth);
  3. looked into Guardian/Ueberauth but I don’t want to use JWT (I want to leverage Phoenix tokens).

So, authentication COULD BE the thing to make me leave Phoenix as IT IS the thing that keeps me periodically testing other languages/Frameworks hopping that there’s a new one as good as Phoenix but with Authentication.

1 Like

It’s nor unfair indeed, but in the competition for user mindshare, it might lose future hsers to other applications that do not require that. We have an embarassment of riches in the web framework department. I’ve only used python frameworks and some JVM based ones. Python alone gives you Django, Flask, Pyramid, Websauna (on top of pyramid), aiohttp, sanic, falcon, CherryPy, Tornado, Twist. The JVM gives you several languages, each with one or more frameworks, like java (Spring, Spark, too many others to list), clojure (Luminus, noir, but it’s mostly DYI based on something like plug), scala (Play, Scalatra, Liftweb), kotlin, etc. This doesn’t even touch frameworks written for C, C++, go, javascript, ruby or many others.

Most of these frameworks are quite similar to each other (i.e, they consume requests and spits responses – booooooring, that’s so '00s!), but they all create noise. At the same time, some are obviously better than others for some uses, even within their sometimes limited capabilities.

A newbie doesn’t have time to run framework.new for every framework out there. This is not a value judgement, it’s just a fact of life. I’ve done it for most python frameworks, but I can’t do it for all frameworks out there. So frameworks must have some way to convince users to try it. For me, it was @chrismccord’s video in the phoenix’s website here. But the video is not thaaat convincing in general for a newbie. It convinced ME because I had a prototype of a web application I couldn’t make work with anything else the way I wanted it. But newbies often want to learn a tool to build something, to get “something” out there. And that video is not convincing for those use cases. I do believe that phoenix is the best tool for almost any web application, unless some other language already has something prebuilt that is just perfect for your project (libraries, frameworks, etc). But nothing in the docs/forums or videos would convince me of that.

To capture newbies, Phoenix must sometimes say: Phoenix is your application! Dump stuff into the controller! Request comes in, response comes out! Web apps are interfaces to a database! 90% of all web apps are CRUD and phoenix is very good with CRUD! Phoenix is Rails + websockets! Even if these are all either bold faced lies or very stupid marketing points.

This is what gets you newbies, not 9 nines of reliability, not code swapping, not references to boring european Telecom companies. I mean, these things totally got me, but I’m not thaaat newbie.

Now, it’s totally OK if you don’t want to get very green newbies, or if you don’t want to get them hard enough to use my sentences above as marketing points. Some users in this or other threads have actually said that too many newbies is not necessary a good thing. But to be really appealing to newbies, Phoenix reeaaaly needs to up its marketing game.

Another problem is the fact that many inexperienced people seem to be jumping to a release candidate that doesn’t even have a proper tutorial that holds your hand through an entire application.

But in any case, I think some concrete steps might help newbies and even some more experiences users that are struggling with contexts.

What might help newbies

First, from the point of view of the general documentation:

Django has the django tutorial. Flask has Miguel’s mega tutorial. Pyramid has the official docs and multiple tutorials. They all walk you through an example application, through which you can get a feeling for how an app is structured and why things are the way they are. Phoenix has some docs for 1.2 that are as good as the ones I’ve linked, but nothing that deals with contexts…

Newbies NEED and example of an application using 2 or more contexts with real boundaries, in a way that makes sense, even if it is a small application or almost a toy. As long as it’s real code it’s good enough. We don’t have anything like this yet, and we should. This is something I’d like to do, but for which I unfortunately have no time right now.

This is even more important if Phoenix wants to capture newbies that are learning Elixir as their first backend app (well, second if you count JS as a “backend app”). I came to phoenix already knowing what DB abstraction layers, models, routes were. Each of this things is a stumbling block. Functional programming (which is much simpler to me than any of the alternatives actually), processes, OTP (even if you can do stuff without knowing what this is) and Supervisors. Now, add the 3-4 levels deep abstraction (as I explain below) of contexts, and you might get the:

Second, from the point of view of context generators:

I think contexts are a very good thing. They are a nice way of organizing the code and very useful even for relatively small apps. But they are also a little abstract. In the end, a schema is a subset of DB columns, and a context is a set of schemas. If you think like this, you can actually draw some pretty Ven diagrams or some pretty tri-partite graphs. But a command line interface is not a very good interface to something that draws tri-partite graphs.

A way of visualizing DB tables, schemas and contexts graphically would be very helpful for some visual thinkers like me. Double so if you could actually plan and generate your contexts/schemas/migrations through a visual interface. The stack from contexts to DB tables is 3 levels deep (and 3 files away!). Also, migrations are “deltas” of tables, so you can count that as a 4th level.

3 Likes

alright then I may be close. Could you say that since in some frameworks you devise your schema from your model that contexts are more discrete/specified models that divide/hones in on the separate concerns? lol if thats not even close then I have a LOT more reading to do.

[quote][quote=“PJextra, post:121, topic:5929”]
2) use Coherence/Openmaize/Phaux but none of them is stable or covers my needs (Oauth);
[/quote]

I hope thats not true. Contexts might turn out not to be a barrier too high but asking newbies to choose among unstable packages (and I had thought by now one of those would be) or roll their own definitely is not newbie friendly. the community should at least have a vetted recommended and stable package.