Contexts - a barrier too high for newbies?

I’m glad I’m not the only one who does this.

I’m honestly really confused about this whole conversation in the first place. So I’ve largely ignored it.

Please forgive me for my ignorance, but why are we arguing over generators?

I mean, if you don’t understand Contexts and need to get something done then don’t use them.
If you feel like Contexts just don’t suite your application’s needs then don’t use them.
If just don’t like Contexts, then don’t use them.

Personally, I think that the Phoenix Core Team saw using Contexts as beneficial enough to change the generators. These kinds of changes aren’t made lightly.

I happily defer hard design decisions to anyone else that’s not me. But if I don’t like your decision, I’m not above doing my own thing.

2 Likes

For newbies everything can be a barrier.
Especially if you’re a programming newbie too. That was my case and I remember my first days with Phoenix/Elixir when after reading some guides and documentation I tried to start my project. I did know HTML and CSS and some jQuery but I still remember that I just didn’t know if I should put my code on an .exs or an .ex file. And exactly on which file.
The good thing was that I asked for help and as much as my questions were strange, I always got the help that I needed. Very fast.
Then, every here and then I felt the same with other issues. I looked to other languages and frameworks, experimented a little bit with them and as long as I could copy some code I was good. When I had to think for myself, then…I always got back to Elixir/Phoenix because not only I always got the help I needed as I realised more and more that the mindset behind it was really powerful. I always realised that they were more or at least as correct as on others languages an frameworks.
And so on.
These last months, with some simple apps running in production I, again, though: contexts…I’ll need to refactor my code AND MY MIND. I delayed it, but then when I started refactoring the code, not only I realised that CONTEXTS ARE BETTER than what existed but I ALSO LEARNED A FEW MORE things along the way.
These are the efforts that make us better. That force us to grow. Nobody likes to change when they don’t feel in the mood to change. I don’t. I try to delay it and avoid it. I go looking for alternatives that are less pailful.
Only to realise that to get to the top and really understand every line of my code I need to go through this.
Of course I have some colleagues that keep copying and pasting code that they find googling. They produce code, not software. They can’t change that code after 1 year.
I hate to comment my code. But I love it after some months.
In the end a newbie must decide: are contexts a barrier too high to be a good software developer? If yes, then copy paste code is what you’ll get. No pain but not really a lot of gain.
If you don’t believe, (as a newbie) please try to do the same solid, fast, scalable and fault-tolerant web app using Go. I tried 3 times and gave up. Try do it using Rust. I tried 2 times and gave up. Try do it using Node/Javascript. I tried, I though I did only to realise that they were not maintainable nor scalable…Try do it with Swift and you’ll realise that Windows is not an option.
Of course there are other languages and frameworks. But I’m focusing only on relatively new languages. And only as a newbie that is not a programming PRO. Because I believe that really pros will find problems either way and they are used to solve them.

PS: It’s very important to trust the ones that were able to continuously evolve Elixir/Phoenix in the right direction. Especially when we don’t have a better idea to solve the issues that were visible and are being solved using contexts. So for me, contexts WILL IMPROVE as everything else and if in a few years we’ll have to change…,then will do!

6 Likes

I had some time to go through https://github.com/hexpm/hexpm as it was mentioned as a good resource to learn how the authors used contexts in practice.

What I found were a bunch of namespaced schemas (let’s be honest, they are models with validations, persistence, and queries), not the proposed contexts that the generators are producing.

Here is my take away from this:

Namespaced Schemas which define their public api == clean, understandable, seems to works well and blessed by the core team as a solid example.

Contexts as proposed == MIA

I’ll say it again, it would be splendid to see real examples of how Contexts actually shake out from people that have been thinking about them for a while. Nothing would benefit this conversation more than a large hunk of code that we can all look at and judge.

Cheers,
Benjamin

P.S. Apologies if my writing comes across anything but friendly. Most of my terrible jokes are terrible :stuck_out_tongue: However I think that the snarkiness that is in this thread gave a lot of the frustrated folks a chance to let out how they really felt. Like they say, you’ve never really used something until you have used it in anger :wink:

1 Like

My ElixirConfEU keynote is up on youtube and the first part of the talk addresses some of the points here – confusions around contexts, feedback we’ve gotten, why we named it “contexts”, etc. It might be worth watching for some added… context :slight_smile:

10 Likes

One important note here is that the way hexpm is using contexts may not be what the phoenix team envisioned and although we have been working on separating our concerns into contexts there are some places they leak through because it’s an ongoing effort. Hexpm started out without contexts and it worked just fine for multiple years but as the project grew we felt we needed to split some parts apart and since contexts were introduced around the same time we felt that was a good way of doing it. We are also not following contexts religiously, for example we don’t use umbrellas, this is also fine.

5 Likes

Two posts were merged into an existing topic: How would you explain Phoenix Contexts to a newbie?

Exactly!! How annoying it’s is to write the same boiler plate over and over again. One of the reasons I like rails is that I can stub out the boilerplate with a generator, delete what I don’t need and move fast from there. If anything, Phoenix needs more and more powerful generators.

Blog posts, articles, documentation and step by step guides are fantastic learning tools and I would argue more effective then machine generated files that you have no idea how they came to be. It’s something that’s been bugging me from the beginning, this notion of generators as learning tools as opposed to productivity tools.

That said, personally I do like the move away from models to bounded contexts, the more time I spend with 1.3.0. The trick as others have mentioned is to not read too much design patterns into it - it’s just files and directories, but they encourage better separation and better separation of concerns is always a good thing in software.

2 Likes

I think 'Context’s are a good idea, just the name is weird (I’ll watch the video when I can ^.^), but the way I do it is shaped by decades of Erlang, C++, Python, and Java, with lots of other languages sprinkled in, so my Elixir projects may be a little different in style. Basically here is what I do:

  • Web: Is strictly for front-end things like controllers, channels, commanders, routing, and views/templates. Everything is kept as minimal as possible just calling out to somewhere else to get data (growingly becoming Absinthe.run in a lot of places…) and send it back out.

  • Models: Yes I still have plain models though the directory is called “db” (as is the namespace), it is literally the schemas that map directly to the database tables, a lot of the tables I did not create and have no control over (other systems) but that I have to deal with (you can definitely feel a lot of Ecto’s limitations when the system was not built ‘by’ ecto…), they hold no data, no processing, not even changesets, they are just the schema.

  • Modules: Not modules like elixir modules but modules of functionality, like one is called ‘Accounts’ (within MyServer.Accounts) but there are many. The Accounts (elixir) module itself just has a set of calls on it, things like confirm_by_login/1 that can be called like Accounts.confirm_by_login(login: username, password: the_password) or Accounts.confirm_by_login(pidm: pidm, password: the_password) or Accounts.confirm_by_login(banner_username: username, password: the_password) or Accounts.confirm_by_login(google: google_oauth_struct) or various others, all of which is called via my Ueberauth callbacks from a variety of systems, and it returns either a valid account id (a UUID built from a variety of internal data so it is easily reversible to access various systems) or it returns an exception struct, no it does not raise it, it returns it, that allows me to build up a list of possible errors (using some of expede’s libraries, which are quite nice I must add) and return many error messages instead of just the first that happened to happen.
     
    The Accounts module has a lot more calls on it to get various information about an account, do things to accounts, etc… etc…
     
    Most of the modules however (very few calls on Accounts) have a lot of query_ and multi_ calls, depending on if getting or setting data calls (many ‘getting’ use multi_ as well because access needs to be logged in many cases), such as the Banner module, it has things like (this is the only short one in that whole module, most are utterly ginormous because of the horror’s of accessing this ancient system and needing to join across 10 tables in most cases and a lot more in a few, this one will grow that big over time as well…):

  def query_classes(selected \\ :processed, refine) do
    squery =
      from course in DB.Banner.SCBCRSE,
      join: section in DB.Banner.SSBSECT, on: section.ssbsect_subj_code == course.scbcrse_subj_code and section.ssbsect_crse_numb == course.scbcrse_crse_numb and section.ssbsect_ssts_code == "A",
      join: dept in DB.Banner.STVDEPT, on: dept.stvdept_code == course.scbcrse_dept_code

    squery =
      Enum.reduce(refine, squery, fn
        ({:pidm, true}, squery) ->
          join(squery, :inner, [course, section, dept],
            student_course in DB.Banner.SFRSTCR,
            student_course.sfrstcr_term_code == section.ssbsect_term_code and
            student_course.sfrstcr_crn == section.ssbsect_crn
          )
        ({:pidm, pidm}, squery) when is_integer(pidm) ->
          join(squery, :inner, [course, section, dept],
            student_course in DB.Banner.SFRSTCR,
            student_course.sfrstcr_pidm == ^pidm and
            student_course.sfrstcr_term_code == section.ssbsect_term_code and
            student_course.sfrstcr_crn == section.ssbsect_crn
          )
        ({:pidm, pidms}, squery) when is_list(pidms) ->
          join(squery, :inner, [course, section, dept],
            student_course in DB.Banner.SFRSTCR,
            student_course.sfrstcr_pidm in ^pidms and
            student_course.sfrstcr_term_code == section.ssbsect_term_code and
            student_course.sfrstcr_crn == section.ssbsect_crn
          )
        ({:registered, true}, squery) ->
          where(squery, [course, section, dept, student_course], student_course.sfrstcr_rsts_code in ["RA", "RE", "RW"])
        ({:withdrawn, true}, squery) ->
          where(squery, [course, section, dept, student_course], student_course.sfrstcr_rsts_code == "WD")
        ({:department, dept_code}, squery) when is_binary(dept_code) ->
          where(squery, [course, section, dept], course.scbcrse_dept_code == ^dept_code)
        ({:department, dept_code}, squery) when is_list(dept_code) ->
          where(squery, [course, section, dept], course.scbcrse_dept_code in ^dept_code)
        ({:subject, subject_code}, squery) when is_binary(subject_code) ->
          where(squery, [course, section, dept], section.ssbsect_subj_code == ^subject_code)
        ({:subject, subject_code}, squery) when is_list(subject_code) ->
          where(squery, [course, section, dept], section.ssbsect_subj_code in ^subject_code)
        ({:course, course_number}, squery) when is_binary(course_number) ->
          where(squery, [course, section, dept], section.ssbsect_crse_numb == ^course_number)
        ({:course, course_number}, squery) when is_list(course_number) ->
          where(squery, [course, section, dept], section.ssbsect_crse_numb in ^course_number)
        ({semester, year}, squery) when semester in [:spring, :summer, :fall] and is_integer(year) and year>=1900 and year<=9999 -> squery # Handled below in `dyn`
        ({:course_group, %DB.Course.Group{}=course_group}, squery) ->
          squery = if(course_group.dept_codes, do: where(squery, [course, section, dept], course.scbcrse_dept_code in ^course_group.dept_codes), else: squery)
          squery = if(course_group.subject_codes, do: where(squery, [course, section, dept], section.ssbsect_subj_code in ^course_group.subject_codes), else: squery)
          squery = if(course_group.course_numbers, do: where(squery, [course, section, dept], section.ssbsect_crse_numb in ^course_group.course_numbers), else: squery)
          # TODO:  Test the term codes and such too as they will be wanted eventually...
          squery
      end)

    dyn =
      Enum.reduce(refine, false, fn
        ({semester, year}, dyn) when semester in [:spring, :summer, :fall] and is_integer(year) and year>=1900 and year<=9999 ->
          term_code =
            case semester do
              :spring -> "#{year}10"
              :summer -> "#{year}20"
              :fall -> "#{year}30"
            end
          case dyn do
            false -> dynamic([course, section, dept], section.ssbsect_term_code == ^term_code)
            dyn ->  dynamic([course, section, dept], ^dyn or section.ssbsect_term_code == ^term_code)
          end
        (_, dyn) -> dyn
      end)

    squery =
      case dyn do
        false -> squery
        dyn -> where(squery, ^dyn)
      end

    squery =
      case selected do
        :all -> squery
        :processed ->
          case Keyword.get(refine, :pidm) do
            v when v == true or is_list(v) ->
              select(squery, [course, section, dept, student_course], %{
                pidm: student_course.sfrstcr_pidm,
                department_code: course.scbcrse_dept_code,
                department_description: dept.stvdept_desc,
                crn: section.ssbsect_crn,
                subject: section.ssbsect_subj_code,
                course_number: section.ssbsect_crse_numb,
                section_number: section.ssbsect_seq_numb,
                title: fragment("coalesce(?, ?)", section.ssbsect_crse_title, course.scbcrse_title),
                section_begins: section.ssbsect_ptrm_start_date,
                section_ends: section.ssbsect_ptrm_end_date,
                registration_code: student_course.sfrstcr_rsts_code,
                effective_term: course.scbcrse_eff_term,
                _effective_term_rank: fragment("rank() OVER (PARTITION BY ?, ? ORDER BY ? DESC)", section.ssbsect_subj_code, section.ssbsect_crse_numb, course.scbcrse_eff_term),
              })
            _ ->
              select(squery, [course, section, dept], %{
                department_code: course.scbcrse_dept_code,
                department_description: dept.stvdept_desc,
                crn: section.ssbsect_crn,
                subject: section.ssbsect_subj_code,
                course_number: section.ssbsect_crse_numb,
                section_number: section.ssbsect_seq_numb,
                title: fragment("coalesce(?, ?)", section.ssbsect_crse_title, course.scbcrse_title),
                section_begins: section.ssbsect_ptrm_start_date,
                section_ends: section.ssbsect_ptrm_end_date,
                effective_term: course.scbcrse_eff_term,
                _effective_term_rank: fragment("rank() OVER (PARTITION BY ?, ? ORDER BY ? DESC)", section.ssbsect_subj_code, section.ssbsect_crse_numb, course.scbcrse_eff_term),
              })
          end
      end

    query =
      from s in subquery(squery),
      where: s._effective_term_rank == 1

    query
  end

(Wtf, without an extra newline here the forum hides this entire next paragraph?! Bug with code fences inside bullets??)
Now in this one, like most queries in the system, follow this pattern, as a lot of the queries are user-driven and built from custom things to create reports that they pull then a lot of the things that are queried are actually built up from a whole set of refinements (that grows over time as their needs grow over time for more and more reports). But as you can see the ‘functionality’ is split into specific areas. Inside of, say, Accounts are more modules like Accounts.PIDM and Accounts.Google and so forth but nothing outside accesses those directly, everything goes through the main interface.

But this is just normal separation of concerns that is pretty universal among ‘many’ programming languages, but I’ve never heard of a Separation of Concerns as Contexts though, nor does it really seem like it should have a special name anyway as it just seems like normal Good Programming?

So yeah, I do not get why ‘Contexts’ are a barrier for newbies to Elixir, this is stuff that any programmer should be doing in any language already. If anything, giving it such a ‘special name’ is the most confusing part about it, but they themselves are simple. I really do think calling them what they are, Separation of Concerns would be far more descriptive and less harrowing to newbies as Contexts has no real context.

/me has never heard of DDD before Phoenix added the ‘Context’ special stuff, and is wondering if that DDD book is just rehashing 50-year-old separation of concern ideas for a money-grab or something…

3 Likes

I’m not really sure if my thoughts should really be posted here, or in the other thread, or a new thread. If it’s not appropriate please feel free to move it.

As someone from a self-trained, and more sysadmin background, the explanation from @slashdotdash is spot on how I feel I should be using contexts. I totally get it in theory. And I suspect many people do.

However in my tiny team of me, myself and I (working on a personal project), I’m finding I’m getting literally anxious about Doing It Right™. Which I know is dumb, but I do wonder if this is how other people feel and what’s causing some of the discussion towards contexts.

I’m hoping a small example might help out where I’m personally getting “stuck” -

Assuming a simple todo web app (because this is 2017, what else?) - users can only see their own todos, unless they’re marked as an admin.

My fictional, web only, system is behind authentication. When a request is made with a token/cookie/whatever currently I can either verify and load the user from the DB, assigning a user struct to the conn (my user struct has some kind of role/permissions/flag/whatever that dictates they’re an admin). Or I can just assign the user id to the conn.

If I just had the user_id on the conn, when I call Todos.list_all I have to load the user from the DB for a second time to determine their permission. My brain screams at the inefficiency of doing it again when I literally just did it to verify the user hasn’t been disabled/whatever. Or if I had a whole user struct loaded, I can just pass that as an argument to list_all - probably using pattern matching to behave differently for admins and standard users.

My anxiety comes in here.

I feel with contexts that I probably shouldn’t really be passing whole user structs around, as I’m coupling my account/login/whatever context to many other contexts. But the other part of my brain tells me querying the DB multiple times for shit I already have, or could have collected when the initial request was made, is dumb. The app isn’t going to become huge. There’s no need to worry about it. By passing around a struct for a user I can use 2 functions with different pattern matching to deal with the permissions differently, and it feels sane. But kinda dirty.

The anxiousness paralyzes me and I have this endless back and forth in my head.

The second part of my anxiety starts coming in my general lack of decent experience with general API design and whilst I totally understand that this is not specific to “contexts”, I suspect is also a factor. Contexts force me to learn and experiment about what is good/better API design. Say I wanted to start paginating the todos, because admins now have a lot of todos (what with being able to see everyone’s). Adding extra arguments for page_number and items_per_page will land me in trouble eventually as I add more and more arguments for specific things. Having a map of arguments feels opaque. The anxiety comes back.

Ultimately I don’t feel like I’m really experienced enough to make these decisions and I end up flipping back and forth between competing designs without ever achieving something.

Edit: These are obviously my personal feelings of inadequacy at play here. I’m not trying to shit on the direction the framework is going. I actually think it’s a good idea, because it will eventually make me better at writing and understanding code (and therefore a better sysadmin I hope). I just feel seeing the same kind of explanations over and over again isn’t really helping me, but adding to my anxiousness of “shit you’re doing it wrong, son”.

9 Likes

Fwiw, one of the things that bothered me about regularly using generators was specifically that you were writing the same stuff over and over. I never understood why that was a good idea.

I get the reason for it, that essentially you are generating the same mappings to fit a REST model for the data…but it always seemed to me that if that was the goal it should be done once with a white labeled list of valid paths. Then, if you needed to modify a part of it create a specific controller for that part.

I realize that wasn’t a popular opinion, but it just always seemed like heavy use of generators was more of an abstraction problem than anything else.

Yes, and if anything this is something that should be pushed. This is not only for phoenix that we are discussing here but for any elixir app so a best practice approach for elixir that can be applied to phoenix would be great

2 Likes

I’m a beginner in this web stuff, but the little experience I have doesn’t necessary agree with you.

What you say seems superficially like a good idea, but in practice there are lots of little details you want to customize. In something like python you’d do it with class inheritance and method/attribute overrides, but in my very limited experience you end up almost rewriting the whole code. If you’re getting too deep into the bowels of the original class, you should probably rewrite ir yourself.

You could probably use elixir macros to mimic something like class inheritance, but in the end, the most flexible “abstraction” is a mix task that generates editable code. Not the most concise abstraction, sure, but it is flexible.

When I first got into Elixir and Phoenix, these generators rubbed me the wrong way, because they weren’t promoting “code reuse” or using clever abstractions, but now I’m not so sure anymore. I’m starting to see the value of generated text files as a mechanism for extensibility.

*cough*oop*cough* Don’t use this ^.^

There definitely is!

I use a ton of snippets in my editor and can fill out entire, say, GenServers or Controllers with but 2 keystrokes, including filling in default values and other things, but those are definitely more useful in an IDE then a command prompt I do think. :slight_smile:

1 Like

This is, for the lack of a better word… Amazing, I guess? xD If you base these things on an Agent or a GenServer you can get mutable class attributes, python style, right? xD I can’t wait to reimplement Flask, Django or Pyramid in Elixir based on this oop library!

The ergonomics of using these generators on a command prompt are not very good. Once I made a typo and spent the next 10 minutes chasing places where I had to change the code. I’ve been thinking on writing a mix task to run the generators based on a yaml file, as yaml files are very lightweight and allow you to use a decent text editor and proofread your template before generating actual code.

Alternatively, something like a GUI “context builder” could be useful. This could be done as a local phoenix app for maximum portability. IDE plugins would be cool too, but they’re less portable.

I believe you have described the feelings of many very very well. Thank you for posting.

Part of the issue here is that many still make contexts more than they are, they are just modules and directories.

But it is also part of how we introduced those modules as hard boundaries, which makes you feel you need to get them right from the beginning, and that’s not necessarily the case. We will work on some improvements for the next RC which hopefully will take some of the burden from developers.

10 Likes

That’s exactly how I use generators too, including Phoenix. The learning tools focus to me means that we won’t shy away from adding comments and that we won’t generate a bunch of unnecessary files on the odd chance you may need to do something with them.

2 Likes

Naming is hard. :slight_smile:

I also think “services” are even more loaded than “context”. Concern feels like a subset rather than superset. For example, a user has many concerns. While “context” expresses the idea of groups / ownership.

A “business module” is not bad but I am afraid calling it “business” will make it even harder for people to come up with names, as they may want to name it directly to an aspect of the business or the organization and that’s not always the case.

@Qqwy’s description, summarized by @AstonJ, is really good: contexts are just dedicated modules for functionality that can easily be classified as a group.

7 Likes

Thanks for confirming that José, I have added it to the learning resources thread:

(I’m sure we can expand on it a little, but for now I think it’s fine to give people a very quick overview.)

Context is just like a scope for the modules, is not more than this, i advice who have negative positions about it to start an 1.3 phoenix project and start coding, you will see that did not change almost anything, just the structure of modules…i recently started an 1.3 project myself and like it. Before start using context was sound more complex than really are. Is a good move to have code more organized. I like it.

1 Like

Ah, seems like this is a quite general issue because I see people in Elm community struggling with composing application written in FP structure too although they also have architecture constraint (TEA).

I brought Phoenix context thinking into Elm files structure, and it turns out ok-ish because there is only single source of truth while elixir/phoenix can use Repo to update several tables independently. I generalize context mindset as a place where functionalities requires different data structures to be working with

Like Chemical Formula is a context where different Elements live together to form certain characteristic and functions. A bunch of elements can form different chemical formulas … (H2O)!