dsignr

dsignr

Understanding contexts, web and DDD

Hi all, my first post here. Certainly not new here.
First of all, huge congrats on the team for 1.3. It’s amazing in so many ways that it has actually forced me to re-think the way I create applications. I found myself spending hours at coffee shops drawing diagrams, contexts, etc. and each time I do it, I discover something new about the business problem space that I didn’t think of before (as a programmer). I’m loving it so far! However, I’ve seem to hit a roadblock..

Before I dive into my question, I’d like to give you a heads up:

  • I’ve so far read all 3 of the major books on DDD
  • Watched Chris’s video of Phx 1.3
  • Searched the forum posts

So, now for the question.

I’m building an ECommerce portal based off Phoenix 1.3. Let’s assume I have just 2 contexts - CMS and Ecommerce.

- CMS
- E-Commerce

Now, it so happens that the CMS domain has a page entity. While the Ecommerce entity has a product entity. I want to create a category model within the CMS context as well as the Ecommerce context.

- CMS
--- Category
- E-Commerce
--- Category

However, when I run the generators for the category, they seem to conflict:
mix phx.gen.html CMS Category categories name:string shortcode:string description:text
mix phx.gen.html ECommerce Category categories name:string shortcode:string description:text

They seem to overwrite the controllers and the views with one another, since they are all generated inside the Web context:

* creating lib/marketer/web/controllers/category_controller.ex
lib/marketer/web/controllers/category_controller.ex already exists, overwrite? [Yn] y
* creating lib/marketer/web/templates/category/edit.html.eex
lib/marketer/web/templates/category/edit.html.eex already exists, overwrite? [Yn] y

So, community, kindly help me to understand:

A. What exactly is the web context for?
B. Why can’t my Ecommerce and CMS namespaces handle their own templates, views, etc, separately

For example:

Ecommerce/
   controllers/
        ....
   templates/
        ....

CMS/
   controllers/
        ....
   templates/
        ....

C. The whole problem seems to arise because everything is being dumped into the Web context. I don’t understand, everything is technically a web-facing entity, right? Phoenix is a web application framework, so my CMS, Ecommerce by right are all web-based functionalities already. Why should I dump it all into the web folder?

D. How would you re-architect this, instead? Clearly, I don’t seem to understand the new structure, or I’m missing something basic. I would really appreciate if someone can point me in the right direction.

Your thoughts, advice or any sort of help is much appreciated :slight_smile:

Thank you for your time.

First Post!

easco

easco

As I understand it, Contexts in Phoenix are meant to separate your model behaviors from the way those models are persisted (e.g. In a database). What you seem to want to do is separate elements in the view and controller layers using the same context concept. Seems like something you could legitimately want to do, but I just don’t think they Context concept from Phoenix 1.3 is meant to do that.

What make make more sense, in your case, is having separate applications (one for E-commerce and one for CMS) in an umbrella framework. Your two web front ends for each of those areas could share underlying logic in a third application for shared model level functionality.

Most Liked

Qqwy

Qqwy

TypeCheck Core Team

@dsignr

So, to give an example. Say you have a ticket-selling service.

Features:

  • Guest must be able to create a User account.
  • Guest must be able to be authenticated as a certain User.
  • User must be able to see upcoming shows.
  • User must be able to buy a ticket, iff the tickets for a certain show are not sold out and the show’s deadline is not yet passed.
  • The Admin must be able to create new shows with a certain limited number of tickets, the price per ticket, etc. (and some other settings).

Now, it might make sense to split this up as follows:

Authentication Domain

This part is used to check if a certain guest can convince the application that they are a certain User. It uses some kind of underwater dispatching (which is an implementation detail and might be changed at any time!) to check:

  • If a user with the given name exists
  • If this user can be authenticated with the given passprase

(possibility to enhance for multi-factor authentication, etc).

When successfull, it returns the symbolic User reference to the outside world, which other domains might use to know that certain actions are (attempted to be) taken by a certain user.

The Authentication Domain asks other parts of the application for certain information. For instance, the User Domain will be asked if the given hashed password matches the one in the user domain user’s hashed password field.

The Authentication Domain does not care about what users actually are or where/how they are stored. (and has no idea about what tickets are at all).

Authorization Domain

This part is used to check if a certain entity can perform a certain action. Some other parts of your application will ask the Authorization domain if the current entity is allowed to do a certain action. The only thing the Authorization Domain knows, is what entities (referenced with some kind of symbolic identifier) are and aren’t allowed to do certain things.

The Authorization Domain consists of some bridges between some of the other domains. To check if a user might alter some user details, we might check if these are details of the current user, or if the current user is an administrator, for instance.

The Authorization Domain does not care what users or tickets actually are.

User Domain

Keeps track of general settings of a user, like their e-mail address, name, phone number, etc. A user can alter this information, but only for their own account.

The User Domain uses ‘some form of persistence’ to store user information for later retrieval.

Ticket Domain

The Ticket Domain stores information about shows and the tickets that might still be available. The Ticket Domain knows all about tickets, but not about the rest of the world. The outside world can ask to buy a ticket, which is then linked to a user. The Administration Domain can ask the Ticket Domain to create a new show.

The Ticket Domain doesn’t know what a user is (other than that a user has zero or more tickets that can be returned given the user’s symbolic identifier).

The Ticket Domain uses ‘some form of persistence’ to store shows, and what tickets have already been bought.

Administration Domain

The Administration Domain wraps the Ticket Domain and the User Domain to show and alter information that only the administrator(s) have access to. Access checking happens using the Authorization Domain, of course.

Phoenix

Enter Phoenix. This is just the web representation. All your controller (and channel) actions should do simple ‘flow redirection’, and call one (maybe two) functions inside a domain application, whose result(s) are then forwarded to the view+template layer (or channel broadcasts).

To allow users to authenticate over the internet, a technology like JSON Web Tokens could be used. A separate domain could be set up that handles the wrapping/unwrapping of these tokens that are passed to/from the user session, after which the Authentication and/or Authorization domains are used to find the unwrapped symbolic user reference.

There is no need to make the other domains care about authentication (how a browser session maps to a user).

Persistence

You might have noticed that I wrote ‘some form of persistence’ in multiple places. Because of the separation of these different concerns/domains, it is completely possible to store the list of users in one kind of persistence, and the list of tickets in a completely different kind. Everything will continue to work, regardless of if you use Ecto, MongoDB, ElasticSearch, Erlang Term Storage, a flat serialized file, or an in-memory ephemeral GenServer or Agent for this, because the persistence layers are only implementation details.

If you want to make the persistence layer as swappable as possible, you can go one step further, and use the (functional variant of) the Dependency Inversion Principle; specifying (for each domain/context/application) a persistence-behaviour that a persistence-layer-provider should implement.

Consuming External APIs

Maybe it is a requirement that we facilitate payments using Stripe, PayPal, iDeal, etc. The only domain that would be hit by this is the Ticket Domain. We could block new tickets from being bought until a pending transaction becomes either successful or failed. The nice thing is that all other parts of the application do not need any changes.

Providing a public API

Because we have a nice separation of concerns, we already have a good definition of how a public API could look like, and most public Domain functions could be wrapped 1:1 by an API call (for the domains we want to expose, of course).

‘Symbolic user reference’

This could be anything, as long as the different sections of your app agree about what it is. The Authentication Domain uses it to authenticate a user, the User Domain to keep user details up-to-date, the Ticket Domain to associate zero or more tickets with it. This might be a database Identifier, an Unique Universal Identifier, a simple struct, or something else that’s up to you.


Something like that. :slight_smile:

mjadczak

mjadczak

You’ve written quite a detailed post, so forgive me if I skip over anything, but here are my impressions on the scenario you’ve proposed.

I don’t think that an admin panel / back office is a separate domain here. A domain isn’t a subsection of your website. A domain is a “subsection” of your system or business. This relates to what you said:

Routes don’t belong in a domain. Routes belong in your web interface, and your other domains shouldn’t really care about them. Your core business logic domains shouldn’t care if they are being accessed through a web interface, the command line, or a touchscreen kiosk somewhere. I think the separation of your actual application from the web interface is something you’re still missing in your mental model. This talk may be useful.

Neither. The routes belong in a web interface layer, built e.g. with Phoenix, which merely interfaces with the system using its API boundaries. All of the other domains shouldn’t know or care that they are being accessed through the web, and hence they will have no concept of routes.

This ties in with the previous point a bit. Something which I feel many people mistakenly believe about the type of model that is being advocated here is the notion that domains must be completely separate systems that share no data. That is not true. What they should not share is internal implementation details—instead they should communicate through their public interface.

It is true that in the web interface context, the different domains will need access to different types of information about the user. It is true that this information needs to be stored into a session or cookies or some other state by the module that interfaces with the web browser (e.g. Phoenix). However, this does not mean that Phoenix needs to be coupled to these domains.

Instead, it is ok for these domains to return things to be treated as opaque, to be stored and given back later. An example of this might be a User ID. It’s ok for the Phoenix layer to store a User ID somewhere in a session, and provide it to domain modules where needed. It is likewise ok for the Flight Booking domain to hold on to a User ID within its own internal flight booking data. The key here is that these domains never take that ID and directly interface with the database to get user data—instead, they use that ID to ask the Users domain to perform any operations.

Let’s take this particular example. We’re going have three “Domains” here, Authentication, Authorization and Flights, as well as the web interface, Web.

First, if a user logs in, we handle that at the Web layer by passing their credentials to Authentication and obtaining a user_id, which we store in session. Note that the way we get a User struct is by calling the Users domain—we don’t query the database ourselves.

def login(conn, %{"user" => username, "pass" => pass}) do
  case Authentication.auth_with_credentials(username, pass) do
    {:ok, user_id} ->
      conn = put_session(conn, :user_id, user_id)
      user = Users.get_user(user_id)
      render conn, "login_success.html", user: user
    {:error, reason} ->
      render conn, "login_failure.html", reason: reason
  end
end

Now suppose that a request has come in to the Web interface, and we are inside a Phoenix controller, wanting to handle it. The request is for a “view flights” page. To render that page, we obviously need to obtain the flight information. As you said, this information will be different for logged in and non-logged in users, but we don’t care about this at the Web layer, as long as we can render a “hidden” flight (e.g. by showing “sign up now to see this deal!”)

def show_flights(conn, params) do
    # extract the fields we care about, in a format which `Flights` is expecting
    flight_params = sanitize_flight_params(params)

    # we stored user_id in our session if logged in, if not, we use the anonymous user
    user_id = get_session(conn, :user_id) || Authentication.anonymous_user_id()
    flights = Flights.get_flights(params, user_id)
    render conn, "flights.html", flights: flights
end

Note that we don’t care at this layer even whether we are logged in or not. Even if we are, we simply pass the user_id that we were given from Authentication through to Flights. Note that at no point we’ve assumed what user_id actually is—it could be an integer, an atom, a string, and we don’t need to know!

Now, let’s take a look at the Flights domain itself. Its job is to return flights matching certain parameters. But, if a certain user (for example the anonymous user) is not allowed to see premium flights, they should be marked as hidden.

def get_flights(params, user_id) do
  params
  |> fetch_flight_data() # local helper function
  |> Enum.map(fn flight ->
    if premium_flight?(flight) && Authorization.cannot?(user_id, :see_premium_flights) do
      %{flight | hidden: true}
    else
      flight
    end
 end)
end

Again, the Flights domain doesn’t have to care what users actually are or how they work. All it needs is some token for the user that we are returning flights for (user_id) so that it can ask the Authorization domain whether this user is allowed to see premium flights. Likewise, Authorization doesn’t need to know what the :see_premium_flights permission actually means—its only job is to store whether a particular user has that permission or not.

Hopefully that example illustrates what I mean. Also, notice that no domain apart from the web interface actually knows anything about sessions or request. You could easily execute the code inside that controller inside iex and obtain the same results.

Finally, this is only one way to split up the responsibilities across the domains. You may decide that actually Flights shouldn’t know about the fact that some users are not allowed to see premium flights, and instead move that code to the controller itself.

Indeed, if you spread data across two databases, you do have to solve that hard problem. I’m not sure why you would though or what it achieves here. It is ok to add referential integrity checks at the database layer, without coding them into the software layer. Suppose that your Authorization permissions lived in a separate table to main user data, and your user profiles were in a separate domain Users. It’s fine to have a foreign key in the auth table to users, because without a user, their authorizations are meaningless. However, at the software layer, you’ll likely model that field in an Ecto schema as field :user_id, :id instead of belongs_to, :user, Users.User, because the second violates your domain at that layer. As I explained before, it is absolutely fine for domains to be passing around and storing opaque tokens belonging to other domains, as long as they don’t peek inside.

This entire thing is a spectrum and as is often the case, extremes have many problems.

You don’t have one system. You have multiple systems, and you’re trying to integrate them as external systems and wondering why you aren’t getting the benefits of having one system. As I said, usually the web layer is just an interface to the system—it is not a separate domain.

Your CMS and ECommerce domains do not care whether they are being displayed with a live preview or JS libraries. For all they know, they could be called from the command line.

You can achieve different resources being loaded for different sections of the site in other ways, but the beauty of the model we are explaining is that you can make these decisions after the fact. Because your application is completely separate from your web interface, you can have all of the modules be accessible through one huge website, or run a few smaller websites as separate Phoenix apps, all calling into the appropriate domains. You can even have multiple Phoenix “apps” being handled by one Cowboy server with a properly-written plug.

Yes it should.

No, it shouldn’t. It should be able to return a list of products or store pages or categories as a bunch of Elixir data. It’s the job of the web interface to turn that data into HTML that users can see.

Indeed it would be difficult, this is the way that people tend to write apps in Rails and this is exactly what we are arguing against :wink:

Separating your application into domains is again the solution here. If lots of people are searching for flights, the bottleneck could be at your web interface (in which case spin up more nodes with the web interface running), or the bottleneck could be with your flights backend (sorting through and processing all those flights).

Suppose that to date your whole application has been running on a single server, and suddenly you find that it can’t handle the load of processing all those flights. Because all of Flight-related functionality is within a single domain, you can change its internal implementation to actually work with a cluster of nodes behind the scenes to spread out all the work, and as long as you keep the same public interface to Flights, you should be able to get away with little to no code changes anywhere else in your entire application.

The entire point here is decoupling where it makes sense. This includes decoupling access (web) from the system itself.

The default model “encouraged” by Phoenix 1.3 is not strict DDD. It may adopt some concepts, but the main point is Phoenix is not your app.. Build your applications as a bunch of modules and then add a web interface on top. Keep the modules self-contained where practical and feasible. You could do proper DDD in Phoenix, but just following these guidelines gets you really far without the overhead that applying a strict model entails.

mjadczak

mjadczak

I don’t see how this is a violation of boundaries—if you have an app whose purpose is to handle authentication and authorisation, you could write a clean interface there and it would absolutely conform to DDD. The Auth app does not necessarily need to “know” about users past e.g. their emails or IDs.

So you could have an interface that lets you, say at signup, say to the Auth app “the user with ID 78 is allowed to buy products”. On login, you could call into the Auth app with the credentials, and if correct, it could give you the user_id (to be used with the Users context) as well as perhaps some sort of identity blob. Later, e.g. in an ECommerce controller, you could have a plug which grabs this identity blob (representing a logged-in user) and ask the Auth app if the user is allowed to, say buy an item. If not, you take appropriate action.

Another thing to consider is whether you actually want to separate your frontends into two apps. It is totally valid to have three apps, one managing ECommerce logic, one managing CMS logic, and one Phoenix app being the frontend to both. I like to separate things into their own modules first, and only if it becomes clear I really am running two independent systems do I refactor those out into separate apps in the umbrella.

One big thing which I’m not sure has been mentioned, and which is really different from Rails, is that Phoenix is not your app. When people say “app” in the Elixir community, they often mean an OTP application, which is just a bunch of modules and processes running together, often under one supervision tree. The advice you’re being given is to create your business logic in separate apps without a web interface, i.e. you should be able to run your app from iex directly (also great for testability), and then you can put a Phoenix-powered web interface on top of this, where the controllers simply call the appropriate business logic functions.

Last Post!

dsignr

dsignr

Absolutely. I came across this fowler article too late in my 2 year journey. But let me make this clear - investment of time reading into DDD is not wasted at all, for DDD != Micro services. DDD will help architect your application with sensible logic. There are many good books on DDD, but almost all of them are too verbose and over 300+ pages to even get started, the one book that I personally found useful was this. I tried almost all books on DDD and this is my personal recommendation.

I use www.draw.io to do all my DDD work. Not sure if there are better tools out there specifically for DDD. At least on Mac.

Cheers.

Where Next?

Popular in Questions Top

New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New

Other popular topics Top

JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1144 55125 245
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
AstonJ
Posting this to see if we can make things easier for people to get into Neovim. If you use Neovim and have a favourite distro please let ...
New

We're in Beta

About us Mission Statement