ElixirConf 2023 - German Velasco - Using DDD concepts to create better Phoenix Contexts

In the video he discusses this at 26:48. He wraps the 3 bounded contexts together in a specific “use case” function at the app layer. He passes an event data structure through these contexts, an event has the minimum amount of information necessary to avoid coupling the contexts together.

Much of the discussion here seems to be about the problem of composing contexts? So what do we think about the approach given in the video?

Screen Shot 2024-03-20 at 4.08.14 PM1200×709 26.2 KB

1. Yes, DDD is overkill for a small/simple project. But today, even a simple indie SaaS project may use a dozen external services for generic and/or supporting domains. You can still use context mapping to map all of these dependencies.

2. Some people in the Elixir community use DDD strategic and tactical patterns simply to better organize their code. This makes it easier to maintain/change.

3. Communication between BCs is carried out through pivot events. It can be implemented using event sourcing or durable transactional messaging (e.g. outbox pattern + message broker).

4. You use orchestration for workflows within BCs, and use choreography for workflows between BCs.

I didn’t imply there was a single database but that a strong binding must exist between domains. Where things are in a single database we absolutely should use referential integrity and where we have disparate systems then we may use various auditing processes and data pipelines to aggregate data to validate and detect data quality issues and other anomalies.

A car dealership is hardly involved in manufacturing and engineering the vehicles they sell but they certainly may use multiple systems vs a single integrated system to run their business.

But so what, we are asking how does DDD help us build an Elxir application within a project scope, not solve every business problem.

When we are building a typcial elixir or phoenix app with a database I don’t see that DDD has a place beyond what I’ve already stated, it can help to understand the business problem domain using business language.

Bounded contexts don’t magically solve anything nor do they co-ordinate cross domain flows because ideally they are not supposed to couple. So the coupling flows must be either pushed up to an integration layer above the contexts, or you just say stuff bounded context ideals and build the necessary integration flows from the context that needs to drive the co-ordination.

I think there are some techniques we can use here to minimise the coupling from one context into another but I’m not seeing it coming from DDD as there hasn’t been a good example presented on how we can minimise the cascading footprint of change when something in one context needs to change whilst also meeting application cross domain flows and coordination.

So my view is for 95% of elixir database apps just go do the best you can and not get hung up on things like DDD, because ultimately that’s all you can do, DDD doesn’t provide a magic bullet and in most apps it looks like a huge time wasting distraction that may result in weaker database models and less referential integrity if you take bounded contexts thing too far.

When you have millions of paying customers you can then worry about reducing coupling because you will understand the problem much better you will also have multiple systems and you will have the flexibility and cash to navel gaze and get less done.

That’s exactly why I spoke of engineering and sales, not not two departments involved in delivering one and the same physical car. Cars build by engineering (as in creating new models of cars) don’t really share much of anything with a car eventually being build for consumers to be sold. It might even be completely scraped.

Also while I never worked in such a domain from what I know about car sales I’d be surprised if they actually have a single system with referencial integrity over all the things you mentioned. There might be multiple distributed systems sharing common identifiers like serial numbers or manufacturing numbers, but those systems don’t use a single shared atomic and transactional database. They depend much more on compensation actions to handle issues than making sure everything is correct everywhere everytime.

E.g if somehow a car is sold twice they either build another one and change the order of one customer. If that delays delivery they appologise and maybe give you a discount.

Would a single database with referencial integrity be nice to have – sure. The company certainly would like to not need to give out those discounts. But there’s many tradeoffs involved in distributed systems and large enough companies are almost by definition running as distributed systems.

That’s fine but they still need something to relate those different car “types” or facets, perhaps it’s a natural key or perhaps it’s a database key but there needs to be a strong binding to ensure that you are referring to the same car across contexts.

And right there we have a coupling because in reality you need that strong binding to trace a car sold for accounting, stock control, compliance, state vehicle registration, manufacturer warranty, maintenance and product recalls, retail finance and insurance.

The whole thing collapses without the coupling across domains and referential integrity is still essential.

I accept DDD as far as it goes for business expression and understanding the problem domain however when it comes to solution domain and bounded contexts we need to recognise what DDD trying to do vs what it can do with its notions of “good” and “not good”.

The only principle in DDD and one generally understood long before DDD is that we should aim for minimal coupling by ensuring bounded contexts or sub-systems know as little as possible and ideally nothing about each other except where they must.

Well that’s wonderful.

That necessarily begets an integration and orchestration layer that must have the coupling to provide the actual cross domain flow that happens within a business.

Every time I bought a new car the sales guy checks the stock inventory and he’s in the sales domain. Then when I order the car and pay my deposit the fulfillment/delivery/service team need to prepare the car, remove all the packaging that new cars come wrapped in and install any ordered accessories. Their delivery team don’t magically work this out, the sales guy allocates the stock and puts in a work order so the delivery team can prepare the car ready for the customer. The business needs to make damn sure it’s the same car I ordered across all their business functions. These are all cross domain flows that “break” the naive notions of BC “goodness” and coupling.

So I’m left with the conclusion that DDD is not really providing much value in the solution domain, perhaps other than some repackaged awareness of various design considerations and tradeoffs but it’s the same tradeoffs and design challenges we have always had before calling it DDD and it I can’t see that it has actually provided any profound advantage outside of describing the business problem domain.

There’s already a lot of interesting discussion here.

In regards to the initial question of “why does DDD promote the a lack of referencial integrity”:

I think this starts at the wrong end. A more useful exploration would be: Does the business you’re working with have the need for referencial integrity between the contexts in question? Does the real world work with shared data or does information in the real world flow from one set of information to another set of information.

Consider a car company. There’s engineering designing cars and there’s sales selling cars. The “cars” those two departments think about are completely separate things. That’s what bounded contexts are about – separating engineerings “cars” from sales’ “cars”. Sure at some point e.g. an engineering sample might become a car being sold. But that’s generally better understood by “oh this is no longer a car engineering works with, but it’s now a car being sold” – a deletion in the engineering context and a creating in the sales context.

I often feel like people try to apply DDD at a scale where this kind of issue doesn’t come up in the first place. If a webshop can be dealt with in a single db with all the referencial integrity possible that’s fine. That however doesn’t mean that model scales to a company with many different departments all dealing with kind of the same core product, but all interacting with it in vastly different ways, contexts, lifecycles, …

I think this distills a key part of the concept and I agree that we should seek to isolate mutual knowledge between BCs thereby reducing coupling.

I see events as a possible way to solve this, as we avoid a sprawling globally shared data model. We instead provide the minimal event data to a bounded context with its minimal data models, it does not know about any other BCs it only knows about its own domain model and the specific events and actions it handles and the events it emits, almost analogous to the isolation and boundary layer of an Elxir/Elrang genserver process.

I am however still unsure how cross business domain communication and coordination is reflected and embodied in BC’s given ideally BCs should be close to 1 to 1 with the business domain but often aren’t due to valid design tradeoffs.

We know that businesses do communicate across their domains though systems so how do we bridge the BC’s without them having knowledge of each other?

There still appears to be a need for some kind of an orchestration or coordinator role that mediates events between bounded contexts, eg as a result of listing a product this may need to trigger processing in other bounded contexts, such as in my example, a promotions BC would receive an event about a new product listing because we listed a product in one BC and some coordinator/orchestrater now must ensure an event or action is delivered to the promotions context.

In which BC should this coordination live?

Does DDD have some recognition of a cross BC coordinator function or is this some kind of “union” context that is used to bridge between two or more BCs?

I just struggle with the idea of bounded contexts being isolated when they never can be because there is always a real business need for business domains to co-ordinate and participate in cross domain workflows. Therefore solution space bounded contexts must also couple in some way. E.g we may use service management and ticketing systems to mediate these cross domain workflows, but hook or by crook every domain must interface with at least one other domain in the business either directly or via some intermediary system.

So it seems the crux is really about asking what is an appropriate coupling because BCs need to couple because business domains couple through workflow coordination whether it’s system supported or manual processes.

Therefore I think we have to ask what coupling approaches in our Elixir programs are minimal, reduce fragility and provide the maximal agility for least reengineering effort.

That’s a lot of hard-won experience packed into a forum post! Thanks for sharing it.

Brilliant post!

Thanks very much for such a considered response.

I’m quite pragmatic about these kinds of things. And I definitely don’t like the word forbidden. Discouraged is can agree with.
But if something is discouraged we also need to know why it’s discouraged, and not just because someone or some practice says so.

Expanding on the discussion of yesterday

So let’s first look at a simple, imaginary example, but with some inspiration from real world situation that I’ve seen.

Imagine after a discussion with 1 domain expert, we came up with the following design:

image867×369 16.8 KB

(so a product has exactly 1 warehouse)

We implemented this and have a database with some constraint that a product must have a warehouse id.

Now a bit later we go talk to an other domain expert who’s main expertise is adding new products in the system.

We show them this picture and explain what it means

Sidenode on technical diagrams

The above picture is trivial to explain, but often showing technical UML diagrams to domain experts is not the best way to engage with them, because they’re often not familiar with this notation. In DDD we have some other techniques like EventStorming or Domain Story Telling to discover the domain, talk with domain experts, …

An example of the same knowledge but captured with EventStorming might look like this:

image502×212 20.6 KB

(blue = command or action, pink = constraint, orange = event or decision)

This notation is really easy for anyone to understand, we can read it like a sentence: when we add a new product, we check that the product has exactly one warehouse and if that’s true, the product will be added in the system.

Back on track,

So we discuss this with the domain expert and they immediately say, that’s not true. They explain

Often when we need to add new products, we don’t know in which warehouse we’ll store it. For some products we do, but for others we don’t know that yet.

“That’s interesting, but I guess that these products can’t be sold on the website yet.” I reply.

Sometimes these products will be shown on the website. People can’t buy them yet, but they can pre-order them.

The conversation continues for a bit and we go back to our design.

As we’re not doing any DDD and we feel that a refactor of the database constraints would be hard we propose the following solution:

If we don't know the warehouse, we'll add 9999 as warehouse_id

A small conclusion

This is how we start to add accidental complexity in our code base. Sure if this is the only thing that we do, maybe it’s ok.
But if later on the rules change again (we now have products that will never have a warehouse as they’re being sold by 3rd parties - we add 9998 as warehouse_id in that case), you can see that our system can quickly start adding a lot of accidental complexity, because we didn’t refactor to the actual business needs.

Is this forbidden?
No, but I’d discourage a design like that for a couple of reasons.
First, the upside of this design: it’s really easy to add it in at right now.

But, the downsides long term are huge.
These kinds of “hacks” accumulate quickly and the more you add in, the harder they are to refactor out. I’ve seen a lot of legacy systems that have columns with magic values like that. Figuring out what they do as someone new is often hard work (hopefully the meaning of a magic value doesn’t depend on an other magic value in a different column )

Even for people who’re experts in the software, this design significantly increases the cognitive load for them, because everywhere they use the warehouse_id they need to take these things into account.

Refactor towards deeper insight

The above design is obviously flawed and it’s a great way to start building systems that after a while no one dares to touch anymore.

In DDD when we encounter such flaws in our model, we want to refactor towards deeper insights. The downside of this is that it takes time, but the upsides are that we now should have a model that aligns much closer with the actual business needs. Thus being able to serve the business better and faster in the future.

Back to the original questions: BC communication

Communication patterns between bounded contexts is something that’s discussed with Bounded Context maps and bounded context patterns. (a great talk on this topic is: https://www.youtube.com/watch?v=k5i4sP9q2Lk)

So in DDD I’d say we don’t forbid or discourage things necessarily, but we try to find out what patterns would work best.

We often try to limit the amount of knowledge 2 bounded contexts need to have about each other (both domain and technical knowledge).

How much knowledge depends on the business requirements (something we don’t have a lot of influence on) and how we design the system (something we can influence hugely).
Bounded context patterns define what kind of coupling relationship there are.

The hardest type of coupling is called a “shared kernel” in DDD, that’s for example 2 bounded context sharing a database, or sharing a domain software module.

It’s often said that this pattern is an anti-pattern in DDD, but it’s still a pattern that’s widely used. For me it’s important to that when you pick a pattern like that you know why you do it, and what the upsides and downsides are.

If you’re 1 team working on 2 bounded contexts that have a shared kernel, you’re not going to have a lot of downsides.
Once you split the 2 bounded contexts across 2 teams, you’ll need a lot of communication between the 2 teams to make sure that you don’t break each others code by changing stuff that impact the shared kernel. For example, you share the database I talked about earlier, if 1 team decides that warehouse_id 9999 means no warehouse, the other team needs to be informed up front about that because who knows what they rely on.
And it’s like that for a lot of choices you make.

Something else to take into account is how fast or slow is your software changing. It’s easy to keep a shared kernel in a slow changing part of the software, but a lot harder if it’s part of your core domain that you’re working on all the time.

And finally there are differences even when talking about a shared kernel. Sharing the full database scheme is obviously tighter coupled than just sharing 1 table for communication.

So, is a shared kernel bad? It depends, and it depends on some of the things I just described.

In your example:

Doing the upsert is an example of the shared kernel. They share a database.
Changing it to an oban worker triggering an api call (or public module function call) would reduce the coupling.

Sending notifications via pubsub, depending on how it’s implemented I would also call that a shared kernel. But promotions could also expose a public interface that you need to call when you want to notify someone, and that reduces the coupling again.

(there’s a lot more nuance, but hopefully you can see what I mean)

Like I just explained, yes it can, we’d call this a shared kernel. It certainly has upsides (like the simplicity of the single transaction), but there are downsides as well. It’s up to you to decide how they balance out.

To conclude. Nothing is forbidden, but some things are discouraged because we know that there are patterns that will make your software hard to maintain and to evolve over time.

If you’re building software as a small team, reducing coupling, splitting into BCs or services, also has a cost. It’s up to you to evaluate what’s important and how to design the software.