Hexagonal architecture in elixir

Yes our ports return only KairosDomain plain structs. Structs that are some quite similar, some loosely and some quite different from the schema.

All the reste of the functions inside the command modules and domain modules do as if saving those struct was easy, use the correct port, and all is well.

The structs that end with Table are Ecto Structs, those with Entities are Domain Struct. (Our naming is like, Entites are readonly struct, Aggregates are the state-modifier structs)

# From KairosInfra.RecurringShiftEntityAdapter module

  alias KairosDomain.RecurringShiftEntity
  alias KairosInfra.RecurringShiftTable

  def from_recurring_shift_table(%RecurringShiftTable{} = recurring_shift_table) do
    %RecurringShiftEntity{
      id: recurring_shift_table.id,
      last_shift_generation_date: recurring_shift_table.last_shift_generation_date,
      timezone: recurring_shift_table.shift_service_request.warehouse.timezone,
      recurrence_pattern: adapt_to_recurrence_pattern(recurring_shift_table),
      recurring_template: adapt_to_recurring_template(recurring_shift_table)
    }
  end

# from KairosInfra.RecurringShiftEntityRepository

  def get(id) do
    result =
      RecurringShiftTable
      |> where(id: ^id)
      |> preload(
        shift_service_request: [warehouse: []],
        recurring_provider_assignments: []
      )
      |> Kairos.Repo.one()

    case result do
      nil ->
        {:error, {:resource_not_found, [name: :recurring_shift_entity, id: id]}}

      recurring_shift_table ->
        {:ok, RecurringShiftEntityAdapter.from_recurring_shift_table(recurring_shift_table)}
    end
  end

Those are one example with a loosely similar domain struct from ecto struct. Yes there is an adapter, yes the repository is not a plain Ecto function, but we gain a better error message, and writing the whole thing is like 5 min testing included. The longest part is creating the base data for the test factories. The rest is copy paste.

Why we did that on that particular struct, is more about consistency. We needed Ecto separation for one, we decided that it would be the way for all. (But as we refactored stuffed as we needed, I guess some stuff will never migrate as they never will be modified again…)

Is that needed for every project, no indeed, I do not think so either. In the startup I am working with right now, we have three different backends, and I see a moderate benefit in changing the architecture for one part of one of the three. What I am hopping is extracting that part into its separate app and going hexagonal on that one. And that isn’t very high on my to-do list

Hexagonal and clean archi are a nice clean way of organising code, extensible and easily maintainable, but expensive to migrate and in my opinion, useful only on a long and big project, or for complex business critical applications.

Likewise – I agree with most of what you said and I like your balanced approach!

One thing I’d grumble about if I was code-reviewing is this:

I am aware there are cases where Ecto structs aren’t enough but I remain sceptical as to how many projects actually need something outside of that. Modelling future datetimes is indeed a notoriously painful problem so you are likely correct that you need to step outside Ecto.

What I did in only one Elixir project ever was that we had one module with business context functions that tackled conversions exclusively: they received Ecto schema structs as input and returned plain Elixir structs as an output (and then vice versa, almost: the outputs were Ecto.Multi structs, not Ecto schema structs). And then all other business functions only operated on those plain Elixir structs.

Is that what you ended up doing (if I am reading you correctly)?

This sounds like a place where trying to map the domain model to the ecto model is no longer a lot of additional complexity, but complexity inherent to the problem domain. This is in my opinion much more worthwhile than trying to separate domain model from the ecto model, when both essentially look the same and one would effectively duplicated code.

I like this thread, and I agree with most of what I read.

But if you are talking about Hexagonal Architecture, you do have to abstract the database mapper layer. Or else you have a conceptual leak into your domain. The database and Ecto are flexible and powerful tools, with lots of possibilities, but they do force you do model things a certain way.

If you have simple responsibilities, and simple relationships between your objects you might not need the decoupling between Ecto and your Business Rules. If your Business Rules are not too complex, Phoenix Contexts are a good start, and do go a long way.

As a matter of fact, in my case, knowing all that, we did start with Ecto and basic Phoenix Contexts, and it proved to be not enough. The database modelling was too limiting, the business rules were too big for contexts, and testing started to become horrible (think inserting tens of objects every time because it was the only way to have a correct state in the database. We had factories, but not enough, not conviennent, and increasingly brittle…). So we chose to migrate to a more flexible architecture.

I do not suggest to anyone to start with a complex and over-engineered architecture like the Hexagonal, in elixir from the start. Indeed, as it was said, you most certainly do not need it. And if you do not it will slow you down. It is a compromise : do you win more time having the freedom to model your business structs and rules as you want, or do you loose more time writing converters between the layers ?

That is not what you do, we had methods named quite explicitly save ou delete etc. But the persisting was juste not represented as manipulating Ecto, and not with Ecto.Structs. Because Ecto and Ecto struct limit your modelling and testing possibilities.

For context, we had to create a calendar (quite a simplification, it was not juste google calendar, but good it is a good enough analogy) but design an ecto struct that represent an infinite recurring series of events on flexible patterns was very, very painful. And that is why we changed and went for the hexagonal archi.

I know. Recently I kind of drift away and side-step topics. That’s on me.

Apologies for that but I still hope you don’t mind the side discussions every now and then.

Yep, completely agreed on that. I also said above that I don’t get the huge efforts to pretend that we’ll never persist anything. And, tests are an additional code base that has to be supported which is a vastly underestimated development cost (and that code base is also usually more brittle than the code that it tests).

To be fair this wasn’t actually meant as a bashing of simple web applications. Web applications can bring business value just by presenting data in a more practical way than a bunch of tabular sheets and without any fancy stuff on the backend. What I wanted to say is that in such cases people need to acknowledge the fact that their core business domain is moving data in and out of some database (airtable is just another db tbh) and there’s no sense in trying to abstract away the fact one is dealing with some kind of persistance.

Reading @al2o3cr and @LostKobrakai made me think that in the rare cases where I care about isolation I just try and make sure that I don’t depend on stuff provided by ecto_sql (and even that can be very hard as @LostKobrakai pointed out). Depending on ecto itself is completely fine – because it does not deal with a specific database.

Leak-less isolation is IMO a pipe dream. Sure it might accelerate your tests but at what cost? Eventually you’ll fail accounting for something that only a live database does and then your tests become more or less worthless. Which is what a lot of unit tests out there are anyway – just a wishful thinking of the dev.

As for the so-called business transactions… meh. Just use Ecto.Multi and stop sweating about it. Ecto is a fantastic and well-tested library. If you go to such levels of paranoia you might as well ditch the interpreted language and VM concepts altogether and go lower-level: to C++, D, Rust, OCaml, Nim, Haskell etc.

LOL, hits too close to home for the teams I’ve worked with in my six years of Rails work. People just love to pretend that they can swap any part of their app anytime they want – including when this part is a hugely important piece without which most of their app wouldn’t be anything more than a student exercise (since, you know, real-world apps want to persist data).

I get the motivation but IMO a lot of teams get carried away and forget where to draw the line.

Now that’s a fact (it’s “glorified” btw, not “glorious”). I’d bet a few beers that a lot of apps out there can be rewritten in any language with an Airtable connector library and that they might work even better compared to the originals (and I include Phoenix app in this category as well).

Web frameworks are in general supposed to bring additional value over the basic “HTTP router to controller code with some views and templates sprinkled in” but many people stop at that point which turns into an epic fight that boils down to “how do we put this square peg into the round hole”.

As I get older I start finding myself thinking how do I make web apps with almost no code and just clever integration between several services. It’s possible, actually.

I’ve seen those goals accomplished simultaneously in Ruby before - it led to a base layer of ActiveRecord models (with a scorching case of AnemicDomainModel because the classes were glorified DTOs), an intermediate layer that mostly just copied ActiveRecord data into plain Ruby objects because Abstraction, and then an ever-widening layer of module functions on top that used AR functions to get results and then translate them to POROs.

The phrase “in case we want to swap out ActiveRecord” was said at least five times a day.

IMO there are two overlapping reasons to not worry so much about “wrapping Ecto”: if the application controls the database (via migrations etc) it’s not really the same thing as the “external dependencies” that need decoupling against unexpected changes, and Ecto is already essentially an “adapter pattern” on top of the raw database layer.

Right, siloing Ecto into the infrastructure portion of your application makes it difficult to take advantage of Ecto’s powerful API.

Yes, this is what I meant.

“no Ecto leakage outside of the infrastructure” and “We never found a good way to take advantage of the power of Ecto’s API” sound like a contradition. How would you expect to use ecto api, but not “leak ecto”?

Abstracting the actual database integration away is a good thing. Abstracting knowledge about persistance away I’d say is a fool’s errand. Especially for all those webapps, which are essentially just glorious excel sheets. It makes no sense to have the core of an application be unaware of persistance, when all it does is manage data going in/out of it.