Hexagonal architecture in elixir

Questions / Help
1

Background

For the longest time now I have been playing with the idea of doing an application that follows the hexagonal architecture:

After reading several books on the matter, this looks like something that everyone should do. It is hard for me to find a compelling reason to not use it given that you are writing any code with a decent amount of complexity (don’t use it for an Hello World app, obviously).

However, ironically, I have never found, in all my life as a programmer, any project using it. In fact, none of my colleagues I have (or ever had) even knew about it.

What now?

So, now I have decided with my free time, to create a pet project where I can implement this architecture.

This project should be simple: It is a command line app, that makes HTTP requests to an external website.

There is no database, authentication, no nothing. Just invoking the app:

# makes a hello world search in google and IO.puts the result
./my_app --greet="hello world"

So, out of the box, I know I need an adapter for an HTTP client (lets say, HTTPoison) and a JSON decoder (lets say Jason) because I want my app to be able to change between decoders.

Questions

And this is where I freeze. There is so much stuff in my head, I can’t even start.

How should I implement the port? Via an interface (module with callbacks) like Jose Valim in hix Mox article?

http://blog.plataformatec.com.br/2015/10/mocks-and-explicit-contracts/

What should that interface be like? Have GET and POST methods or have a “greet” method?

What about my adapters?

It really feels like that although I have read extensively about the topic, I am incapable of processing the information into something useful.

A code sample would really help.

Has any of you ever did something similar to this in Elixir?

2 Likes
2

Very much the latter - GET and POST are concepts of the outside world, not the core (unless you’re modeling an HTTP proxy or something).

One of the greatest challenges with an approach like this is distinguishing exactly which side of that boundary things live on, especially early on when there’s exactly one implementation of each interface.

Destroy All Software has a popular writeup “Functional Core, Imperative Shell” that talks through a lot of similar tradeoffs:

https://www.destroyallsoftware.com/screencasts/catalog/functional-core-imperative-shell

3 Likes
3

I am very familiar with his talks:

This is a very interesting discussion (IMO), you may want to check it :stuck_out_tongue:

4

I feel like this might actually make it harder than easier, because there’s hardly any business logic to speak of. There’s “CLI App” – which is the bad outside world – and “http requests” – which is the bad outside world – and nothing in between worth noting by the description you gave.

Try answering the question what your app does on it’s own without the outside world.

For a different example: a calculator. There’s the input of the calculation to be done (maybe string based), then the business logic of parsing the input into an AST of calcuations to be done and then the application of the calculations, which give you the result. This internal result is then converted back to a CLI based visualisation.

For something like this the boundaries are much more clear. The functional core is the parsing to ast, and the calculation itself, while the imperative shell is the CLI stuff of retrieving argv and turning the result into something you can print to the CLI (stdout), possibly adding CLI specific coloring tags and such.

5 Likes
5

Nothing. Does this mean my app can’t follow the hexagonal and functional architectures?
I guess I could read some translations from a file and make a request for 3 greeting languages, but then again I would just be reading from a file and doing nothing else.

This is rather confusing … everything…

6

You still can, but the inner part will be empty. The whole app will be interacting with the outside world on either end and just some glue code in between. Your app basically doesn’t have a core, as it’s doing nothing with the data it handles of external resources.

3 Likes
7

Do you happen to know where I can find a good example of an app in Elixir that uses this architecture?

Anything really goes at this point.

8

Last year, I am pretty obsessed with the Hexagonal architecture and trying to implement it in Phoenix. I learn that if your application has a portion where there are some logic to it (i.e. not CRUD related), then hexagonal architecture makes sense.

For example, think of a Todo application that give you analytics on how many tasks you have completed every day, and even forecast how many tasks you will complete tomorrow given the current productivity.

The code to tabulate the tasks completed everyday, and the forecasting logic for tomorrow is something that make up the core of your hexagonal architecture. It can and should be shield from your HTTP and your DB layer. Ideally, the core should also be pure. It doesn’t maintain state. It merely just takes in data, and return the output.

However, most applications will have their fair share of CRUD operations. Can we use hexagonal architecture for this? I think it is possible, and I have tried out an example by defining my own mock Repo. But after some time, I think it is not worth the effort.

1 Like
9

This is exactly the reason why I feel the more trivial an application becomes the more difficult it’s to starting using an hexagonal architecture for it as one will constantly be thinking about all the added complexity without much tangible benefits. If all an app does is forward inputs to storage there’s no “core functionality” to speak of.

That’s why I suggested a calculator as a better example. It does actually have core functionality. This core functionality might be powered by a CLI interface, but could just as well be adapted to e.g. an http interface.

Tbh most phoenix projects use an architecture, where the http interface is already optional, as functionality is contained in contexts not in controllers. So there are lot’s of examples on that end. A good one might be nerves_hub, as it actually has both a CLI and a web based interface. There are few projects I know however, which treat their database as the bad outside world, which you’d need an interface/adapter for.

There’s a cost/benefit decision to take for keeping functionality hidden behind an interface and as most people seem to be controlling their db in their stack the benefits are not as great as e.g. for a third party api, where uncontrolled network and services are involved. For third party apis you often see them hidden behind interfaces even just to enable mocking. I found the unit testing book of manning to actually explore the space of internal vs external dependencies quite well. Most people only put dependencies behind interfaces, which they’ll also mock (Unmanaged dependencies* in the books terms), but don’t do that for things they don’t mock (Managed dependencies** in the books terms).

  • out-of-process dependencies you don’t have full control over
    ** out-of-process dependencies you have full control over
4 Likes
10

I have read that book not long ago actually :stuck_out_tongue:
My problem with that book is that it focuses on the Detroid School of TDD, instead of the London School of TDD, which I am now trying to learn.

It is somewhat confusing to me, most people I know of seem to prefer the Detroit school of TDD even though the London one is considered a gateway to functional programming due to its different set of practices. Elixir is seen as a somewhat functional language, so it should fit the second model.

That book also mentions the hexagonal architecture and its evolution, the functional architecture, which the author claims to to be so hard to implement it might was well be impossible, something I have already discussed in this forum and found to be not as he says (the author is simply not aware of patterns to solve the issues he mentions).

But this is another discussion for another time.

You wouldn’t happen to know any examples of calculators, right? xD

11

From the perspective on how you implement DI/Mocks/… there’s no difference between both. The London School just mocks out / puts behind interfaces almost any depenendencies and their tests are concerned with smaller chunks of code – units in the language sense a.k.a. classes/modules and not units in the sense of set of code of a single exposed functionality.

Take what people do for third party api’s (which both schools mock) and apply the same approach to all your other dependencies (db and so on) and you should reach what the london schools suggests.

12

I think you are suffering from analysis paralysis… or have too much free time. :003:

Filter everything you do through practical concerns first and foremost:

  1. Erlang/Elixir are not that great for CLI apps: slow startup time, potentially superfluous OTP baggage.
  2. Your tool is a simple relay. It has no functional core as it has been pointed out.
  3. Hexagonal is kinda sorta implemented already by Phoenix: it isolates the web stuff through Endpoint / Router and macros injecting code in various places so your code can be mostly purely functional (minus DB and session state). Additionally, you are expected to achieve Hexagonal architecture through contexts: your business code should be completely unaware of how the data is stored: it receives primitive parameters and/or structs, does something with them, and returns other primitive values and/or structs. It shouldn’t contain any calls to Ecto functions, for example. (Although I do make an exception for Changeset whose functionality has come very handy many times.)

Hexagonal is quite simple to summarise for everyday work even if that omits some important details: use purely functional code absolutely everywhere you can (no side effects: meaning no DB, no file changes, etc.) and then only use imperative code (i.e. the one that mutates state) where you absolutely can’t avoid it – like when you have to store stuff in the DB or change the user’s session.

As a general advice: when you have a practical goal you should get more done and think less until you arrive at the point where you feel a refactoring should be done because you find yourself fighting with your code when you need to add new features or fix a bug.

The process you are trying to follow now is setting you up to become an academic: you are overthinking, having an analysis paralysis and almost nothing will get done as a result.

4 Likes
13

As a father of a hyperactive baby who has to conciliate remote work with chores, taking care of him and working, while also having to do an active effort to keep my mental sanity from exploding after being confined for over 3 months, I genuinely wish “Having too much free time” was the case here :stuck_out_tongue:

Truth be told, I want to go on vacations, but if I can’t leave my home, then there is no point. No matter what everyone says, that physic guy who is charging you 50 bucks so your soul can have a nice trip to Yellow Stone and you can experience it is kinda ripping you off. Don’t trust shady psychics man! Everyone knows a soul trip to Yellow Stone should be at least half of that. :smiley:

I like to think this is the case. Makes me feel smart :smiley:

Funny you mention that because for a great part of my life I was one.

Anyway, I have a very quick and dirty app that does what I want. I am not trying to refactor it to fit the architecture.

14

I had a period when I studied Hexagonal, onion and other architectures and London vs Detroit schools. I find them all different ways to achieve the same thing: split business logic from implementation details.

There are two main benefits that I see:

  • your business code usually changes more often than implementation details, so it is a less mental burden to change it when it doesn’t touch DB, HTTP params and what-not
  • if you decide to change the database (a rather rare occurrence, but sometimes scaling requires this), stuff is less tangled, and you don’t break the business logic

What they don’t tell you is how are you going to pay for it.
Doing everything via the interface is another layer of abstraction. It is also easy to miss the rule and use the direct call in the code. Elixir doesn’t have interfaces. The closest thing is behaviours. They have quite a lot of boilerplate because it is a separate module usually in a separate file.

Plus, you need another layer of configuration for specifying which implementation to use (config.exs is a bad fit for that).

That’s why I like an approach described by Rafał Studnicki in this talk: https://www.youtube.com/watch?v=XGeK9q6yjsg He splits every use case into three sets of files: Model - pure business logic, IO - external dependencies like DB, and service - which coordinates. The rules are:

Models can’t use anything in IO or Service to not pollute business logic.
IO can use Models (mainly, if you read something from DB, you translate it to Model instead of returning raw schemas)
Services call IO and Model functions and can be the place to choose proper implementation (if there is one, direct calls are sufficient).

For projects I worked on, it is the lightest approach with the least amount of trade-offs.

There are, of course, trade-offs:

  • typically, the web layer can extract information from Ecto.Schema and use it (have you wondered how Phoenix.Form decides on which form it should use PUT and POST?), if you use this approach, you need to distinguish yourself
  • you define schemas for your DB and often define an almost identical struct for your Model

Any architectural patterns can pay off in more significant projects where you often jump between implementing different parts of business logic.

As for TDD with or without mocks. The SQL.Sandbox abstraction makes running tests asynchronously and in isolation so straightforward that not using real DB in tests seems wrong. There is no downside to doing it the simple way. I only use mocks for calling external services.

16 Likes
15

I worked for the better part of a year on a Phoenix Application that ended up being something like a GraphQL - DDD - CQRS - Hexagonal like architecture.

It started a small innocent Phoenix HTML server but well thing got complex very fast and we had to structure it better than with contexts, plus we ended up needing a separate front-end app.

I quit last month to work elsewhere, but it was a good architecture. I now work on a node.js mess, and I regret it every day.

I do not know what exactly where I can help, but here is where the architecture ended up when I left. (the app is named Kairos)

(I did leave out some very specific and legacy stuff for clarity)

/kairos_domain  <-- where the business logic lives
	/aggregates   <-- the aggregate modules + structs per DDD teachings
	/services     <-- other domain relatated logic
/kairos_command <-- the "write" part of the application
	/commands     <-- the commands dispatched from the GraphQL mutations mostly
	/ports        <-- the interfaces for the commands to interact with dependencies
/kairos_query   <-- the "read" part of the application
	/queries      <-- the queries dispatched from the GraphQL query resolvers 
	/models       <-- read structs
/kairos_infra   <-- the infrastructure
	/repositories <-- the repositories, getting data from the database, and adapting to domain aggregates
	/tables       <-- ecto structs
	/adapters     <-- the adapter modules to go from aggregates to ecto struct and the otherway around
	/dataloaders  <-- the dataloaders to read ecto structs with caching for better request with absinthe
/kairos_web     <-- the application part
	endpoint.ex   <-- phoenix endpoint
	router.ex     <-- phoenix router
	schema.ex     <-- absinthe graphql schema
	/schema       <-- absinthe graphql types
  /adapters     <-- adapters between mutations and commands, or between query models and graphql objects

Port were implemented this way :

defmodule KairosCommand.Ports.ShiftAggregateRepository do
  defmodule Behaviour do
    alias KairosDomain.ShiftAggregate

    @type uuid :: String.t()

    @callback get(id :: uuid) ::
                {:ok, ShiftAggregate.t()}
                | {:error, {:resource_not_found, [name: :shift_aggregate, id: String.t()]}}
                | {:error, term}

    @callback save(ShiftAggregate.t()) ::
                {:ok, ShiftAggregate.t()}
                | {:error, {:validation_error, [KairosDomain.ValidationError.t()]}}
                | {:error, {:resource_not_found, [name: :shift_aggregate, id: String.t()]}}
                | {:error, term}

    @callback delete(ShiftAggregate.t()) ::
                {:ok, :deleted}
                | {:error, {:resource_not_found, [name: :shift_aggregate, id: String.t()]}}
                | {:error, term}
  end

  @behaviour Behaviour

  @impl_module Application.get_env(
                 :kairos,
                 :shift_aggregate_repository,
                 KairosInfra.ShiftAggregateRepository
               )

  @impl Behaviour
  defdelegate get(id), to: @impl_module

  @impl Behaviour
  defdelegate save(shift_aggregate), to: @impl_module

  @impl Behaviour
  defdelegate delete(shift_aggregate), to: @impl_module
end

For the tests, in the test config you can specify a mock module (using mox for example, we used a fork of mox named erzats for the job) to use only unit tests. It simlifies immensly your life when your database model is quite complexe as was ours. You can than return whatever you want from the repositories, and not having to insert tens of ecto structs in the right order.

We did nice things with the command module, structs that represented the command, and a protocol implementation for the command handler. But it is a bit out of scope.

Feel free to ask any questions you want. I do not know to help you on your discovery, but would love to help :slight_smile:

8 Likes
16

I see you took inspiration in Commands and Queries. Perhaps you followed this pattern?

The way I see it, my code has a similar structure in the sense where I also make ports (the interfaces) and adapters (their impls) explicit and I also use behaviours to model interfaces.

I don’t use however the notion of Commands and Queries, Martin Fowler does mention it adds risky complexity so I wonder how you dealt with it!

17

The fact is we did not indent from the start to use Commands and Queries, but as we used GraphQL, it kinda appeared naturally.
GraphQL mutations are natural commands. It added a lot of clarity for us to separate mutation/command from queries. Because for the query part we had to use the dataloader lib, which has a very specific way of working.

This is the protocol used to dispatch the commands :

defprotocol KairosCommand do
  @moduledoc """
  KairosCommand is the modules in which the usecase / commands that define your applications
  actions are.

  All Commands should implement this protocol to ease the call and the formatting of the parameters
  """

  alias KairosCommand.Context

  @type result :: term
  @type reason :: term

  @doc "modify the builder data with the opts (keyword list of the params)"
  @spec run(KairosCommand.t(), Context.t()) ::
          {:ok, result}
          | {:error, :forbidden}
          | {:error, :unauthorized}
          | {:error, {:validation_error, [KairosDomain.ValidationError.t()]}}
          | {:error, {:impossible_action, reason}}
          | {:error, {:argument_missing, missing_argument :: atom}}
          | {:error, {:resource_not_found, [name: atom, id: String.t()]}}
          | {:error, term}
  def run(command_data, command_context)
end

And here is the command definition (one of the most simple command we have) and the protocol implementation for the command handler.

defmodule KairosCommand.DeleteOneShift do
  alias KairosCommand.DeleteOneShift

  @type uuid() :: String.t()

  @type t() :: %DeleteOneShift{id: uuid() | nil}

  defstruct [
    :id
  ]
end

defimpl KairosCommand, for: KairosCommand.DeleteOneShift do
  alias KairosCommand.Policy
  alias KairosCommand.Context
  alias KairosCommand.DeleteOneShift
  alias KairosCommand.Ports.ShiftAggregateRepository

  @spec run(DeleteOneShift.t(), Context.t()) ::
          {:ok, :deleted}
          | {:error, :forbidden}
          | {:error, :unauthorized}
          | {:error, {:argument_missing, :id}}
          | {:error, {:resource_not_found, [name: atom, id: String.t()]}}
          | {:error, term}
  def run(%DeleteOneShift{id: shift_id}, ctx) do
    data = %{shift_id: shift_id}

    data
    |> Chain.new()
    |> Chain.next(&verify_is_authorized(&1, ctx))
    |> Chain.next(&load_shift_aggregate/1)
    |> Chain.next(&ShiftAggregateRepository.delete(&1.shift_aggregate))
    |> Chain.run()
  end
  
  ... (private functions are missing for clarity)
end

The mutation resolver creates the command and the context (with user permissions mainly) and calls KairosCommand.run(command, context)

Having one way to call commands and a standard response helped us mutualise and standardise the mutation resolver logic, and thus clarify the web/http part of the application. Quite a bit of plumbing in the end, but it happened over the course of 6 months, if we had to recreate that plumbing from scratch that would suck.
(Error handling, especially the form errors were automatically filled from the domain logic, translated and filled in the react form. Quite satisfying. Quite a bit of work for something Rails or Phoenix give for free… But Ecto.Changesets caused quite a bit of headaches so happy it ended up only in the infra, and business rules validations are free from it)

7 Likes
18

@Apemb this is really awesome. Thank you for sharing!

I’m curious where you put integrations with 3rd party API’s if you dealt with that.

19

Thanks, I appreciate it.

Concerning 3rd party API, we had none when I left, but I guess it would be the same more or less that the repositories. But in a different folder in the infra.

It should be in the infra, as it is a “driven” part of the “technical details” - as in not Core Domain nor Command/Query logic.
The “driver” part being in the Web modules, which really should be named app. But Phoenix naming stuck.

For a GitHub API integration for example:

  • A port like Command.Ports.GithubService
  • The corresponding Infra.GithubService module implementing Command.Ports.GithubService.Behaviour

And the Infra.GithubService would be in the /infra/services folder.

3 Likes
20

Hello.
I’m working also with some kind of hexagonal architecture with Phoenix, NoSQL Databases, TCP endpoints, Crypto , where all those parts are adapter to some ports using behaviors.
One article which helped me is coming from Aaron Renner https://aaronrenner.io/2019/09/18/application-layering-a-pattern-for-extensible-elixir-application-design.html

Hopefully it may helps you

1 Like