Ecto.Multi nearly a monad

When with was proposed, one of the possible additions was transactional with, which is pretty much what you proposed. The problem is that with/for are compile time constructs, Multi works at runtime which is quite more flexible.

Regarding the “after” callback, we should add a new operation to the multi that receives the result of the Repo operation {:ok, ...} or {:error, ...} and return either {:ok, map} | {:error, map}. Can you please open up an issue? I would like to hear yours and @michalmuskala’s feedback.

Do you have a concrete example of some benefit you get because of this flexibility. I’m not 100% sure I understand the comment.

To clarify, the approach you proposed still classifies at runtime for me because you are still building a multi data structure that you pass around at runtime.

If everything was syntax base, imagine something like:

Repo.transact_with x <- ..., y <- ... do

things such as conditionally adding something to the Multi would be really hard. It would be the equivalent of conditionally adding an <- to the syntax, which is not possible, or you would need a way to express noop then. It would be hard to inspect the operations in the multi, compose through multiple functions, and so on.

I’m thinking about extending multi for some time now, and I have some ideas. I have to say, though, that this is not the area I was exploring.

My main thinking was in expanding the Multi abstraction and building something lower-level that could be later used by ecto to build Ecto.Multi on top. Some of those ideas are explored by my friend @AndrewDryga in his library Sage.

Looking at your example, I’m not sure it’s really different from doing something like (besides some additional verbosity with the explicit rollback and repo):

fun = fn ->
  with {:ok, command} <- Repo.insert(command_changeset),
       {:ok, event1} <- Repo.insert(%{event | command_id: command.id}),
       {:ok, event2} <- Repo.insert(%{event | command_id: command.id}) do
    IO.inspect(event1)
    IO.inspect(event2)
    command.id
  else
    {:error, reason} -> Repo.rollback(reason)
  end
end

case Repo.transaction(fun) do
  {:ok, command_id} ->
    # stuff
  {:error, reason} ->
    # different stuff
end

The problem is of course, that in this situation (and in the proposed syntax) - we don’t really know which operation failed. Sometimes that doesn’t matter, but sometimes it does - the current multi provides the information at the expense of requiring an explicit name for every operation.

I’ve definitely wanted this in the past as well. Would be quite useful for things like keeping an external system in sync with the database.

So, I’m not really sure doing things like that inside multi is a good idea - this means that for the duration of the remote communication the transaction is open and connection tied up.

I use Ecto.Multi in my own CQRS/ES Elixir apps, but for read model projections rather than persisting events. In my opinion it’s a really nice fit.

I built a project/2 macro in my Commanded Ecto projections library to provide a DSL for projecting events that uses an exposed multi variable:

defmodule MyApp.ExampleProjector do
  use Commanded.Projections.Ecto, name: "example_projection"

  project %AnEvent{name: name}, _metadata do
    Ecto.Multi.insert(multi, :example_projection, %ExampleProjection{name: name})
  end
end

You should consider persisting the command separately to the events as this will allow you to record failed commands such as those returning an error, raising an exception, or just not creating any domain events. For Commanded I built support for command dispatch middleware and have written an audit middleware that records every dispatched command, it’s success/failure outcome, and execution duration using Ecto.

It’s also worth using causation and correlation ids to track the flow of commands and events in your app.

• causation_id - the id of the command causing an event, or the event causing a command dispatch.
• correlation_id - an id used to correlate related commands/events.

They allow you to correlate related commands/events and look at causaility chains, useful for debugging purposes.

To persist events to Postgres I created the EventStore library. It uses a multi row INSERT statement to append a batch of events in one query and also assigns each event a globally unique, monotonically incrementing, and gapless event sequence number.

For more resources on the subject I’m compiling an awesome Elixir and CQRS/ES list.

I thought that this is would be an “after” callback and will be called once the transaction is complete. Is that incorrect?

Of course, here the success tuples are used as (explicitly matched) optional/maybe type.

As far as my current understanding goes, Ecto.Multi is not a monad because I cannot think of a sensible way to implement wrap and chain for them.But yes, their return values have a very clear succeed/fail scenarios which you could expose using explicit with-syntax or semi-explicit ‘monad do’-syntax. But the monad here is the success tuple and not Elixir.Multi.

Yes, it should be done after the transaction.