Crowdhailer

Crowdhailer

Creator of Raxx

Ecto.Multi nearly a monad

I am writing an application that makes use of a CQRS architecture. In this application a single command may create multiple events. There are certain concurrency issues that are managed by database constrains and so writing the command and events should be done as a single action.

@michalmuskala
Thought I would try and elucidate more what I said about Multi on twitter. https://twitter.com/CrowdHailer/status/942712473718837255

So a query is something that can be run later. And to my mind multi is about extending this to writes + querys in a composable way.

Queries can be run against a DB using Repo.all etc. If there was enough information on the Query you could replace that with Repo.run. Probably the nearest thing we have to run is Repo.transaction given a multi.

Multi.merge is the analogue to flat_map (close enough for my argument) and once you have that you can merge with a for comprehension. What I would like to write is

multi = Ecto.Multi.for do
  command <- insert(command_changeset)
  event1 <- insert(%{event1 | command_id: command_id})
  event2 <- insert(%{event2 | command_id: command_id})
after
  IO.inspect(event1)
  IO.inspect(event2)
  command.id
end

case Repo.transaction(multi) do
  {:ok, command_id} ->
    # stuff
  {:error, reason} ->
    # different stuff
end

NOTES

  1. Picked this example because both events rely on the command so piped calls to something like merge are not necessarily the best solution
  2. insert is Ecto.Multi.insert but without needed to explicitly give it a name in the multi object. Is not necessary once you can compose by binding to the potential return values.
  3. The merging of command_ids is simplistic for illustration, it’s more likely you’d use a changeset.
  4. The after block would make more sense as yield but limited to Elixir keywords, it should be executed on success.
  5. I added the IO.inspect lines because that was something I could not work out how to do in the current multi.
    i.e. How do I log some of the data I have written to db but only after the transaction but only after it has succeeded.
    I really don’t want to have to manually pull them out of the multi object.

Most Liked

josevalim

josevalim

Creator of Elixir

When with was proposed, one of the possible additions was transactional with, which is pretty much what you proposed. The problem is that with/for are compile time constructs, Multi works at runtime which is quite more flexible.

Regarding the “after” callback, we should add a new operation to the multi that receives the result of the Repo operation {:ok, ...} or {:error, ...} and return either {:ok, map} | {:error, map}. Can you please open up an issue? I would like to hear yours and @michalmuskala’s feedback.

michalmuskala

michalmuskala

I’m thinking about extending multi for some time now, and I have some ideas. I have to say, though, that this is not the area I was exploring.

My main thinking was in expanding the Multi abstraction and building something lower-level that could be later used by ecto to build Ecto.Multi on top. Some of those ideas are explored by my friend @AndrewDryga in his library Sage.

Looking at your example, I’m not sure it’s really different from doing something like (besides some additional verbosity with the explicit rollback and repo):

fun = fn ->
  with {:ok, command} <- Repo.insert(command_changeset),
       {:ok, event1} <- Repo.insert(%{event | command_id: command.id}),
       {:ok, event2} <- Repo.insert(%{event | command_id: command.id}) do
    IO.inspect(event1)
    IO.inspect(event2)
    command.id
  else
    {:error, reason} -> Repo.rollback(reason)
  end
end

case Repo.transaction(fun) do
  {:ok, command_id} ->
    # stuff
  {:error, reason} ->
    # different stuff
end

The problem is of course, that in this situation (and in the proposed syntax) - we don’t really know which operation failed. Sometimes that doesn’t matter, but sometimes it does - the current multi provides the information at the expense of requiring an explicit name for every operation.

Crowdhailer

Crowdhailer

Creator of Raxx

It would be if you tried to use a for for everything but the nice thing about recognising a monad is it tells you all the functions you need. so as long as for was built ontop of map/flat_map and they were exposed then they would handle all your needs. The for syntax being left to only the cases it is suitable.

My suggestion is really should only be syntactic sugar on what exists below. I haven’t discovered all of ecto yet and your example is actually really helpful in telling me how I will probably handle the problem today.

Unreliable because there is always the case where the transaction success and the communication with the external system fails

We have considered this case, and have it covered, we however write commands + events in same transaction because we use command as idempotency indicator for handling retries from the client. But this is definetly a topic for another thread

No. this is not what I was discussing. The DBMonad I am talking about wraps a potential value that is extracted by running against a DB. because the contents of a DB varies the most common case is to have the wrapped value be a result monad but that is not a firm requirement.

I started writing a more detailed explanation but that turned out to just be the Slick documentation. So I am no and have decided to let people refer to them if they want to find out more.

p.s. it took me a good week to get my head around what it was doing
p.p.s That alone might suggest it’s not a good abstraction but I was able to use it without grokin how it was monady

Last Post!

LostKobrakai

LostKobrakai

Ecto.Multi.run does only expect {:ok | :error, any()} to be returned by the passed function, so you could just as well query the db within that function.

Where Next?

Popular in Questions Top

minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New

Other popular topics Top

Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New

We're in Beta

About us Mission Statement