How to handle error handling boilerplate

verkhovin · July 7, 2021, 4:37pm

Hi guys!
Want to open here another discussion about error handling concepts in Elixir because it looks like I really stuck here by myself.

My question is about errors that are related to users interactions, where you can’t just “Fail fast and forget / Let it crash” but need to respond with some appropriate error message.
Elixir community’s standard is to return tuples like {:error, reason} when something goes wrong during function execution. Overall, it looks good at the first sight.
But at some point, I noticed that to handle this approach I have to write tons of case or with really often just to check that underlying call didn’t return this {:error, reason} tuple.

Consider an example of a multilayered application (like Controller → Business Logic → database CRUDs). If there is some expected error that occurs on the layer of CRUDs, I will need to have a case to catch this error on a Business Logic layer and pass it to Controller. After that, I need to have another case on the Controller’s layer. And it becomes even worse if you have more than 3 layers there, or your business logic includes something more than just a call to CRUD layer.
Maybe I really missed something but it seems to me that this approach makes you write a lot of boilerplate.

I’m actually from Java world and it would be quite natural for me to resolve this issue by throwing (raising) an exception from the CRUDs level and catch in on the Controller level. But if I understand correctly, exceptions in Elixir philosophy are more about unexpected errors that should be handled by this “Let it crash” way.

My question still is: is it normal and common way to write elixir code having this error checks (case/with) on every layer of the application?

I can provide some naive code example if my thoughts are not clear.

hauleth · July 7, 2021, 7:35pm

I would say that it depends on the requirements and what kind of error is that:

if there is reasonable way to do fallback, for example fetching data failed, but we can return cached result or something like that, then using ok/error tuple is ok
if there is no reasonable way to fallback, for example there is no user in the DB for given ID, then I would throw and let Plug.Exception do it’s thing

kokolegorille · July 7, 2021, 7:53pm

What You don’t have in Java is pattern matching and functions with multiple heads.

The first condition can often be solved with a function pattern matching some kind of result. ok, error is quite common in FP.

But if You are sure the operation cannot fail, You can use the ! operator.

It’s a good practice to push DB related to the outer boundaries. It’s also possible to use onion architecture. Where the central part is purely functional, and does not care about details of the DB layer.

The let it crash means something different. In case there is a problem You cannot predict, You are covered by the supervisors.

joddm · July 7, 2021, 8:04pm

I think business logic and CRUD operations are usually handled by Ecto Changesets or multiple function heads which give an error tuple when applicable. Then you can handle any errors in the controller with a case statement as you’ve said.

verkhovin · July 7, 2021, 8:35pm

Yes, I kind of feel this the same way.
I saw opinions here on the forum that throwing exceptions is some kind of “dirtier” solution than returning ok/error tuples. Maybe I misunderstood it somehow

Thank you for the answer!

verkhovin · July 7, 2021, 8:42pm

Thank you for your answer!
If I understand you correctly, your point is that I can replace case/with with functions with multiple heads.
I think it is still the same boilerplate but in another form. Maybe I am too dramatic about this boilerplate issue by the way I was just wondering if it is ok in FP world to pattern match almost every function call result to determine erroneous responses and handle them. As you said, it is quite common approach, right?

verkhovin · July 7, 2021, 8:53pm

Thank you for your answer!

Sometimes it is not the case just to map response from Ecto directly up to the controller.

For example, you can have a chain of calls of functions from different modules. And each of these calls returns this ok/error tuple. You can’t just pass them all to the controller and the only option here is to pattern match each of these responses with with expression. So you handle it on this level in this way and then return ok/error tuple to the controller. And in turn, in the controller, you need to do the similar pattern matching again.
Hope it is clear.

sezaru · July 7, 2021, 10:27pm

I’m not sure if I follow you when you say you cannot redirect them all to the controller.

Let’s say you have a function CRUD.get that returns {:ok, value} or :error, reason} and that reason can be multiple types of errors.

Now let’s say you want to redirect the errors to the controller from your business logic and handle the :ok case, if you redirect something to the controller via this function return, you can simply do something like:

with {:ok, value} <- CRUD.get() do
  # Do something with value
end

This will handle the :ok case and redirect all the other ones.

If you need to call a function to redirect it to the controller, you can do something like:

with {:ok, value} <- CRUD.get() do
  # Do something with value
else
  {:error, reason} -> Controller.redirect({:error, reason})
end

or

case  CRUD.get() do
 {:ok, value} ->
    # Do something with value

 {:error, reason} -> 
    Controller.redirect({:error, reason})
end

In all these cases you don’t need to have a case for each error tuple, you can simply pattern match all of them.

Now at the Controller, if you want to handle each one of the errors, then you will need to use case or function pattern match to handle it, but that would be the same with java handling the catch cases.

Doesn’t that solves the issue or am I missed the point entirely of what the real problem is in your case?

verkhovin · July 8, 2021, 6:43am

Thank you for you detailed answer!
Yes, this is almost what I was talking about. I’ll provide an example to show what I meant.
Let’s say that CRUD operation is not the only thing I need to do during the request handling. And let’s say there are more than 2 layers in the application. So Controller calls a function from SurfaceLogic module. And this function calls another one from DeepLogic module. DeepLogic module works with some CRUDs and external APIs through clients.

#DeepLogic module
def do_deep_logic() do
  with {:ok, response} <- ExternalApiClient.get_info(), # can return  {:error, :api_500_error}
         {:ok, _} <- CRUD.insert() # can return {:error, changeset}
  do
      {:ok, do_some_logic(response)} 
  else
     {:error, %Changeset{ ... } = changest} -> {:error, :already_exists}
     {:error, reason} -> {:error, reason}
  end
end

# SurfaceLogic module
def do_surface_logic() do
  #pattern match on {:ok} and just pass all the errors to the upper level:
  with {:ok, value} <- DeepLogic.do_deep_logic() do
     {:ok, do_some_response_preparations(value)}
  end
end

#Controller
def handle_post_request(conn, _) do
  case SurfaceLogic.do_logic() do
    {:ok, result} -> render( ... ) # render success
    {:error, :already_exists} -> render (...) # render already exists
    {:error, :api_500_error} -> render ( ... ) # something wrong with external data provider
  end
end

So now I need to have case/with on every single layer here. (Yes, I still have action_fallback on controller level, but it will just move matching from this controller to Fallback Controller, so I’ll leave it here now to make it easier to understand).

My point here is that all these case/with statements look like boilerplate that you need to have everywhere (on every single function call, if this function returns :ok/:error). And I just try to understand if it is common practice (considered as ok) or not.

A more natural way to handle this issue for me would raise exceptions on the level of CRUD and ExternalApiClient and catch them on the controller level. In this case there wouldn’t be a need in case/with on the level of DeepLogic and SurfaceLogic. But it seems like this way is more uncommon in Elixir/FP community.

soup · July 8, 2021, 11:25am

How you’ve written it is how I would have probably done so also, but I can’t speak as a community luminary.

Related: Good and Bad Elixir which may give you a bit of peripheral insight at least.

Specifically “Don’t pipe results into the following function”, which you maybe could have used in surface logic, i.e:

def do_surface_logic() do
  DeepLogic.do_deep_logic()
  |> do_some_response_preparations()
end

def do_some_response_preparations({:ok, val}) ...
def do_some_response_preparations({:error, e}) ...

But as you say, it’s only really shuffling stuff around.

See also “Raise exceptions if you receive invalid data.” in the same article.

By it’s nature, the article has to be opinionated but it’s definitely worth reading once or twice, “know when to break the rules”, etc blah blah blah.

I think the existence of {:ok | :error} and the line “In practice, Elixir developers rarely use the try/rescue construct” in the getting started guide gives the impression that exceptions should be avoided in Elixir, but they exist to be used and is probably a few good blog posts waiting to be written re: navigating between them and tuples.

verkhovin · July 8, 2021, 11:34am

Thank you for the link. The article seems really useful! Going to read through it a couple of times.

existence of {:ok | :error} and the line “In practice, Elixir developers rarely use the try/rescue construct” in the getting started guide gives the impression that exceptions should be avoided in Elixir

Yes, I think this is exactly what makes me so confused about it

joaquinalcerro · July 8, 2021, 12:02pm

You can also check this interesting post from @michalmuskala - Error Handling in Elixir Libraries | Michał Muskała

verkhovin · July 8, 2021, 12:40pm

Nice post too, thank you!

verkhovin · July 8, 2021, 12:53pm

So I’ve read through the articles @soup and @joaquinalcerro referred.

The main idea I took from these articles that in general raising an exception is not a bad practice. Moreover, you should raise an exception if the error breaks your application invariants so you don’t have to bother with unnecessary error handling on each application level (see the example I provided here: How to handle error handling boilerplate - #9 by verkhovin).

{:ok, result}/ {:error / reason} way is still very useful when errors are part of your business logic and can somehow determine the application’s execution flow.
They are also useful if you do not know exactly how your module will be used by a callers side. i.e. It is the case when you develop some API for a library.

Basically, it is what @hauleth said in the first reply to this topic.

Thank you all for your help! The community seems really friendly, it was a real pleasure to read all your answers!

karlosmid · July 8, 2021, 2:05pm

Hi! Here is how I handle errors in the MVC application.

You first get Ecto errors in Business Layer. Do not just case error in order to pass it to the upper layer. If in a specific layer you can not do anything about this error, do not pattern match on it. It will be propagated to the upper layer.
Do pattern match on errors in the controller layer. Handle it by passing some meaningful error to the user of the controller.

amnu3387 · July 9, 2021, 7:55am

I personally am very inclined to force explicitness and use more verbosity to describe flows. When using with I normally resort to tagging the branches, but sometimes you do need to pass through the underlying returns from the failing branch. When writing API endpoints I’ve settled in just having a %Response{}/%Output{} struct with :errors, :notices, :payload fields that the frontend knows how to interpret - this approach requires you to do it almost from the beginning and be in control also of how the front-end interacts. It’s also not completely feasible for html pipelines, as you can have redirects and so on, but could probably be made so.

Controllers/channels call only into context functions, and these entry points in the contexts either return those payload structs, or they return normal results and I have a translation module/functions for the returns. Again, it’s more boilerplaty.

But the issue that I think deserves a bit more thought is not that raising exceptions in themselves is bad (outside of things that really require raising, like a db disconnect, or inaccessibility of a resource, or inside a loop/recursion that no longer makes sense, but usually there you use throw), when you look at the flow of a single entry point, it can usually look clearer and in a way less verbose. The problem comes when all your code base applies this style. Then it basically becomes a mine field because everything can raise somewhere and you’re left having to keep in mind every detail of all functions you’re calling and what those themselves are calling.

When I have too much time to think about this nonsense I imagine I would like to see a variation of with (a special form named weave or steps) that would look like this:

weave # notice the new line 
   [not_found: %User{} = user] <- Db.Repo.get(User, user_id),
   [not_authorized: true] <- Contexts.Authorization.is_authorized?(user, some_action),
   [changeset: {:ok, %User{} = new_user}] <- Contexts.Users.update(user, some_action)
do
   {:ok, user}
else
  [changeset: changeset] -> {:error, changeset}
  [{error, _}] -> {:error, error}
end

Perhaps it could even be that when the do block was ommitted it would pass the last value, without the keyword.

weave 
   [not_found: %User{} = user] <- Db.Repo.get(User, user_id),
   [not_authorized: true] <- Contexts.Authorization.is_authorized?(user, some_action),
   [changeset: {:ok, %User{} = new_user}] <- Contexts.Users.update(user, some_action)
else
  [changeset: changeset] -> {:error, changeset}
  [{error, _}] -> {:error, error}
end

This can be confusing the first time you encounter it (not_authorized: true) but afterwards it’s simple to understand.

Other option would be:

weave
  %User{} = user <- Db.Repo.get(User, user_id) -> :not_found,
  true <- Contexts.Authorization.is_authorized?(user, some_action) -> :not_authorized,
  {:ok, %User{} = new_user} <- Contexts.Users.update(user, some_action) -> :changeset
else
  [changeset: changeset] -> {:error, changeset}
  [{error, _}] -> {:error, error}
end