How do you deal with partial failures

Learning Questions & Discussions What/How...
1

I was thinking on how to deal with partial failure. usually, the ok/error tuples works fine when the side effects of a function can be conveyed in some sort of transactional behavior. But when you need to deal with something can’t be dealt with in that way, how do you prefer to communicate this to the caller?

2 Likes
2

Do you mean there are N things you are doing and some of them may fail while some pass? There are a few things I can think of:

  1. Return a 2-tuple where the first entry is the list of passes and the second is the list of errors
  2. Return {:ok, list_of_results_good_and_bad} if you were able to successfully execute what you wanted to do and some passed/some failed. Return {:error, error} for catastrophic failure i.e. you couldn’t reach the database because it’s down.
1 Like
3

i’m not a good fan of nesting tuples, because of that i’ve thinking to use something like:

{:ok, data} for full success
{:partial, data, error} for when an error happens in middle of the process.
{:error, error} for when it’s not possible to start working with provided parameters.

I’m not sure if this is a good idea, but thinking of an external library or something else that is an opaque box from the perspective of the caller.

edit: that’s why i’m interested in what people already used to deal with that kind of situation.

4

IMO it’s more important to think of this from the other side: what does the caller expect when a partial failure occurs? Do they care about the specifics of what failed? Can they do anything in response to that information?

There’s also an additional option, for situations where the provided input is completely wrong (missing important keys, wrong types, etc) - raise ArgumentError etc and crash the caller.

6 Likes
5

So, hence my question, how people model their contracts to allow the user to choose what they care about.
ok/error is already an idiomatic thing, but doesn’t allow the user to decide if they want to deal with the partial result.

I don’t like raising ArgumentError unless it’s a contract break, for example, I define that the function expects an integer and the caller passes a float or a string.
Other than that, I prefer to return an error tuple and let the caller decide if they want to raise or not. if their choice is to raise no matter what, they always can just pattern match over the ok tuple, like {:ok, result} = SomeMod.some_fun(params).

My question lies on how people deal with providing flexibility to the caller at the same time that provides enough information to decide if they want to handle it as a failure or as a “good enough” result.

6

I think {:error, error} is the correct thing. What you put as the second element is up to you — you can put some composite value that represents what succeeded and what failed. In either case, there’s some error that the caller can choose to handle or not.

Consider how Ecto changesets work with repos. If you do a Repo.update, you either get {:ok, resource} or {:error, %Ecto.Changeset{}}. If you care, you can dig into that changeset’s errors field, or you can just ignore it and carry on in a generic way.

1 Like
7

:thinking: I could embed the {:partial, data, error} in the error tuple. This would keep consistency with the already established idiom of ok/error tuples. I was thinking to always do {:error, {data, error}} too, this way if it’s partial, data will have whatever is needed there and error states what happened. if it’s a full error data can be nil.

8

I’ve used {:ok, success_results, error_results} in the past for multiple homogeneous operations.
The reasoning is that the operation succeeded(hence the :ok), but some of the processed values may have falied.

For a single operation that might fail midway, I like to model it after Ecto.Multi: {:ok, result} | {:error, failed_operation, failed_value, changes_so_far} Ecto.Multi — Ecto v3.9.2

6 Likes
9

I guess the Ecto.Multi pattern is a good way to handle it. it provides visibility on what failed, why failed and gives back what worked so far. :thinking:

4 Likes
10

Yep, I usually imitate the way that Ecto.Multi returns results in my code. It’s a sensible way of doing things.

3 Likes