Advice on how to avoid Go-like error handling patterns in Elixir?

Hi all! I’m fairly new to Elixir. I’ve read a book or two, but am just getting into using the language in practice. I recently needed to write a simple function to update a key in a JSON file. However, I found myself running into a lot of repetitive Go-esque error checking. I’m new to functional programming and I was wondering if there is a more idomatic way to handle this type of situation, or just a way to improve it. I’m not a fan of this style as it is because it’s a huge block of ever-indenting lines like old JavaScript. Here is a slimmed down example I came up with:

defmodule Test do
  @doc """
  Inserts the given JSON into the `data` field of `file`.
  """
  def insert(file, data) do
    # Read the contents of the file
    case File.read(file) do
      {:ok, content} -> 

        # Decode the file to JSON
        case Poison.decode(content) do
          {:ok, decoded_content} ->

            # Encode the given data to JSON
            case Poison.encode(data) do
              {:ok, encoded_data} ->

                # Prepare the updated data for insertion
                final_data = Map.update(decoded_content, "data", encoded_data, fn _ -> encoded_data end)

                  # Encode the updated data
                  case Poison.encode(final_data) do
                    {:ok, encoded_final_data} ->

                      # Write the updated data to the file
                      case File.write(file, encoded_final_data) do
                        :ok -> IO.puts "Successfully updated file!"
                        {:error, reason} -> IO.puts "Could not write to file because #{reason}!"
                      end
                    {:error, {:invalid, reason}} -> IO.puts "Could not encode updated JSON because #{reason}!"
                  end

              {:error, {:invalid, reason}} -> IO.puts "Could not encode given JSON because #{reason}!"
            end

          {:error, :invalid} -> IO.puts "Could not parse file to JSON!"
          {:error, :invalid, reason} -> IO.puts "Could not parse file to JSON because #{reason}!"
        end

      {:error, reason} -> IO.puts "Could not read file because #{reason}!"
    end
  end
end

Keep in mind, I just wrote this as a quick example. It’s not my actual code nor is it perfect code. I’m just trying to demonstrate the error handling pattern I’m Talking about. Thanks in advance!

I got this tip courtesy of the Pragmatic Studio Elixir course.

Instead of nesting case statements, use function pattern matching to handle conditional branches.

File.read(file)
|> handle_file

def handle_file({:ok, content}) do
  Poison.decode(content) 
  |> handle_encode
end

def handle_file({:error, reason}) do
  # Something someting
end

And so on.

5 Likes

I’m currently playing with the Exceptional library:

In short, it explains that error tuples are annoying for error handling. They break pipelines, they have variable shapes which makes them hard to pattern-match ({:error, :reason} or sometimes {:error, {:foo, :bar}} or whatever). On the other hand, pattern matching on maps gives you total flexibility in terms of what your error value can contain. So instead of returning error tuples, what if you returned error maps? and Exceptions in elixir are just structs which are maps.

Add a new operator f ~> g which behaves like f |> g except that if the input is an exception then it will bypass g and return the exception. Yes we are not raising exceptions, but returning exceptions like any other values.

Check the post on medium, it’s well explained and that is working well for me now.

2 Likes

You can use the with/else syntax in some cases, pipelining through several functions that match on their input and return it unmodified if it’s an error is also an option.

More generally though, it’s better not to handle an error case explicitly unless it’s expected to happen and you need to send a helpful message back to the caller/user. A supervisor crash log is a lot more useful for debugging than a vague log message like “could not read file” etc. See Erlang and code style: Musings on mostly defensive programming styles.

5 Likes

Using with helps with that kind of logic, basically you describe the happy path and can be as generic or specific as you want when handling errors (s. else example).

5 Likes

All the other advice is good - but at the most basic level keep your functions small, tiny even - e.g.:

defmodule Test do

  defp reportOnFileWrite(:ok),
    do: IO.puts "Successfully updated file!"
  defp reportOnFileWrite({:error, reason}),
    do: IO.puts "Could not write to file because #{reason}!"

  defp writeEncodedFinalData({:ok, encoded_final_data}, file),
    do: reportOnFileWrite (File.write file, encoded_final_data)
  defp writeEncodedFinalData({:error, {:invalid, reason}}, _),
    do: IO.puts "Could not encode updated JSON because #{reason}!"

  defp writeEncodedData({:ok, encoded_data}, file, decoded_content) do
    # Prepare the updated data for insertion
    final_data = Map.update(decoded_content, "data", encoded_data, fn _ -> encoded_data end)
    # Encode the updated data
    writeEncodedFinalData (Poison.encode final_data), file
  end
  defp writeEncodedData({:error, {:invalid, reason}}, _, _) do
    IO.puts "Could not encode given JSON because #{reason}!"
  end

  defp writeDecodedContent({:ok, decoded_content}, file, data),
    do: writeEncodedData (Poison.encode data), file, decoded_content
  defp writeDecodedContent({:error, :invalid}, _, _),
    do: IO.puts "Could not parse file to JSON!"
  defp writeDecodedContent({:error, :invalid, reason}, _, _),
    do: IO.puts "Could not parse file to JSON because #{reason}!"

  defp rewrite({:ok, content}, file, data),
    do: writeDecodedContent (Poison.decode content), file, data
  defp rewrite({:error, reason}, _, _),
    do: IO.puts "Could not read file because #{reason}!"

  @doc """
  Inserts the given JSON into the `data` field of `file`.
  """
  def insert(file, data) do
    # Read the contents of the file
    rewrite (File.read file), file, data
  end
end

This is essentially just a starting point for all the other pieces of advice you are getting. Once things are broken down in this manner it becomes much easier to factor things out.

Never forget that everything is an expression (forget about statements).

6 Likes

I wrote about how i think about this problem a while ago. http://insights.workshop14.io/2015/10/18/handling-errors-in-elixir-no-one-say-monad.html

In summary I think with is useful but I prefer to sacrifice the flexibility of with for something that only handles {:ok, value} and {:error, reason} and built a library just for those cases. https://github.com/crowdhailer/ok

Finally keeping function small is always good ideas. If you have 8 places that can throw an error ( no matter how neat the code) its probably too many

3 Likes

… but they don’t have to. See https://github.com/CrowdHailer/OK for a way this could be done.

edit: aaaand now I see that @Crowdhailer posted that same link, too … if only I’d read the thread all the way. oops!

3 Likes

Thank you all for your informative responses! Because this functionality is only a small section of my app, I don’t want to use any libraries so I’ve taken the advice and refactored my code using functional pattern matching like handle_file_read({:ok, content}), etc. However, I must say the libraries provided in this thread do look enticing and I would certainly consider them if my app had a larger need for such error handling. I’m going to try to use with statements as well to improve my code as I haven’t used those before or in other languages that I’ve written in even. Thanks again everyone!

1 Like

I’d like to respond to @dom’s suggestion separately. I’m very much considering dropping all of this error handling that I’m doing manually since all I am really accomplishing is printing out user-friendly error messages each step of the way so that the user does not have to decode them. Since I’m still raising exceptions and stopping execution each time I get something like an {:error, reason} response, it would make sense for me to take the optimistic route and let the functions naturally raise exceptions themselves if something is wrong. However, the end-user would not easily be able to decode these messages. It seems like a trade-off of code maintainability and conciseness for user-friendliness that I have to weigh.

1 Like

In implementations with imperative languages I often encounter a certain unwillingness to acknowledge the complexity burden that detailed error handling imposes which typically devolves in rampaging Arrow(head)s (especially in JavaScript) and insufficient separation/segregation of “happy path” from “unhappy path” code. “Unhappy path” code deserves the same “management” consideration as the “happy path” code that the domain logic is primarily concerned with.

defmodule Test do

  @doc """
  Inserts the given JSON into the `data` field of `file`.
  """
  def insert(file, data) do
    # Read the contents of the file
    file
    |> File.read()
    |> rewrite(file, data)
    |> to_user_message()
    |> IO.puts
  end

  defp rewrite({:ok, content}, file, data) do
    content
    |> Poison.decode()
    |> write_decoded_content(file, data)
  end
  defp rewrite({:error, reason}, _, _) do
    {:error, {:on_read, reason}}
  end

  defp write_decoded_content({:ok, decoded_content}, file, data) do
    data
    |> Poison.encode()
    |> write_encoded_data(file, decoded_content)
  end
  defp write_decoded_content({:error, :invalid}, _, _) do
    {:error, {:on_decode_invalid, ""}}
  end
  defp write_decoded_content({:error, :invalid, reason}, _, _) do
    {:error, {:on_decode, reason}}
  end

  defp write_encoded_data({:ok, encoded_data}, file, decoded_content) do
    # Prepare the updated data for insertion
    # and encode the updated data
    decoded_content
    |> Map.update("data", encoded_data, fn _ -> encoded_data end)
    |> Poison.encode()
    |> write_encoded_final_data(file)
  end
  defp write_encoded_data({:error, {:invalid, reason}}, _, _) do
    {:error, {:on_encode_data, reason}}
  end

  defp write_encoded_final_data({:ok, encoded_final_data}, file) do
    file
    |> File.write(encoded_final_data)
    |> report_on_file_write()
  end
  defp write_encoded_final_data({:error, {:invalid, reason}}, _) do
    {:error, {:on_encode_final_data, reason}}
  end

  defp report_on_file_write(:ok),
    do: :ok
  defp report_on_file_write({:error, reason}),
    do: {:error, {:on_file_write, reason}}

  defp to_user_message(:ok),
    do: "Successfully updated file!"
  defp to_user_message({:error, {source, reason}}),
    do: to_error_msg source, reason

  defp to_error_message(source, reason) do
    case source do
      :on_read ->
        "Could not read file because #{reason}!"
      :on_decode_invalid ->
        "Could not parse file to JSON!"
      :on_decode ->
        "Could not parse file to JSON because #{reason}!"
      :on_encode_data ->
        "Could not encode given JSON because #{reason}!"
      :on_encode_final_data ->
        "Could not encode updated JSON because #{reason}!"
      :on_file_write ->
        "Could not write to file because #{reason}!"
    end
  end
end

And ultimately “error management” ignores the fact that not all kinds of errors can be predicted.

3 Likes

I totally agree with you. The code I ended up with after some refactoring looks similar to yours, although I handle all of the error messages in the handle_ functions instead of defining them all in one place. I’m not sure if I’m entirely content with my code so far but the suggestions in this thread have helped me simplify and refactor a lot and I’m certainly happier with my code than I was before. Here it is for reference:

def insert(database, table, row) do
  get_path(database, table)
  |> File.read()
  |> handle_file_read(database, table, row)
end

defp get_path(database, table) do
  Path.join([Application.get_env(:simplexdb, :root_dir), database, table <> ".json"])
end

defp handle_file_read({:ok, content}, database, table, row) do
  Logger.info "Successfully opened file for table #{table} in database #{database}!"
  handle_decode(Poison.decode(content), database, table, row)
end
defp handle_file_read({:error, reason}, database, table, _row) do
  Logger.error "Could not open file for table #{table} in database #{database} because of #{reason}!"
  raise "File read error!"
end

defp handle_decode({:ok, content}, database, table, row) do
  Logger.info "Successfully decoded JSON file for table #{table} in database #{database}!"
  verify_key(content, database, table, row)
end
defp handle_decode({:error, {:invalid, reason}}, database, table, _row) do
  Logger.error "Could not decode JSON in file for table #{table} in database #{database} because #{reason}!"
  raise "Invalid JSON error!"
end
defp handle_decode({:error, :invalid}, database, table, row), do: handle_decode({:error, "it was invalid"}, database, table, row)

defp verify_key(content, database, table, row) do
  if Map.has_key?(content, "rows") && is_list(content["rows"]) do
    handle_encode(content, database, table, Poison.encode(row))
  else
    Logger.error "Invalid data key in JSON for #{table} in database #{database}!"
    raise "Invalid JSON error!"
  end
end

defp handle_encode(content, database, table, {:ok, row_content}) do
  Logger.info "Successfully encoded row table #{table} in database #{database}!"
  insert_data(content, database, table, row_content)
end
defp handle_encode(_content, database, table, {:error, reason}) do
  Logger.error "Could not encode JSON in file for table #{table} in database #{database} because of #{reason}!"
  raise "Invalid JSON error!"
end

defp insert_data(content, database, table, row_content) do
  final_content = content
    |> Map.update!("rows", fn rows -> rows ++ [row_content] end)
    |> Poison.encode!()

  get_path(database, table)
  |> File.write(final_content)
  |> handle_file_write(database, table)
end

defp handle_file_write(:ok, database, table) do
  Logger.info "Successfully inserted new row in table #{table} in database #{database}!"
  table
end
defp handle_file_write({:error, reason}, database, table) do
  Logger.error "Could not write to file for table #{table} in database #{database} because of #{reason}!"
  raise "File write error!"
end

The idea here was to get relatively granular error messages for the end user which would allow users to easily decipher what is wrong and fix it themselves.

I don’t understand why “with” was not given as the standard, end all, be all answer. It’s really good, and not many languages have a such a convenience. You are hurting yourself by not using it all the time!

Rust is about the only language I know of with such a convenience. Haskell has something similar, but it is not half as user friendly as it is in elixir. Untested code:

defmodule Foo do
  def insert(file, data) do

    with {:ok, content} <- File.read(file) |> (fn {:ok, res} -> {:ok, res}; e -> {:file_read_error, e} end).(),
         {:ok, decoded_content} <- Poison.decode(content) |> (fn {:ok, res} -> {:ok, res}; e -> {:decode_error, e} end).(),
         {:ok, encoded_data} <- Poison.encode(data) |> (fn {:ok, res} -> {:ok, res}; e -> {:encode_error, e} end).(),
         final_data <- Map.update(decoded_content, "data", encoded_data),
         {:ok, decoded_final_data} <- Poison.encode(final_data) |> (fn {:ok, res} -> {:ok, res}; e -> {:final_encode_error, e} end).(),
         :ok <- File.write(file, decoded_final_data) |> (fn :ok -> :ok; e -> {:file_write_error, e} end).()

    do :ok
    else
      {:file_read_error, _e} -> IO.puts("Failed to read file"); :fail
      {:decode_error, _e} -> IO.puts("Failed to decode data"); :fail
      # ... etc. ...
      e -> IO.puts("Unknown error"); IO.inspect(e); :fail
    end
  end
end

And if you define a to_error function this becomes very, very concise:

defmodule Foo do

  def to_error({:ok, res}, _err) do {:ok, res} end
  def to_error(:ok, _err) do :ok end
  def to_error(res, err) do {err, res} end

  def insert(file, data) do

    with {:ok, content} <- File.read(file) |> to_error(:file_read_error),
         {:ok, decoded_content} <- Poison.decode(content) |> to_error(:decode_error),
         {:ok, encoded_data} <- Poison.encode(data) |> to_error(:encode_error),
         final_data <- Map.update(decoded_content, "data", encoded_data),
         {:ok, decoded_final_data} <- Poison.encode(final_data) |> to_error(:final_encode_error),
         :ok <- File.write(file, decoded_final_data) |> to_error(:file_write_error)

    do :ok
    else
      {:file_read_error, _e} -> IO.puts("Failed to read file"); :fail
      {:decode_error, _e} -> IO.puts("Failed to decode data"); :fail
      # ... etc. ...
      e -> IO.puts("Unknown error"); IO.inspect(e); :fail
    end
  end
end

If there is a good argument against “with”, I’d love to hear it.

2 Likes

What I hate are the commas, I’d prefer this syntax for your last example:

with do
  {:ok, content} <- File.read(file) |> to_error(:file_read_error)
  {:ok, decoded_content} <- Poison.decode(content) |> to_error(:decode_error)
  {:ok, encoded_data} <- Poison.encode(data) |> to_error(:encode_error)
  final_data = Map.update(decoded_content, "data", encoded_data)
  {:ok, decoded_final_data} <- Poison.encode(final_data) |> to_error(:final_encode_error)
  :ok <- File.write(file, decoded_final_data) |> to_error(:file_write_error)
  :ok
else
  {:file_read_error, _e} -> IO.puts("Failed to read file"); :fail
  {:decode_error, _e} -> IO.puts("Failed to decode data"); :fail
  # ... etc. ...
  e -> IO.puts("Unknown error"); IO.inspect(e); :fail
end

And it is entirely doable with a macro as-it-is-right-now, which also confuses me why with is a special form at all. There are libraries that do this style for note.

3 Likes

I am using Railway Programming pattern to better handle error handling. This is a great source that explains the concept: http://www.zohaib.me/railway-programming-pattern-in-elixir/

To implement it, I forked the OK library (https://github.com/CrowdHailer/OK) to include the macros described on the article (map, tee, try_catch) and some others for added quality of life: found to handle functions that can return a resource or nil (I convert those to {:ok, resource} or {:error, :not_found} and tag_error to convert errors in the format {:error, reason} to {:error, {:tag1, :tag2, reason}} (:tag1 and :tag2 are arguments for the tag_error macro).

Using this approach, my code looks like this:

def update_job_offer(%JobOffer{} = job_offer, %{} = job_offer_params) do
job_offer
|> (try_catch JobOffer.changeset(job_offer_params, :update))
~>> (tag_error Repo.update, :changeset, :job_offer)
~>> Search.execute
~>> update_job_offer_candidates_search_list()
~>> JobOffer.add_filtered_candidates
~>> JobOffer.add_aggregates_count
end

and after the last call, you can use a case do to handle all possible outcomes of each function call.

By using this approach, the functions don’t need to handle failures from previous calls, so you just handle the happy path (for input arguments).

2 Likes

You make a great case. To be honest, I haven’t read much about the with statement throughout my journey learning Elixir nor have I seen it used very frequently. For this reason, I don’t believe I was as aware of it as I should have been. However, it seems like the exact construct I need. In fact, I found this link in the getting started guide about using with to refactor exactly this kind of scenario. Cheers for the advice!

This looks way more clean than the regular with construct. I am relatively new to the language so I don’t know too much about the syntax but I can understand why the normal version is there because it is consistent with if and case and things like that. However, using a huge block of code like this in the same way that you use a small expression in an if statement is just not very nice. Thanks for you input!

Just checking, but your example seams to be full of commas. I think they should be removed?

Indeed, fixed! ^.^