Option type compatible with comprehensions in Elixr?

Background

I am trying to up my Functional Programming (FP) skills and one of the things that newcomers first learn in FP is the Option Type (aka, Maybe Monad).

Option what?

This construct is present in many languages, Haskell has Maybe and Java and Python (yes, Python!) have Optional.

Basically this type models a value that may or may not be there.

05_47

How it all comes down to Elixir

Most FP languages have comprehensions, Scala and Elixir have the for construct while Haskell has its famous do notation.

In Scala and Haskell, these comprehensions work not only with Enumerables (such as Lists) but also with our Option type (which is not an enumerable).

I mention this, because according to my understanding, Elixir’s comprehensions only works on Enumerables. Furthermore, as far as I know, there is not Option type datastructure in Elixir.

What does Elixir have?

Elixir has tagged tuples in the form of {:ok, val} or {:error, reason}. Now while Elixir comprehensions can pattern match with tagged tuples:

iex> values = [good: 1, good: 2, bad: 3, good: 4]
iex> for {:good, n} <- values, do: n * n
[1, 4, 16]

It also ignores values that do not pattern match:

iex> values = [good: 1, good: 2, bad: 3, good: 4]
iex> for {:bananas, n} <- values, do: n * n
[]

However, this does not replicate the behaviour of the Option type correctly. Following is an example in Scala:

  for {
      validName  <- validateName(name)
      validEnd   <- validateEnd(end)
      validStart <- validateStart(start, end)
    } yield Event(validName, validStart, validEnd)

Having in mind this signatures:

def validateName(name: String): Option[String]
def validateEnd(end: Int): Option[Int]
def validateStart(start: Int, end: Int): Option[Int] 

The result of the full comprehension expression, should any function return None , will be None.

With Elixir, the bad result would be ignored and the pipeline would simply continue happily ever after.

Questions

At this point I am thinking that implement this Option type as a structure that implements the Enumerable Protocol (so it can be used in Elixir comprehensions) is something that should be possible.

However, I am not sure I want to go down that route if I can simulate similar behavior using tuples.

So I have the following questions:

  1. Is it possible to simulate the Option type using tagged tuples inside Elixir comprehensions?
  2. Are there any Elixir libraries in the wild that have Monadic types (like the one we saw here) usable within Elixir comprehensions? (I know about witchcraft but they have their own construct for comprehensions, which for the time being, I think is a little overkill. I am interesting in something that works with Elixir’s native comprehension functionality).
1 Like

Hi, I have strong Scala background, but I never missed the for comprehensions or the option type.
Instead I use more pattern matching to solve those problems. So a Scala for comprehension with multiple generators becomes a with statement in my elixir solution space.

4 Likes

The with construct will only execute one line at a time, while the comprehensions will act as nested flatMaps.
While some functionality can be shared between the two (the with construct in elixir is a good way of replacing the Result Monad, for instance) comprehensions are more extensive in that they will apply the results of any given generator to those of the other generators.

For this reason, I am interested in see how far I can push comprehensions and how I can use them.

1 Like

There are a bunch of other versions listed in the readme here.

Personally, I do not miss the Option type from F#, though their async and result comprehensions were great. I hated working with the Option/Result type outside of the result comprehension. Noise and ceremony!

1 Like

I’m personally not a big fan of how monads are always tought boundled with their specific syntax in language XY. Many monads would be considered way less magicy if people actually tought the idea unrelated to syntax, because as you said a option monad in elixir is usually just a {:ok, term} | {:error, term} tuple, but handled with a bunch of explicit case statements. You can see similarities between Functors and Enum.map.

Generally if you want to have for work with “maybe” in elixir I’d just do this:

for result <- list do
  case result do
    {:ok, x} -> handle_x(x)
    err -> err
  end
end
5 Likes

Please do bear in mind that this dicussions’s purpose is not to focus only on the Option type, I am merely using it because I believe it to be the most well known example to people that do FP.

It is fine for people not liking the Option type, what I am really interested in here is in how I can expand this into other (more useful) types and improve my style with the knowledge gained from trying it both ways :smiley:

Unfortunately, this will not work for multiple generators, unless I have a colossal with statement / case expression at the end for all possible results.

I still appreciate the time you took for your proposal, thank you!

I’d be really curious for a usecase for that. While in theory I can see the problem in that in practice I’d be wondering why one got to needing it in the first place.

2 Likes

Instead of trying to force H-M types into tagged tuples (which they fit most of the time, but not always), what about using a type that already plays nice with Enumerable: a single-element list?

  • Some(A) is then represented by [A]
  • None is represented by []
6 Likes

That is exactly the approach I am using now:

defmodule ParsingWithOption do
  alias Event

  @type option(t) :: some(t) | nothing
  @type some(t) :: [t]
  @type nothing :: []

  @spec validate_name(String.t()) :: option(String.t())
  def validate_name(name) do
    if String.length(name) > 0 do
      [name]
    else
      []
    end
  end

  @spec validate_end(integer) :: option(integer())
  def validate_end(the_end) do
    if the_end < 3000 do
      [the_end]
    else
      []
    end
  end

  @spec validate_start(integer(), integer()) :: option(integer())
  def validate_start(start, the_end) do
    if start <= the_end do
      [start]
    else
      []
    end
  end

  @spec parse(String.t(), integer(), integer()) :: option(Event.t())
  def parse(name, a_start, an_end) do
    for valid_name <- validate_name(name),
        valid_end <- validate_end(an_end),
        valid_start <- validate_start(a_start, an_end) do
      %Event{name: valid_name, start: valid_start, end: valid_end}
    end
  end
end

However, I am not really sure I am happy with the result.
Getting None at the end of the comprehension is a hell lot more friendly (for my brain) then getting [] and then making the link in my head that [] == None.

And this is having in mind that I am using polymorphic typing with dialyzer, which is the coolest thing ever since … (insert cool thing here).

My function signatures are also totally sick, too bad dialyzer can’t make sense of them and actually catch false positives… (but that is a story for another post. gradient is actually being incredibly useful here).

Some trivial macros can tidy this up a little, if you’re so inclined:

defmodule Blargh do
  defmacro some(x), do: [x]

  defmacro none(), do: []
end

defmodule BlarghTest do
  import Blargh

  def foo(x) do
    case x do
      some(value) -> some(2*value)
      none() -> none()
    end
  end
end
4 Likes

At risk of going off on a tangent, let me point out that the type system of Erlang (and BEAM) pre-date Scala/F#/Haskell. It was developed in industry, not academia. The creators did not set out to create a functional language, but it happened to meet their requirements best, so that’s what we got.

I’m impressed with the levels of static typing people have been able to achieve, but at a fundamental level there will be mismatches with BEAM-level types because the language designers made different design tradeoffs originally.

This doesn’t help with your specific question(s), but perhaps it will help to moderate your expectations of what is possible/practical and when to expect compromises.

5 Likes

I completely share the sentiment.

In my experience we can still achieve quite a lot with tagged tuples by the way, especially if they are multiple-level e.g. {:payment_result, {:ok, %{amount: "3.45", currency: "EUR"}}} or {:payment_result, {:error, :payment_gateway_timeout}} etc.

One weird thing I noticed all over my Elixir tenure in companies is how teams avoid nested tuples almost religiously and I have no clue why (especially having in mind that some very prominent library modules like Ecto.Multi make heavy use of them!). Nested tuples helped me in personal projects and when helping various acquaintances bootstrap businesses (by writing some code for them). Haven’t seen downsides so far.

1 Like

Can’t talk about Scala or Haskell or OCaml (which I still want to learn!) but e.g. in Rust you have this method:

…that transforms a Some(value) / None variable into a thing you can iterate on (Enumerable in Elixir lingo).

That would also fit the spirit of “be explicit” in the FP world nicely – don’t convert stuff to something they are not usually supposed to be unless you really need it in your case. So you’d end up with:

defmodule Option do
  def to_enumerable({:option, {:some, value}}), do: [value]
  def to_enumerable({:option, :none}), do: [])
end

Which Rust kind of does as well under the hood – you get an iterable collection with one or zero elements.

Of course you can go all in and just implement the Access protocol, too.

But I don’t think you could get away with emulating things a la Scala style in Elixir without some macro magic or some quite heavy libraries like ok or witchcraft. I personally would prefer having explicit wrappers and using them at the right places for code readability points (and my future self hating me less – but I am aware that your interest in this is mostly academical and not industry-oriented so the point of making future maintenance easier likely doesn’t apply to you at all).

1 Like

I am trying to up my Functional Programming (FP) skills and one of the things that newcomers first learn in FP is the Option Type (aka, Maybe Monad).

Depends on where you learn FP. I learned FP through Lisp, Erlang, and Javascript, and didn’t encounter Option/Maybe until 15 years after I was already happily using most of what FP has to offer.

As with any concepts, do not try to force some programming language paradigms into another programming language: they are usually a bad fit. People have been trying to bolt Haskell/Haskell-like on top of Erlang and then Elixir for literally decades. Nothing works, nothing sticks, and for a good reason.

4 Likes

I learned Ocaml, Scala and only missed Haskell.
Concepts like these were always useful to me. I didn’t know about Rust Option type, but truth is I am right now exploring a similar solution.

Of course you can go all in and just implement the Access protocol, too.

I believe you mean the Enumerable protocol, since comprehensions work with Enumerables :smiley:

1 Like

A random thought: the simplest possible option(t) would be t | nil, for all t besides booleans.

map is then spelled &&:

some_optional && do_thing_with_value(some_optional)

orElse/withDefault is spelled ||:

some_optional || "default value"

Alas, do-notation-style chaining doesn’t compile:

# DOES NOT COMPILE - variables are not in scope

(valid_name = validate_name(name))
&& (valid_end = validate_end(an_end))
&& (valid_start = validate_start(a_start, an_end))
&& %Event{name: valid_name, start: valid_start, end: valid_end}
3 Likes

An interesting thought that merits its own discussion in my opinion.
For now however, I just really want to make something that works with Elixir’s native comprehensions, hence, the Option type which can be built using a list of 1 value internally.

(I am still working on it, believe it or not … )

Answer

After searching for all the functional libraries Elixir has on hex, at the time of this writing none matched my main requirement:

  • Being usable with Elixir comprehensions.

Some say Elixir comprehensions are not powerful enough for such cases. This is a falsifiable claim, so I decided to go ahead and try to falsify it.

Say Hi to Option.ex

Yes, the name is not inspiring. Originality has never been my forté.
But what is this?

Simply put, this is an option type for elixir, aka, Option/Maybe monad. Yes, another one.

And just like what most people coming from languages like Scala/Haskell/Python have come to know, it has a couple of subtypes Some and None.

option.ex

defmodule Option do
  @type t(elem) :: __MODULE__.Some.t(elem) | __MODULE__.None.t()

  defmodule Some do
    @type t(elem) :: %__MODULE__{val: elem}

    defstruct [:val]

    defimpl Collectable do
      @impl Collectable
      def into(option), do: {option, fn acc, _command -> {:done, acc} end}
    end

    defimpl Enumerable do
      @impl Enumerable
      def count(_some), do: {:ok, 1}

      @impl Enumerable
      def member?(some, element), do: {:ok, some.val == element}

      @impl Enumerable
      def reduce(some, acc, fun)

      def reduce(_some, {:halt, acc}, _fun), do: {:halted, acc}
      def reduce(some, {:suspend, acc}, fun), do: {:suspended, acc, &reduce(some, &1, fun)}
      def reduce([], {:cont, acc}, _fun), do: {:done, acc}

      def reduce(%Option.Some{} = some, {:cont, acc}, fun),
        do: reduce([], fun.(some.val, acc), fun)

      @impl Enumerable
      def slice(_option), do: {:error, __MODULE__}
    end
  end

  defmodule None do
    @type t :: %__MODULE__{}

    defstruct []

    defimpl Collectable do
      @impl Collectable
      def into(option) do
        {option,
         fn
           _acc, {:cont, val} ->
             %Option.Some{val: val}

           acc, :done ->
             acc

           _acc, :halt ->
             :ok
         end}
      end
    end

    defimpl Enumerable do
      @impl Enumerable
      def count(_none), do: {:error, __MODULE__}

      @impl Enumerable
      def member?(_none, _element), do: {:error, __MODULE__}

      @impl Enumerable
      def reduce(none, acc, fun)

      def reduce(_none, {:cont, acc}, _fun), do: {:done, acc}
      def reduce(_none, {:halt, acc}, _fun), do: {:halted, acc}
      def reduce(none, {:suspend, acc}, fun), do: {:suspended, acc, &reduce(none, &1, fun)}

      @impl Enumerable
      def slice(_option), do: {:error, __MODULE__}
    end
  end

  @spec new(any) :: __MODULE__.Some.t(any)
  def new(val), do: %__MODULE__.Some{val: val}

  @spec new :: __MODULE__.None.t()
  def new, do: %__MODULE__.None{}
end

This works with Elixir comprehensions, and it makes use of the fact that the Optional type is a Functor. This means its main requirement is being able to be mapped over. By converting an abstract container into specific implementation detail (like lists in Elixir) I was able to make it work.

How can I use it?

The main purpose of this was to add an Option type to elixir to use with comprehensions. So a comparison to other languages is useful:

In Scala:

def parseShow(rawShow: String): Option[TvShow] = {
  for {
    name <- extractName(rawShow)
    yearStart <- extractYearStart(rawShow)
    yearEnd <- extractYearEnd(rawShow)
  } yield TvShow(name, yearEnd, yearStart)
}

In Elixir:

  @spec parse_show(String.t()) :: Option.t(TvShow.t())
  def parse_show(raw_show) do
    for name <- extract_name(raw_show),
        year_start <- extract_year_start(raw_show),
        year_end <- extract_year_end(raw_show),
        into: Option.new() do
      %TvShow{name: name, year_end: year_end, year_start: year_start}
    end
  end

You will see, these two pieces of code are basically identical, with the exception of the line into: Option.new(), which is implicit in the Scala example. Elixir requires it to be explicit, which I personally prefer as well.

I could go on with examples from other languages, but they would all read basically the same. This is because comprehensions are basically the same in most FP languages.

But this doesn’t answer the full original post …

What about an Elixir equivalent in tagged tuples?

You can’t use tagged tuples to achieve the same thing using comprehensions. This is impossible.
However, if we discard comprehensions and focus on Elixir’s other constructs, we can come a little bit closer.

Quoting another prominent member of our community, @OvermindDL1 :

Is it possible to simulate the Option type using tagged tuples inside Elixir comprehensions?

Yes, or with with if you want an else, but you’ll want to make the tagged typed be ok: value and error: reason (which is closer to a result type, but it’s a limitation of elixir tuple lists in that they are always tuples). Traditionally {:ok, value} and :error is the “option” type in Elixir, where {:ok, value} and {:error, reason} is the “result” type in Elixir.

So, if you are coming from a different setting, from a functional language into Elixir, this post and my option.ex is most certainly going to help you.

If however, you’d rather stay away from Mathematical Categories and other functional concepts like you want to stay away from the plague, with statements with other elixir’s constructs ought to serve you well enough.

One is not better than the other, they have different costs/benefits. It’s up to you.

4 Likes

Hey @Fl4m3Ph03n1x I am curious about your Option.ex module.
Could you please show us some real-world scenario to better understand the usage? For example, would it be possible for your parse_show function to somehow return the information that year_end is missing in the raw_show or in wrong format?

1 Like