Maybe: nil protection for nested structs

I have a problem. It’s very common to have nested structs when you deal with Ecto and associations. For example, a Person can have a City, which has a name.

The problem is, city can sometimes be nil. In that case, this code will raise an error. (“nil doesn’t respond to :name”) I want something like Ruby try here.

person.try(:city).try(:name)

Elixir has a built-in thing like this, called get_in:

get_in map, [:nonexistent, :keys]
# => nil

However, it only works with maps, not structs. The documentation says it’s specifically ​_not_​ supposed to work with structs.

So, I have a proposal. We make a module with a functional API like this:

maybe(person, [:city, :name])

And a macro to pretty things up a bit:

maybe(person.city.name)

If any element in the chain returns nil, the expression returns nil.

Here’s my sample implementation:

defmodule Maybe do
  defmacro maybe(ast) do
    [variable|keys] = extract_keys(ast)
​
    quote do
      maybe(var!(unquote(variable)), unquote(keys))
    end
  end
​
  def maybe(nil, _keys), do: nil
  def maybe(val, []), do: val
  def maybe(map, [h|t]) do
    maybe(Map.get(map, h), t)
  end
​
  defp extract_keys(ast, keys \\ [])
  defp extract_keys([], keys), do: keys
  defp extract_keys({{:., _, args}, _, _}, keys) do
    extract_keys(args, keys)
  end
  defp extract_keys([{{:., _, args}, _, _}|t], keys) do
    keys = keys ++ extract_keys(args)
    extract_keys(t, keys)
  end
  defp extract_keys([{:., _, args}|t], keys) do
    keys = keys ++ extract_keys(args)
    extract_keys(t, keys)
  end
  defp extract_keys([key|t], keys) do
    keys = keys ++ [key]
    extract_keys(t, keys)
  end
end

And example usage:

import Maybe

defmodule Person do
  defstruct city: nil
end
	
defmodule City do
  defstruct name: nil
end
	
person = %Person{city: %City{name: "Portland"}}
maybe(person.city.name) # => "Portland"
maybe(person.nonexistent.name) # => nil

Is this a good idea? Should it be named something else? Is there something in the standard library that I missed?

7 Likes

Remember you can use pattern matching:

case person do
  %{city: %{name: name}} -> name
  %{} -> default
end
4 Likes

I’m always confused when matching on %{} (or #{} in erlang)… My intuition does tell me we are matching on an empty map (as we were when matching on []), but in reality it is like anything when is_map(anything).

Is there a certain reason why pattern matching on %{} and [] is treatened differently?

3 Likes

Is there a certain reason why pattern matching on %{} and is treatened differently?

They are different structures. Maps were designed to always map on a subset, therefore the empty map ends up matching all maps. Otherwise, imagine how useless pattern matching on maps would be if every time I wanted to take a key from the map, I had to match on all keys. The cases where I would want to do a full matching are rather rare (and can always be achieved with map_size).

12 Likes

This makes absolutely sense, thank you for your answer.

4 Likes

Pattern matching does work, but is quite verbose compared to what @danielberkompas is suggesting.

1 Like

I agree with @jamonholmgren. @danielberkompas 's solution seems like a good way to concisely read data.

Of course, the ruby try() construct can sometimes be considered a code smell, as we’re actually breaking the Law of Demeter here, and a better solution (But often not worth the hassle/extra abstraction) would be to create a proper null-object.
But… I’m getting sidetracked.

I think that Maybe.maybe(some.field.lookup) is a very reasonable way to solve this problem. When using it, a developer explicitly acknowledges that the answer might be nil at any depth of the field lookup.

2 Likes

I see applications for this, definitely. Especially if not only considering structs.

But… doesn’t this actually do what you want, @danielberkompas ?

person |> Map.get(:city, %{}) |> Map.get(:name)

Note the use of the empty map as default. The Map.get/3 works on any struct or map as far as I can tell and returns nil or default if the value is not present. Returning by default the empty map makes it save to string these together, and the last one can simply return nil (the default anyway).

2 Likes

@Oliver, yes, Map.get does do what I want. I actually use it here to provide the functional API for Maybe:

def maybe(nil, _keys), do: nil
def maybe(val, []), do: val
def maybe(map, [h|t]) do
  maybe(Map.get(map, h), t)
end

You can call it like so: maybe(map, [:first, :second, :third]). I think it’s a little more convenient than doing a pipeline.

The maybe macro just converts calls like this: maybe(map.first.second.third) into functional calls for maybe(may, [:first, :second, :third]).

Edit: Since the Maybe module is so small, I decided not to release a hex package for it. If you’re interested, you can find it here: https://github.com/infinitered/phoenix_base/pull/14

5 Likes

Is this still true as of today?

If a structs module implements the Access behaviour, then you can use it with the lense like accessors.

Though a 1:1 mapping to struct keys is often not really what you want for structs. Eg. for a set you might have a couple of struct keys required for the implementation, but each on its own doesn’t have any value. Instead you use Access behaviour to make a lookup in the set and either return nil for inexisting items or the item itself if it is in the set. (This is just an example)

1 Like

What I actually need is a “safe navigation” through Phoenix “models”, exactly the case brought on the very beginning of this thread. A quick test shows the get_in to work so it looks to me that either things changed or there are some gotchas I am not aware of

Hmmm…

For me I can not use get_in on an arbitrary struct…

iex(1)> defmodule S do
...(1)>   defstruct [:a, :b]
...(1)> end
iex(2)> get_in(%S{}, [:a])
** (UndefinedFunctionError) function S.fetch/2 is undefined (S does not implement the Access behaviour)
    S.fetch(%S{a: nil, b: nil}, :a)
    (elixir 1.11.4) lib/access.ex:286: Access.get/3

What have you done?

1 Like

To be frank I don’t know. Now it doesn’t work. So I must have done something differently and can no longer reproduce it :frowning:

UPDATE:
The thing that works for my case is something like:

row = MyApplication.MyContext.MyModel |> MyApplication.Repo.one()
column_value = if is_nil(row), do: nil, else: row.column

Yet I am surely not happy with how it looks. OTOH get_in(row, [:column]) works nicely when row is nil but fails when it is not. get_in(row, [Access.key(:column)]) does the opposite. It works when row is not nil, but fails when it is. And I didn’t come up with a more elegant one-liner to cover both cases. And this doesn’t even touch nested structures/associations.

This makes me wonder… is there really no need for something like an equivalent of Rails’ try in Elixir/Phoenix? Sure I could probably implement something like that somewhere in every project, but that kind of “safe traversal” does look like a very common case to me. At least coming from the Rails (and a few other languages/frameworks) world.

1 Like

I ran into a similar issue where I had lots of structs that were basically wrappers for maps, where I just wanted the compile time guarantee of the structs. I wrote a small using macro that I can add to the top of a struct and then use it with the Access module

defmacro __using__(_) do
    quote do
      @behaviour Access

      defdelegate fetch(term, key), to: Map
      defdelegate get(term, key, default), to: Map
      defdelegate get_and_update(term, key, fun), to: Map
      defdelegate pop(date, key), to: Map
    end
  end

Well, I for one just came into need of this today. Given how clever a language Elixir is, I’m actually surprised there isn’t a feature like this already

Syntax wise I wonder if ? (in place of .) could work.

person?city?name

That reads pretty well to me.

[Update]

I ended doing this, e.g. (person.city || %{name: ""}).name. Which is okay for one level, but still rather ugly for more, e.g. ((person || %{city: nil}).city || %{name: ""}).name.

2 Likes

I really wish this were the case. Kotlin provides this out of the box, and it’s super convenient.

person?.city?.name

Kotlin uses this ? syntax for a lot of other things concerning nulls, which don’t necessarily apply to Elixir, but I think if it worked for structs, it would be a really nice convenience.

I am not sure ? operator exactly fits the current feature set.
There is multiple semantics at play here:

For maps when the key is missing we get a runtime error KeyError.

For structs when the key is missing we get a compile-time error (great for preventing typos)

Elixir can track even nested structs but we need the information about them in the source code. That means it is impossible to create a macro that allows accessing nested struct and preventing typos at the same time. E.g. nothing stops you from populating person.city with a string instead of city struct.

We could resign from “typo safety” and do the checks at runtime but then we have next choice:

  • do we throw KeyError when key is missing?
  • or do we return nil?

I’d say KeyError is better because we are talking about structs and they populate all keys by default with nils.

Accessing nested values is very much Maybe monad. So if we want to have an infix operator for nested access I’d use >>> that stands for bind in https://github.com/witchcrafters/witchcraft

  def nil >>> _, do: nil
  def struct >>> atom when is_struct(struct) and is_atom(atom), do: Map.fetch!(struct, atom)

and a full example of all the things I talked about:

defmodule Outer do
  defstruct [:inner]
end

defmodule Inner do
  defstruct [:a]
end

defmodule NestedAccess do
  def nil >>> _, do: nil
  def struct >>> atom when is_struct(struct) and is_atom(atom), do: Map.fetch!(struct, atom)
end

defmodule SafeStructs do
  def test do
    empty = %Outer{
      inner: nil
    }

    full = %Outer{
      inner: %Inner{
        a: "a"
      }
    }

    # dot nation protects from typos
    IO.inspect empty.inner
    # nil

    #IO.inspect empty.lol
    # does not compile

    IO.inspect full.inner.a
    # "a"

    # pattern matching without structs does not protect from typos
    case empty do
      %{inner: %{a: a}} -> a
      %{} -> nil
    end
    |> IO.inspect
    # nil

    case empty do
      # note the non existent foo key
      %{foo: %{a: a}} -> a
      %{} -> nil
    end
    |> IO.inspect
    # nil

    # pattern matching with structs protects from typos
    # because it gives compiler enough information
    case empty do
      %Outer{inner: %Inner{a: a}} -> a
      %Outer{} -> nil
    end
    |> IO.inspect
    # nil

    #case empty do
    #  %Outer{foo: %{a: a}} -> a
    #  %Outer{} -> nil
    #end
    #|> IO.inspect
    # does not compile

    import NestedAccess
    empty >>> :inner
    full >>> :inner >>> :a
  end
end

To sum up: you can solve the problem with a two-line function defining an infix operator. Just be careful to think about the semantics of missing keys. For structs missing a key is definitely a typo. For maps, you could add a case like:

def map >>> atom when is_map(map) and is_atom(atom), do: Map.get(map, atom)

because missing keys might be a legitimate thing.

And yet another question is: what if the city is suddenly a string instead of a struct? Do you return that string or throw an error? I think it is best to leave that decision to people implementing actual business logic and leave it out of the language :slight_smile:

2 Likes