What's the most idiomatic way to require a key in a keyword list or make it optional

[Secretmapper]

Hello! Elixir’s syntax is pretty and expressive, but at the same time this comes on the cost of knowing just what the best way to write something is.

I have a function:

 def my_fun(options, sup_opts)

This is what I want to achieve. I will refer to ‘body’ as the general function logic.

2. If options has a and false, then pass options to the body
3. If options has a and b, (and a is not false) pass options to the body
4. If options has a and b, and a is false error out
5. If options has a (and it’s not false), error
6. If options has b only, error.
7. If options has neither a nor b, error out

options in this case might have different keys in it.

This would be easy enough to do with a bunch of if statements, but I’m curious if there’s a more idiomatic way to do this through guards and function declarations.

[NobbZ]

My usual way is to convert Keywordlists to Maps, then use the map:

def my_fun(options, sup_opts) when is_list(options), do: options |> Enum.into(%{}) |> my_fun(sup_opts)
def my_fun(opts = %{a: false, b: _}, sup_opts), do: # this is case 3
def my_fun(opts = %{a: false}, sup_opts), do: # this is case 1
def my_fun(opts = %{a: a, b: b}, sup_opts), do: # this is case 2
def my_fun(opts = %{a: _}, sup_opts), do: # this is case 4
def my_fun(opts = %{b: _}, sup_opts), do: # this is case 5
def my_fun(opts = %{}, sup_opts), do: # this is case 6

beware of the fact, that I had to reorder your requirements in a way that previous patterns do some filtering. Here case 1 will match on :a beeing false, but it will never see a Map containing a :b as key, since those match on the previous clause for case 3, but if you swap them you will never reach case 3, because case 1 would always match.

[peerreynders]

@NobbZ’s approach is superior if you are going convert the keyword list to a Map anyway (for later convenience). But I don’t see anything wrong with the “low-tech” approach if you are trying to stick with the keyword list:

defmodule Demo do

  defp my_fun_do_it(_options, _sup_opts),
    do: IO.puts "Do it!"

  def my_fun(options, sup_opts) do
    case {Keyword.fetch(options, :a), Keyword.fetch(options, :b)} do
      {{:ok, false}, {:ok, _b}} -> IO.puts "Error case 3"
      {{:ok, false}, _}         -> my_fun_do_it(options, sup_opts)
      {{:ok, _a},    {:ok, _b}} -> my_fun_do_it(options, sup_opts)
      {{:ok, _a},    _}         -> IO.puts "Error case 4"
      {:error,       {:ok, _b}} -> IO.puts "Error case 5"
      _                         -> IO.puts "Error case 6"
    end
  end
end

(Demo.my_fun [a: false], [])
(Demo.my_fun [a: nil, b: nil], [])
(Demo.my_fun [a: false, b: nil], [])
(Demo.my_fun [a: nil], [])
(Demo.my_fun [b: nil], [])
(Demo.my_fun [], [])

$ elixir demo.exs
Do it!
Do it!
Error case 3
Error case 4
Error case 5
Error case 6

Note however that there is a subtle difference in the “fetch” and “convert to Map” approach. Given that keyword lists can have multiple occurrences of the same key “fetch” will simply grab the first occurrence while the “conversion to Map” will only contain the last occurrence.

iex(1)> Keyword.fetch([a: false, a: 0],:a)
{:ok, false}
iex(2)> Keyword.fetch([a: 0, a: false],:a)
{:ok, 0}
iex(3)> [a: false, a: 0] |> Enum.into(%{}) 
%{a: 0}

Whether or not that makes a difference depends entirely on the circumstances.

[NobbZ]

@peerreynders

Given that keyword lists can have multiple occurrences of the same key “fetch” will simply grab the first occurrence while the “conversion to Map” will only contain the last occurrence.

Which can be easily worked around:

iex(1)> [a: false, a: 0] |> Enum.reverse |> Enum.into(%{})
%{a: false}

[peerreynders]

@NobbZ

Which can be easily worked around

The fact I was nudging towards was that a keyword list can have multiple values for a single key.

However in 99.9% of the cases there is no intent to support multiple values per key - and nothing in the original post suggested that there was any intent to support multiple :a or :b values.

I suspect keyword lists being the default mechanism for passing options to Elixir functions motivated the choice here - not the fact that a keyword list can have multiple values for the same key.

At the same time I’m not aware of a convention that suggests which value should be given preference when an option appears multiple times in an option list - the one closest to the beginning of the list or the one closest to the end?(‡)

My choice would be to pass options in a Map to begin with but I suspect that the “option keyword list mechanism” was devised during a time where there were no Maps but only HashDicts (Edit: The reasons are actually quite different).

Again - in 99.9% of the cases the possibility of multiple values per key is a non-issue but it probably doesn’t hurt to keep this potential “gotcha” in mind.

Passing in options: Maps vs. Keyword lists

Hey everyone, this has been on my mind for some time and I’d love your input on it! TLDR: I feel like maps are superioer for storing and working with configuration options as compared to keyword lists and am wondering what I might be missing So, in Elixir it seems like Keyword lists are the de facto standard for passing options around. It’s what Mix.Config does and sort of advocates, to my understanding. Also it is advocated in the guides These characteristics are what prompt…

‡ Based on José’s comment:

I agree pattern matching on maps is really handy and for such cases I would likely accept a keyword list but convert it to a map with defaults:

map = Enum.into(user_options, %{key: "default", other_key: false, ...})

so

> Enum.into([a: 2, a: 3], %{a: 1})
%{a: 3}

the last value should likely be the “honoured” value (which interestingly could be the “first” value added to the list when it was created by cons-ing) if only one value is desired. However since something like for cmd_opts reverses the order of the options during validation, I guess, no definitive call can be made.

[OvermindDL1]

@peerreynders

The fact I was nudging towards was that a keyword list can have multiple values for a single key.

…snip…

Again - in 99.9% of the cases the possibility of multiple values per key is a non-issue but it probably doesn’t hurt to keep this potential “gotcha” in mind.

Sounds like you want the Erlang PropertyList format. ^.^

You can query for a single value (gets the first listed in the list). The key can be any term. If a key appears in a list not wrapped in a 2-tuple then the second value is automatically true. You can query for ‘all’ values of a key. Etc… I still think that is what Elixir KWLists should have been, especially as Erlang already had that concept down.

[peerreynders]

You can query for a single value (gets the first listed in the list).

This is what Keyword.fetch/2 does.

You can query for ‘all’ values of a key.

Keyword.get_values/2 which seems to preserve order.

If a key appears in a list not wrapped in a 2-tuple then the second value is automatically true.

You got me there.

My point of contention is that as a data structure the keyword list is ambiguous when I only want at most one value for a particular key - which must be implicitly true for all the keys if I’m about to convert it to a Map.

As such when data such as [a: true, a: false] can enter the system it needs to be sanitized (i.e. duplicates rejected when they are not supported and contradictions rejected when duplicates are supported) - but I suspect that in many cases it will simply be subjected to the “pick the one that happens to be last” selection strategy when it finally hits opts_map = Enum.into(opts_kw, default_map) (which lacks a certain predictability unless you are intimately familiar with the processing that the option list goes through).

[OvermindDL1]

@peerreynders

You got me there.

Also the fact that proplists support any arbitrary term for its key. ^.^

@peerreynders

My point of contention is that as a data structure the keyword list is ambiguous when I only want at most one value for a particular key - which must be implicitly true for all the keys if I’m about to convert it to a Map.

I quite often make a helper to ‘sanitize’ my property lists, throwing an error on unknown ones if they are not supposed to be accepted. Translated to Elixir from Erlang it’s something like (from memory, don’t have it in front of me):

def sanitize_opts(valid_keys, unknown_opt \\ :error, opts) do
  Enum.flat_map(opts, fn
    {key, value} -> sanitize_opts_is_allowed(valid, unknown_opt, key, value)
    key -> sanitize_opts_is_allowed(valid_keys, unknown_opt, key, true)
  end)
end

defp sanitize_opts_is_allowed(valid_keys, unknown_opt, key, value) do
  case :proplists.get_value(key, valid_keys) do
    true -> [{key, value}]
    fun when is_function(fun, 2) -> fun.(key, value)
    :error -> throw {:DENIED_OPT, key}
    :deprecated -> IO.inspect({:DEPRECATED_OPT, key}, label:  :Deprecation); [{key, value}]
    :undefined ->
      case unknown_opt do
        :passthrough -> [{key, value}]
        :prune -> []
        :error -> throw {:INVALID_OPT, key, valid_keys}
      end
  end
end

Then just use it like:

def some_func(a, opts \\ []) do
  sanitize_opts([:a, :b, c: :deprecated, d: &[{&1, &2+1}]], opts)
end

Although I guess in elixir-land the opts should be first for easy piping or something (Elixir is so backwards…). ^.^;
But the above will accept only keys of :a, :b, :c, and :d, where :a and :b are passed through, :c will log a message but otherwise passthrough, and :d will add one to the value but otherwise pass through.

I really should make a helper library of all the things I’ve made over time… >.>

However, for defaults I usually just ++ [the defaults here] the opts as I pass it into that above function.

[peerreynders]

I’m thinking idiomatic “Keywords -> Map” conversion shouldn’t be:

iex(1)> [a: 1, b: 2, a: 3] |> Enum.into(%{})
%{a: 3, b: 2}

but

iex(2)> [a: 1, b: 2, a: 3] |> Enum.reverse() |> Enum.reduce(%{}, fn({k,v}, acc) -> (Map.update acc, k, [v], &([v|&1])) end)
%{a: [1, 3], b: [2]}

[Ryzey]

@NobbZ

def my_fun(options, sup_opts) when is_list(options), do: options |> Enum.into(%{}) |> my_fun(sup_opts)

You could extend the above to include default nil values for a and b:

def my_fun(options, sup_opts) when is_list(options), do: options |> Enum.into(%{a: nil, b: nil}) |> my_fun(sup_opts)

This allows you to be specific in pattern matching that a or b should not have been supplied, which removes the requirement to have functions appear in a specific order.

def my_fun(opts = %{a: false, b: nil}, sup_opts), do: # this is case 1
def my_fun(opts = %{a: true, b: true}, sup_opts), do: # this is case 2
def my_fun(opts = %{a: true, b: false}, sup_opts), do: # this is also case 2

If you do not care about separate functions for the various error conditions, you could then end with:

def myfun(_, _), do: # this is all cases other than 1 and 2

[peerreynders]

@Ryzey

You could extend the above to include default nil values for a and b:

nil is a Value. In some cases (by convention or agreement) nil is used to represent the absence of a value - but in some domains nil could have an entirely different meaning.

In this context the standalone pattern of %{a:, _} will match the “presence of any value (including nil) under the key :a” - so any match subsequent to that pattern implies the absence of the key :a in the map.

Ultimately that is why one has to be careful when using Map.get/3 - by relying on the default, a return value of nil could indicate the absence of the key or that nil was stored under the key (which is why an alternate default can be specified).

It is for this reason that Erlang doesn’t use nil (though :undefined is used when something cannot be found/doesn’t exist).

[Ryzey]

@peerreynders

nil is a Value. In some cases (by convention or agreement) nil is used to represent the absence of a value - but in some domains nil could have an entirely different meaning.

Good point. I guess you could use a custom default value e.g. :no-option that you should then consider to be equivalent whether it was provided as such or assigned as a default.

It would be good if you could explicitly pattern match for undefined values without having to match “_” first.

Edited to include quote.