Kword - A library of keyword-list handling functions to complement Keyword

Your Libraries & OS Mentoring Libraries
, ,
1

A recent Reddit post Keyword.get Considered Harmful nudged me to tidy up and publish a small utility library that I’ve been using to deal with keyword lists in an ergonomic way.

Elixir keyword lists as a common representation of optional arguments to functions feels a bit clunky. You may find yourself writing something like:

  def update_user_details(opts \\ []) do
    opts = Keyword.validate!(opts, [:name, :email, role: :guest, gender: :unspecified])
    name = Keyword.fetch!(opts, :name)
    email = Keyword.fetch!(opts, :email)
    ...
  end

The intent of Kword is to enable list matching (there’s an order of parameters in the second argument) and to write instead:

  def update_user_details(opts \\ []) do
    [name, email | _rest] = Kword.extract!(opts, [:name, :email, role: :guest, gender: :unspecified])
    ...
  end

If you want to ensure required parameters are in fact supplied, use extract or extract!.

If you don’t want to allow parameters that aren’t specified, use extract_exhaustive or extract_exhaustive!.

And if you just want to pluck values out of a keyword list in the order specified, use extract_permissive which will default parameters to nil that have no default specified.

Perhaps I’ve missed something and this library isn’t actually useful, or there may be some improvement that would make it more worthwhile.

6 Likes
2

I also read that post and liked it very much, however I think it missed one opportunity to make the code better with Keyword.validate!:

def update_user_details(opts \\ []) do
  opts = Keyword.validate!(opts, [:name, :email, role: :guest, gender: :unspecified]) |> Map.new()
  %{name: name, email: email} = opts
  # ...
end

By simply converting validated keyword list to a map you can use established Elixir idioms to check for required things, either by using opts.email notation, or by pattern matching. But maybe I’m missing something too :wink:

2 Likes
3

It highly depends on what kind of options you accept and how you treat them. The key concept when using keyword list vs map is that you can have duplicated keys in case of the list, then decide whether the new option overrides/appends to the old one.

2 Likes
4

“Considered Harmful” titles always make me think of “Considered Harmful” Essays Considered Harmful :grin:

I’m in favour of validating keyword args and do it myself (personally I use NimbleOptions) but that article hardcore handwaves through the entire premise of how they got there: how on earth did their tests not catch a mis-spelled option?? It sounds like they were passing the option but not actually asserting on its effects. Am I wrong or is there an obvious valid hiccup you can have here?

4 Likes
5

While it’s getting outside the realm of options, keywords also allow us to have order matter in the rarer situations where that’s useful:

from q in query,
  join: u in User,
  on: u.id == q.user_id,
  join: o in Org,
  on: o.id == u.org_id
4 Likes
6

Fair point, but with using Keyword.get or Keyword.pop you don’t really decide, just take the first value. Most of the usage of keyword lists outside of Ecto I see is a workaround for Elixir not having named arguments, so converting to map makes sense. However you’re right that to keep Keyword.gets behaviour in place you actually need Enum.reverse() |> Map.new().

Well, but we are in the realm of options.

1 Like
7

I support everything that helps people make less mistakes and dynamic languages like Elixir don’t provide as much protection there.

So firstly, good job. :+1:

Secondly, the linked article does not sell the resulting library to me. Keyword.get and Map.get are something that many Elixir devs, myself included, consider an anti-pattern simply because they don’t discern between “I don’t have the key” and “I have the key but the value is nil” – in some situations this difference is meaningless but I’d bravely claim that in at least 80%, if not 90%, of the Elixir code I had to author or maintain that difference was in fact important but people ignored it and introduced bugs. So just by using functions like take and fetch you can replace most of the conveniences of this library – though granted, it would take more boilerplate so the library still looks compelling.

Thirdly, not comparing the new library with NimbleOptions is giving homework to the reader so I am going to skip it and just use the former instead.

3 Likes
8

The discussion is turning into a more general “why keyword lists over maps or bringing in keyword args,” so not quite, no.

9

Converting to Map was my first impulse, and I guess I should benchmark it but seems a bit heavyweight compared to constructing a list. Also there is the gotcha of Map.new taking the last one if a key occurs more than once.

10

I have seen NimbleOptions but bounced off when I saw how verbose it was, defining a schema with types. Looking closer now I do think there would be times I’d want exactly that.

The aim of Kword is to provide a few relatively fast functions that I found myself wishing were in the Keyword module.

2 Likes
11

Oh ya, I got that! I wasn’t speaking against the utility of your library, just criticizing the article you linked.

1 Like
12

I think he meant to reply to me, maybe. But yeah I looked at the library and I find it nice on a second look.

2 Likes
13

I definitely want to know more on the tradeoffs with this stuff, and this thread has been very informative. The feedback has been constructive, thanks :slightly_smiling_face:

2 Likes
14

For me I usually want it all with typing or I just use Keyword.get, which I do not agree is “harmful”. I don’t want to get to into it, though, otherwise @dimitarvp and I might get going :sweat_smile: I could actually see your lib as a nice middle ground. I bookmarked it and will try to remember to try it next time I go to use Keyword.get.

EDIT: Maybe I do consider it a little harmful :rofl:

15

As with everything else we use, it is considered harmful when it expresses the wrong intent.

I use Keyword.get/3 in situations where an option might be missing and I have a default value for it. For example:

Keyword.get(opts, :option, default_value)

Using it in cases where having that key missing is invalid logic, you are just propagating a bug further into the system. This is especially bad when you leave the default value to nil and propagate it further, hard to debug and trace down.

3 Likes
16

Noooo arguments here!

It’s mostly I’m starting to somewhat buy @dimitarvp’s argument that it doesn’t make a distinction if the key is there or not. Now, this is not news to me, not by a long shot! My of use Keyword.get is for simple options where I don’t want to break out NimbleOptions, but I still operate under the idea that there is a contract in place. IE, I create a world where nil is never going to be passed (which is the whole “parse don’t validate” thing I love to talk about ad nauseam). I would never use Keyword.get for user-passed data!!! However, in the “real world” where juniors are running rampant on your dynamic-language codebase, maybe a nil slips in there. AFAIC, if you’re using nil for its intended use to mean “nothing”, when it comes to options there shouldn’t be a difference between nil and “key wasn’t specified.” If you buy that (and maybe you don’t!), Keyword.get is actually dangerous:

Keyword.get([some_list: nil], :some_list, []) #=> nil

Undoubtedly to @dimitarvp’s horror, one conclusion I draw from this is that Access is actually better:

some_list = options[:some_list] || [] #=> []

Of course this sucks if your option is a boolean. You can’t do:

some_bool = options[:drop_database] || true

We could always exploit truthiness there but even as a fan of truthiness I don’t like this one bit.

Circling back, having what should be a list become a nil should fail pretty fast but I guess you never know in this scenario.

So really I’m starting to think I just always want typed options. Even the other day I passed a non-existant option to a well-known library and it didn’t yell at me and I was pretty annoyed. There certainly is a difference between library and application code, though, so I’ll see :grin:

2 Likes
17

This is true 100%. It is strange to me how OTP has checks for options from long ago, but elixir never adopted that convention until lately.

1 Like
18

On a tangent, Perl5 has a // operator, which is like the logical or || but only treats undef as falsey. Useful for disambiguating nil/null/undefined/missing values from defined but false ones.

3 Likes
19

Maybe a little bit of scope might help here. How long do you think keyword lists will get in the context of passing them into functions? 10, 100 but even if it had 1000 I would not consider optimizing and simply do the following

opts
|> Keyword.validate!(...)
|> Enum.into(%{})
|> Map.values

and similar for Keyword.validate/2 (with a with, I guess :wink: )

20

Have to correct myself, (bad Ruby influence :angel: ) Map.values will not get us the correct order