TaggedTupleShorthand - Field Punning in Elixir

Hey all!

I’m releasing TaggedTupleShorthand today as an experiment with Elixir syntax.

Field Punning

The aim of the library is to support a commonly-requested Elixir feature^{1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17}:

Short-hand for constructing, deconstructing, and pattern matching on key/value pairs in associative data structures, interacting with variable names in the current scope.

This is called “field punning”—essentially trying to turn this

def handle_in(
      event,
      %{
        "chat" => chat,
        "question_id" => question_id,
        "data" => data,
        "attachment" => attachment
      },
      socket
    )
    when is_binary(chat) do...

into this:

def handle_in(event, %{chat, question_id, data, attachment}, socket)
    when is_binary(chat) do...

This syntax sugar exists with many variations in many other languages. It is popular for several reasons:

• This syntax saves on visual noise, expressing destructuring key/value data tersely in the common case of the key making for a sufficient variable name.

• This syntax calls attention to the cases where we are intentionally not re-using the key as a variable name, placing emphasis on a subtle decision a developer decided was important for readability or understanding.

• This syntax prevents common typos, and ensures that variable names match keys throughout refactors when that is the desired behaviour.

Generally speaking, Elixir’s pattern matching and destructuring functionality greatly surpasses other languages, so people are often surprised to discover that no syntax for field punning exists!

Why not?

Well, for a lot of reasons. I encourage you to explore discussions around previous proposals, but in my opinion it boils down to:

• We have two different common associative data structures, Keyword lists and Maps
• We have two different common key types, Atoms and Strings
• We have two different common syntaxes for key/value associativity, arbitrary => value (maps only) and atom: value (atom keys only)
• We have another common container type, Tuples, that look a lot like Maps, and many proposed syntaxes blur the line between them visually
• It is hard to invent a syntax that spans all of these considerations consistently
• Everyone has a different, incompatible preference as to which permutations of these considerations to support and which to ignore completely
• Consensus is hard, syntax sugar is subjective, and a good compromise leaves everyone dissatisfied
• Therefore, a working field punning proposal for the language that satisfies everyone has remained elusive

Anyways, that’s field punning…

Tagged Tuple Shorthand

“Tagged Tuples” describe a pattern where you have a value, and want to “tag” it with information as to what it represents. In Elixir, this is generally accomplished with a two-tuple, where the first element of the tuple is an atom “tag”, and the second element is the value itself:

case try_to_procure_an_integer() do
  result when is_integer(result) -> {:ok, result}
  other ->
    reason = "not an integer: #{inspect other}"
    {:error, reason}
end

Here, {:ok, result} and {:error, reason} are the tagged tuples. This idiom is overwhelmingly common across the ecosystem; you’ve probably wrote similar Elixir code yourself before you learned the term for it.

What if there was a way of constructing, deconstructing, and pattern matching on tagged tuples that tied into variable names in the current scope? An operator for it, even?

Introducing TaggedTupleShorthand.@/1

Let’s devise an operator that takes either an atom or a string, and expands at compile-time to a tagged tuple, with a variable reference mirroring that atom/string as the second element. We’ll use the unary @ operator since it is already overridable via Elixir macros, and has the correct operator arity, precedence, and associativity for this use-case.

use TaggedTupleShorthand

foo = 1
@:foo # aka {:foo, foo}
#=> {:foo, 1}

bar = 2
@"bar" # aka {"bar", bar}
#=> {"bar", 2}

Essentially,

This is… certainly a thing? A novel bit of syntax, hard to acclimate to, but perhaps justifiable in a language with such widespread idiomatic usage of tagged two-tuples?

We could refactor the previous example by choosing our variable names carefully:

use TaggedTupleShorthand

case try_to_procure_an_integer() do
  ok when is_integer(ok) ->
    @:ok # {:ok, ok}
  other ->
    error = "not an integer: #{inspect other}"
    @:error # {:error, error}
end

We could even craft string-tagged tuples, for some reason, why not:

case try_to_procure_an_integer() do
  ok when is_integer(ok) ->
    @"ok" # {"ok", ok} <- nobody wants this
  other ->
    error = "not an integer: #{inspect other}"
    @"error" # {"error", error} <- get this right out of here
end

I’m sure you see where this is going.

Tagged Tuple Shorthand as Field Punning

It turns out, this simple TaggedTupleShorthand.@/1 macro is all we need to get field punning in Elixir, with expressive power inline with our robust pattern matching and exceeding other languages’ capabilities.

The Abstract Syntax Tree for Elixir—its AST that macros can operate on—already represents key/value pairs in map and keyword list literals as tagged tuples. That is, in any literal map or list we might want to use field punning in, we can use a macro that expands into a tagged tuple and a variable reference, without losing any other expressive powers of Elixir’s pattern matching syntax. A macro like TaggedTupleShorthand.@/1.

Put another way, if we can acclimate to this:

foo = 1
@:foo
#=> {:foo, 1}

and learn to swallow/write a linter to forbid this:

ok when is_integer(ok) ->
  @"ok" # {"ok", ok} <- nobody wants this

We can get rich field punning in places like these, and anywhere else two-tuples are used in Elixir AST:

destructure_map = fn %{@:foo, @"bar"} ->
  {foo, bar}
end

map = %{"bar" => 2, foo: 1}
destructure_map.(map)
#=> {1, 2}

Phoenix Channels

Before:

def handle_in(
      event,
      %{
        "chat" => chat,
        "question_id" => question_id,
        "data" => data,
        "attachment" => attachment
      },
      socket
    )
    when is_binary(chat) do...

After:

def handle_in(event, %{@"chat", @"question_id", @"data", @"attachment"}, socket)
    when is_binary(chat) do...

Diff:

-def handle_in(
-      event,
-      %{
-        "chat" => chat,
-        "question_id" => question_id,
-        "data" => data,
-        "attachment" => attachment
-      },
-      socket
-    )
+def handle_in(event, %{@"chat", @"question_id", @"data", @"attachment"}, socket)
     when is_binary(chat) do...

Phoenix Controller Actions

Before:

def show(conn, %{"id" => id, "token" => token}) do
  case Phoenix.Token.decrypt(conn, "file", token, max_age: :timer.minutes(1)) do
    {:ok, %{id: ^id, vsn: 1, size: _size}} ->
     path = MediaLibrary.local_filepath(id)
     do_send_file(conn, path)

    _ ->
      send_resp(conn, :unauthorized, "")
  end
end

After:

def show(conn, %{@"id", @"token"}) do
  case Phoenix.Token.decrypt(conn, "file", token, max_age: :timer.minutes(1)) do
    {:ok, %{@^:id, vsn: 1, size: _size}} ->
     path = MediaLibrary.local_filepath(id)
     do_send_file(conn, path)

    _ ->
      send_resp(conn, :unauthorized, "")
  end
end

Diff:

-def show(conn, %{"id" => id, "token" => token}) do
+def show(conn, %{@"id", @"token"}) do
   case Phoenix.Token.decrypt(conn, "file", token, max_age: :timer.minutes(1)) do
-    {:ok, %{id: ^id, vsn: 1, size: _size}} ->
+    {:ok, %{@^:id, vsn: 1, size: _size}} ->
      path = MediaLibrary.local_filepath(id)
      do_send_file(conn, path)

What do you think?

Have you ever wanted field punning in Elixir? Is it worth introducing a general tagged tuple syntax to get it?

Worth noting that repurposing the module attribute operator @ for field punning is not my preference, nor do I encourage the use of the tagged tuple shorthand outside of field punning—but this is the only way to deliver this feature in library form.

TaggedTupleShorthand is compatible with normal module attribute usage, so you can use it everywhere and still @doc everything to your heart’s content. If this gains popularity and merits a language proposal, we can always introduce a new distinct unary operator as needed and restrict the scope of where this expression is allowed.

Try it out!

Grab the library and play with it in your codebase or console today!

I’m still not sure how I feel about this syntax myself, but now that I have a library as a proof-of-concept, I’m going to start playing with it in personal projects, and I invite you to do the same.

https://github.com/christhekeele/tagged_tuple_shorthand