tmbb

DarkKernel - elixir functions which use dark magic

Dark Kernel

Link here: GitHub - tmbb/elixir_dark_kernel: Elixir functions that use dark magic · GitHub
(not available on hex yet)

A couple years ago Andrea Leopardi and Chris Meyer created some pretty cool libraries to add ES6-style shorthand maps. In elixir notation, one would be able to write somthing like %{a, b, c} which would be equivalent to %{a: a, b: b, c: c}. It turns out the libraries didn’t get much use. I’ve always thought that they were very intersting libraries and came up with the idea to extend those concepts to keyword lists.

This library is the result of those ideas. It is very simple, and it was coded in an afternoon, but I think it has some pretty cool capabilities. The library embeds the source of ShorterMaps, which I plan to extend in the near future. @OvermindDL1 might enjoy this.

Despite having named this library the “Dark Kernel”, I actually think it could be useful in real-world projects. It does have some “magic” (it calls Code.string_to_quoted!/1 at least once), but all (dark!) magic works at compile time, and the macroexpanded code is not that complex.

Dark keyword lists

Meet dark keyword lists, the simplest form:

import DarkKernel

opts = [a: 1, b: 2, c: 3]
# Pattern match the keyword list against an existing variable
~k[a, b, c, d = opts]

assert a == 1
assert b == 2
assert c == 3
assert d == nil

Dark keyword lists with constant defaults:

import DarkKernel

opts = [a: 1, b: 2]
# Pattern match the keyword list against an existing variable
~k[a, b, c: "a string", d: :an_atom, e: 42, f = opts]

assert a == 1
assert b == 2
assert c == "a string"
assert d == :an_atom
assert e == 42
assert f == nil

Dark keyword lists with defaults which are arbitrary expressions:

import DarkKernel

opts = [a: 1, b: 2]
# Pattern match the keyword list against an existing variable
~k[a, b, c: 1 + 2, d: :rand.uniform(), e: 3.14 + :rand.uniform() = opts]

assert a == 1
assert b == 2
assert c == 3
assert is_float(d)
assert is_float(e)

Instead of returning nil for missing keys, we can raise an error. To do it, just prefix the required key with a bang (!):

iex(1)> import DarkKernel
DarkKernel

iex(2)> opts = [a: 1, b: 2, c: 3]
[a: 1, b: 2, c: 3]

# This will bind the variable to `nil`
iex(3)> ~k[a, b, x = opts]
[a: 1, b: 2, x: nil]
iex(4)> x                   
nil

# This will raise an error
iex(5)> ~k[a, b, !y = opts]
** (KeyError) key :y not found in: [a: 1, b: 2, c: 3]
    (elixir 1.14.1) lib/keyword.ex:595: Keyword.fetch!/2
    iex:5: (file)
iex(5)>

Dark maps

Dark maps are currently implemented using ShorterMaps. Due to dificulties of using a macro inside another macro, and because I wanted all sigils to be importable from the same module, dark_kernel embeds the source code of ShorterMaps.

In the (near) future, dark_kernel will support more advanced pattern matching capabilities for dark maps, similar to the ones supported by dark keyword lists.

iex(1)> import DarkKernel
DarkKernel
iex(2)> ~M{a, b, c} = %{a: 1, b: 2, c: 3}
%{a: 1, b: 2, c: 3}
iex(3)> a
1
iex(4)> b
2
iex(5)> c
3
iex(6)>

11 comments

#library #quality #macro #dark_kernel

3 1038 11

2022-12-11 18:33:26 UTC

Most Liked

al2o3cr

Some initial reactions to this syntax:

the = inside the brackets feels strange, but that may be because it makes me think of Ruby’s default argument syntax.
having a d in the “pattern” but then not failing to match feels odd compared to matching the literal forms where those keys are required to match

A less vibes-based issue I encountered trying this out is that it fails if a default expression calls a function with two arguments:

iex(3)> opts = [a: 1, b: 2]
[a: 1, b: 2]
iex(4)> ~k{a, b, c: Integer.pow(2, 4) = opts}
** (DarkKernel.DarkKeywords.InvalidKeywordError) Invalid keyword: 4)
Keywords must be valid Elixir variable names and contain only ASCII characters.
They can be preceded by a bang (e.g. `!key`) or contain a default value (e.g. `key: default`).

    (dark_kernel 0.1.0) lib/dark_kernel/dark_keywords.ex:116: anonymous fn/2 in DarkKernel.DarkKeywords.parse_keyword_match/1
    (elixir 1.14.0) lib/enum.ex:2468: Enum."-reduce/3-lists^foldl/2-0-"/3
    (dark_kernel 0.1.0) lib/dark_kernel/dark_keywords.ex:103: DarkKernel.DarkKeywords.parse_keyword_match/1
    (dark_kernel 0.1.0) lib/dark_kernel/dark_keywords.ex:193: DarkKernel.DarkKeywords.do_match_keywords/1
    (dark_kernel 0.1.0) expanding macro: DarkKernel.sigil_k/2
    iex:4: (file)

or if a default argument is a literal with a comma in it:

iex(4)> ~k{a, b, c: "Hello, world!" = opts}
** (DarkKernel.DarkKeywords.InvalidKeywordError) Invalid keyword: world!"
Keywords must be valid Elixir variable names and contain only ASCII characters.
They can be preceded by a bang (e.g. `!key`) or contain a default value (e.g. `key: default`).

    (dark_kernel 0.1.0) lib/dark_kernel/dark_keywords.ex:116: anonymous fn/2 in DarkKernel.DarkKeywords.parse_keyword_match/1
    (elixir 1.14.0) lib/enum.ex:2468: Enum."-reduce/3-lists^foldl/2-0-"/3
    (dark_kernel 0.1.0) lib/dark_kernel/dark_keywords.ex:103: DarkKernel.DarkKeywords.parse_keyword_match/1
    (dark_kernel 0.1.0) lib/dark_kernel/dark_keywords.ex:193: DarkKernel.DarkKeywords.do_match_keywords/1
    (dark_kernel 0.1.0) expanding macro: DarkKernel.sigil_k/2
    iex:4: (file)

In at least these two cases, Code.string_to_quoted! could be helpful if you wrap the whole expression passed to ~k in [ ]:

iex(5)> Code.string_to_quoted!("[a, b, c: \"Hello, world!\" = opts]")
[
  {:a, [line: 1], nil},
  {:b, [line: 1], nil},
  {:c, {:=, [line: 1], ["Hello, world!", {:opts, [line: 1], nil}]}}
]

iex(6)> Code.string_to_quoted!("[a, b, c: Integer.pow(2, 4) = opts]")
[
  {:a, [line: 1], nil},
  {:b, [line: 1], nil},
  {:c,
   {:=, [line: 1],
    [
      {{:., [line: 1], [{:__aliases__, [line: 1], [:Integer]}, :pow]},
       [line: 1], [2, 4]},
      {:opts, [line: 1], nil}
    ]}}
]

A little (ok, probably LOTS) of pattern matching should be able to split / transform these into the output AST. It could even support cases like two equals signs that currently raise an exception:

iex(8)> Code.string_to_quoted!("[!a, b: (_ = 1 + 2) = opts]")      
[
  {:!, [line: 1], [{:a, [line: 1], nil}]},
  {:b,
   {:=, [line: 1],
    [
      {:=, [line: 1], [{:_, [line: 1], nil}, {:+, [line: 1], [1, 2]}]},
      {:opts, [line: 1], nil}
    ]}}
]

Post #2

tmbb

Thanks for the feedback!

True, I think that’s pretty strange too, but it’s still the most succint way I’ve found support this kind of matches with a sigil. For very good reasons, the sigil can’t influence what’s outside the sigil! This means that all “magic” syntax must happen inside the sigil.

Ideally, it should be something like this:

~k[a, b, c, d] = opts

but I don’t think I can make this work with Elixir’s macro system. I could use one of the undefined operators for something like this:

~k[a, b, c, d] <~ opts

What do you think?

Ok, maybe a match is not the right word for this. The goal of these keyword “matches” is actually to extract variables from keyword lists. I’ll think of a better name/better syntax

You’re probably right. This library kinda grew up “organically”, in that I started with something very simple based on regexes (actually not even regexes, I just used String.split()) and then I started adding to it. But I should look into implementing it with Code.string_to_quoted!/1.

Post #3

christhekeele

Yep, personally I would!

Post #11

Last Post!

tmbb

I’ve already implemented that as an operator. When debugging is set to active it prints the generated code.

iex(43)> import DarkKernel
DarkKernel
iex(44)> ~k[a, b, c] <~ [b: 1, c: 2, a: 3]
[a: 3, b: 1, c: 2]
iex(45)>
18:32:00.996 [debug] iex:44
┌────────────────────────────────────────────────────────────────────────────────────────────────────
│ ⤷ Code:
├╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶
│ ~k[a, b, c] <~ [b: 1, c: 2, a: 3]
├────────────────────────────────────────────────────────────────────────────────────────────────────
│ ⤷ Generated code:
├╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶
│ constant_kw_list = [b: 1, c: 2, a: 3]
│ a = Keyword.get(constant_kw_list, :a, nil)
│ b = Keyword.get(constant_kw_list, :b, nil)
│ c = Keyword.get(constant_kw_list, :c, nil)
│ [a: a, b: b, c: c]
├────────────────────────────────────────────────────────────────────────────────────────────────────
│ ⤷ Generated AST:
├╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶
│ {:__block__, [],
│  [
│    {:=, [],
│     [
│       {:constant_kw_list, [counter: -576460752303417279],
│        DarkKernel.DarkKeywords},
│       [b: 1, c: 2, a: 3]
│     ]},
│    {:=, [],
│     [
│       {:a, [], nil},
│       {{:., [], [{:__aliases__, [alias: false], [:Keyword]}, :get]}, [],
│        [
│          {:constant_kw_list, [counter: -576460752303417279],
│           DarkKernel.DarkKeywords},
│          :a,
│          nil
│        ]}
│     ]},
│    {:=, [],
│     [
│       {:b, [], nil},
│       {{:., [], [{:__aliases__, [alias: false], [:Keyword]}, :get]}, [],
│        [
│          {:constant_kw_list, [counter: -576460752303417279],
│           DarkKernel.DarkKeywords},
│          :b,
│          nil
│        ]}
│     ]},
│    {:=, [],
│     [
│       {:c, [], nil},
│       {{:., [], [{:__aliases__, [alias: false], [:Keyword]}, :get]}, [],
│        [
│          {:constant_kw_list, [counter: -576460752303417279],
│           DarkKernel.DarkKeywords},
│          :c,
│          nil
│        ]}
│     ]},
│    [a: {:a, [], nil}, b: {:b, [], nil}, c: {:c, [], nil}]
│  ]}
└────────────────────────────────────────────────────────────────────────────────────────────────────


nil
iex(46)> a
3
iex(47)> b
1
iex(48)> c
2
iex(49)>

Post #12