tmbb

tmbb

Dark Kernel

Link here: GitHub - tmbb/elixir_dark_kernel: Elixir functions that use dark magic · GitHub
(not available on hex yet)

A couple years ago Andrea Leopardi and Chris Meyer created some pretty cool libraries to add ES6-style shorthand maps. In elixir notation, one would be able to write somthing like %{a, b, c} which would be equivalent to %{a: a, b: b, c: c}. It turns out the libraries didn’t get much use. I’ve always thought that they were very intersting libraries and came up with the idea to extend those concepts to keyword lists.

This library is the result of those ideas. It is very simple, and it was coded in an afternoon, but I think it has some pretty cool capabilities. The library embeds the source of ShorterMaps, which I plan to extend in the near future. @OvermindDL1 might enjoy this.

Despite having named this library the “Dark Kernel”, I actually think it could be useful in real-world projects. It does have some “magic” (it calls Code.string_to_quoted!/1 at least once), but all (dark!) magic works at compile time, and the macroexpanded code is not that complex.

Dark keyword lists

Meet dark keyword lists, the simplest form:

import DarkKernel

opts = [a: 1, b: 2, c: 3]
# Pattern match the keyword list against an existing variable
~k[a, b, c, d = opts]

assert a == 1
assert b == 2
assert c == 3
assert d == nil

Dark keyword lists with constant defaults:

import DarkKernel

opts = [a: 1, b: 2]
# Pattern match the keyword list against an existing variable
~k[a, b, c: "a string", d: :an_atom, e: 42, f = opts]

assert a == 1
assert b == 2
assert c == "a string"
assert d == :an_atom
assert e == 42
assert f == nil

Dark keyword lists with defaults which are arbitrary expressions:

import DarkKernel

opts = [a: 1, b: 2]
# Pattern match the keyword list against an existing variable
~k[a, b, c: 1 + 2, d: :rand.uniform(), e: 3.14 + :rand.uniform() = opts]

assert a == 1
assert b == 2
assert c == 3
assert is_float(d)
assert is_float(e)

Instead of returning nil for missing keys, we can raise an error. To do it, just prefix the required key with a bang (!):

iex(1)> import DarkKernel
DarkKernel

iex(2)> opts = [a: 1, b: 2, c: 3]
[a: 1, b: 2, c: 3]

# This will bind the variable to `nil`
iex(3)> ~k[a, b, x = opts]
[a: 1, b: 2, x: nil]
iex(4)> x                   
nil

# This will raise an error
iex(5)> ~k[a, b, !y = opts]
** (KeyError) key :y not found in: [a: 1, b: 2, c: 3]
    (elixir 1.14.1) lib/keyword.ex:595: Keyword.fetch!/2
    iex:5: (file)
iex(5)>

Dark maps

Dark maps are currently implemented using ShorterMaps. Due to dificulties of using a macro inside another macro, and because I wanted all sigils to be importable from the same module, dark_kernel embeds the source code of ShorterMaps.

In the (near) future, dark_kernel will support more advanced pattern matching capabilities for dark maps, similar to the ones supported by dark keyword lists.

iex(1)> import DarkKernel
DarkKernel
iex(2)> ~M{a, b, c} = %{a: 1, b: 2, c: 3}
%{a: 1, b: 2, c: 3}
iex(3)> a
1
iex(4)> b
2
iex(5)> c
3
iex(6)>

Showing Posts 1 to 10

al2o3cr

al2o3cr

Some initial reactions to this syntax:

  • the = inside the brackets feels strange, but that may be because it makes me think of Ruby’s default argument syntax.

  • having a d in the “pattern” but then not failing to match feels odd compared to matching the literal forms where those keys are required to match

A less vibes-based issue I encountered trying this out is that it fails if a default expression calls a function with two arguments:

iex(3)> opts = [a: 1, b: 2]
[a: 1, b: 2]
iex(4)> ~k{a, b, c: Integer.pow(2, 4) = opts}
** (DarkKernel.DarkKeywords.InvalidKeywordError) Invalid keyword: 4)
Keywords must be valid Elixir variable names and contain only ASCII characters.
They can be preceded by a bang (e.g. `!key`) or contain a default value (e.g. `key: default`).

    (dark_kernel 0.1.0) lib/dark_kernel/dark_keywords.ex:116: anonymous fn/2 in DarkKernel.DarkKeywords.parse_keyword_match/1
    (elixir 1.14.0) lib/enum.ex:2468: Enum."-reduce/3-lists^foldl/2-0-"/3
    (dark_kernel 0.1.0) lib/dark_kernel/dark_keywords.ex:103: DarkKernel.DarkKeywords.parse_keyword_match/1
    (dark_kernel 0.1.0) lib/dark_kernel/dark_keywords.ex:193: DarkKernel.DarkKeywords.do_match_keywords/1
    (dark_kernel 0.1.0) expanding macro: DarkKernel.sigil_k/2
    iex:4: (file)

or if a default argument is a literal with a comma in it:

iex(4)> ~k{a, b, c: "Hello, world!" = opts}
** (DarkKernel.DarkKeywords.InvalidKeywordError) Invalid keyword: world!"
Keywords must be valid Elixir variable names and contain only ASCII characters.
They can be preceded by a bang (e.g. `!key`) or contain a default value (e.g. `key: default`).

    (dark_kernel 0.1.0) lib/dark_kernel/dark_keywords.ex:116: anonymous fn/2 in DarkKernel.DarkKeywords.parse_keyword_match/1
    (elixir 1.14.0) lib/enum.ex:2468: Enum."-reduce/3-lists^foldl/2-0-"/3
    (dark_kernel 0.1.0) lib/dark_kernel/dark_keywords.ex:103: DarkKernel.DarkKeywords.parse_keyword_match/1
    (dark_kernel 0.1.0) lib/dark_kernel/dark_keywords.ex:193: DarkKernel.DarkKeywords.do_match_keywords/1
    (dark_kernel 0.1.0) expanding macro: DarkKernel.sigil_k/2
    iex:4: (file)

In at least these two cases, Code.string_to_quoted! could be helpful if you wrap the whole expression passed to ~k in [ ]:

iex(5)> Code.string_to_quoted!("[a, b, c: \"Hello, world!\" = opts]")
[
  {:a, [line: 1], nil},
  {:b, [line: 1], nil},
  {:c, {:=, [line: 1], ["Hello, world!", {:opts, [line: 1], nil}]}}
]

iex(6)> Code.string_to_quoted!("[a, b, c: Integer.pow(2, 4) = opts]")
[
  {:a, [line: 1], nil},
  {:b, [line: 1], nil},
  {:c,
   {:=, [line: 1],
    [
      {{:., [line: 1], [{:__aliases__, [line: 1], [:Integer]}, :pow]},
       [line: 1], [2, 4]},
      {:opts, [line: 1], nil}
    ]}}
]

A little (ok, probably LOTS) of pattern matching should be able to split / transform these into the output AST. It could even support cases like two equals signs that currently raise an exception:

iex(8)> Code.string_to_quoted!("[!a, b: (_ = 1 + 2) = opts]")      
[
  {:!, [line: 1], [{:a, [line: 1], nil}]},
  {:b,
   {:=, [line: 1],
    [
      {:=, [line: 1], [{:_, [line: 1], nil}, {:+, [line: 1], [1, 2]}]},
      {:opts, [line: 1], nil}
    ]}}
]
tmbb

tmbb OP

Thanks for the feedback!

True, I think that’s pretty strange too, but it’s still the most succint way I’ve found support this kind of matches with a sigil. For very good reasons, the sigil can’t influence what’s outside the sigil! This means that all “magic” syntax must happen inside the sigil.

Ideally, it should be something like this:

~k[a, b, c, d] = opts

but I don’t think I can make this work with Elixir’s macro system. I could use one of the undefined operators for something like this:

~k[a, b, c, d] <~ opts

What do you think?

Ok, maybe a match is not the right word for this. The goal of these keyword “matches” is actually to extract variables from keyword lists. I’ll think of a better name/better syntax

You’re probably right. This library kinda grew up “organically”, in that I started with something very simple based on regexes (actually not even regexes, I just used String.split()) and then I started adding to it. But I should look into implementing it with Code.string_to_quoted!/1.

tmbb

tmbb OP

Ok, to my great surprise, both:

[a, !b, c: default = opts]

and

[a, !b, c = opts]

are valid elixir syntax.

This means that using Code.string_to_quoted! will work to parse ~k[] expressions. Now the only question is whether I should keep the equals sign inside the sigil or move it outside with something like ~k[a, b, c, d] <~ opts. I guess I might prefer the ~k[a, b, c, d = opts] because it suggests we’re actually using a new magic syntax instead of importing an operator out of the blue… Also, the Elixir assignment operator is = and not <~, so if we’re to extend the syntax, I guess we should try to keep the =.

cmo

cmo

It’s a list with three elements, where you’re matching opts on the last element. Which is why ~k[a, b, c, d = opts] that matches the list, not d on opts is rather odd syntax for Elixir.

christhekeele

christhekeele

One option you may consider is using <-. You should be able to do this inside the sigil without Elixir disliking it; and it’s kind of the language standard operator for enumerable on the right goes into left in some unique way in this context (see: comprehensions, with statements).

tmbb

tmbb OP

Yes, I understand what Elixir thinks that syntax is. It just surprises me that it’s valid. Elixir’s syntax is quite quirky after all. The newest version uses Code.string_to_quoted!/1 to parse the list and operates on that.

tmbb

tmbb OP

That’s a nice idea. It took me about 15s to add support for <- as well as = inside the ~k[...] expression. You can now do this:

opts = [a: 1, b: 2, c: 3]

assert Keyword.keys(~k[a, b <- opts]) == [:a, :b]
assert Keyword.keys(~k[a, b = opts]) == [:a, :b]

I rather like the <- syntax. Which one do you prefer:

# Option 1)
~k[a, b <- opts]

or

# Option 2)
~k[a, b] <~ opts

I think I prefer the first one.

christhekeele

christhekeele

I’d agree with @cmo:

tmbb

tmbb OP

So you prefer the <~ operator outside, right? like this: ~k[a, b, c] <~ opts

christhekeele

christhekeele

Yep, personally I would!

Where Next? Top

Trending in Announcing Top

bluzky
You may know https://ui.shadcn.com/, a UI component library for React. I really love it’s design style and components. I’ve built some co...
387 15136 120
New
wojtekmach
Hey everyone! Req is an HTTP client for Elixir that I’ve been working on for quite some time. There is already a lot of HTTP clients out...
New
handnot2
Samly can be used to enable SAML 2.0 Single Sign On in a Plug/Phoenix application. This library uses Erlang esaml to provide plug enabl...
New
woylie
Flop is an Elixir library that applies filtering, ordering and pagination parameters to your Ecto queries. offset-based pagination with...
New
restlessronin
The repo is at GitHub - cyberchitta/openai_ex: Community maintained Elixir library for OpenAI API · GitHub. Docs are at OpenaiEx User Gu...
152 11030 135
New
JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
shahryarjb
The Chelekom project is a library of Phoenix and LiveView components generated via Mix tasks to fit developer needs seamlessly. One of i...
New

Other Trending Topics Top

mudasobwa
I am seeing a lot of aplications of Argumentum ad Vericundiam in software discussions. They do link some piece of writing and point us to...
New
bartblast
Hey folks, I just published a post about Hologram’s funding and where the project goes next - the short version: Curiosum as Main Spons...
New
alexslade
Fly’s CEO posted this recently - Turn And Face The Strange · The Fly Blog It says that Fly is going all-in on sprites, which is a worry ...
New
Herve37
We’re evaluating API mocking tools for OpenAPI-based projects and would love to hear what other teams are using. We’re particularly inte...
New
sorenone
Today we’re releasing Oban for Python. Not an Oban client in Python. Not a pythonx wrapper embedded in Elixir. Nope, it’s a fully operati...
New
lawik
I was thinking since Goatmire Elixir turned out pretty good I should maybe do another one. 30th of Sep - 2nd of Oct this year./ The firs...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews