How have you used Metaprogramming in Elixir?

Chat / Discussions
,
1

Has anyone tried exploring Metaprogramming before. I think it’s a really powerful technique. I was looking for someone who has implemented it on any feature he or she was building. This would help me further know where I can apply it. Thanks @chrismccord …this code is from your book , Metaprogramming Elixir :blush:

defmodule Translator do
  defmacro __using__(_options) do
    quote do
      Module.register_attribute(__MODULE__, :locales,
        accumulate: true,
        persist: false
      )

      import unquote(__MODULE__), only: [locale: 2]
      @before_compile unquote(__MODULE__)
    end
  end

  defmacro __before_compile__(env) do
    compile(Module.get_attribute(env.module, :locales))
  end

  defmacro locale(name, mappings) do
    quote bind_quoted: [name: name, mappings: mappings] do
      @locales {name, mappings}
    end
  end

  def compile(translations) do
    translations_ast =
      for {locale, mappings} <- translations do
        deftranslations(locale, "", mappings)
      end

    quote do
      def t(locale, path, bindings \\ [])
      unquote(translations_ast)
      def t(_locale, _path, _bindings), do: {:error, :no_translation}
    end
  end

  defp deftranslations(locale, current_path, mappings) do
    # deftranslations("en", "", mappings)
    # TBD: Return an AST of the t/3 function defs for the given locale

    # * example of a mapping
    # * flash: [hello: "Hello %{first} %{last}!", bye: "Bye, %{name}!"],
    # * users: [title: "Users"]

    for {key, val} <- mappings do
      # e.g path = append_path("", flash) -> "flash"
      path = append_path(current_path, key)
      # append_path("flash", hello) -> "flash.hello"

      if Keyword.keyword?(val) do
        # deftranslations("en", "flash", [hello: "hello", bye: "bye"])
        deftranslations(locale, path, val)
      else
        quote do
          # t("en", "flash.hello", bindings)
          def t(unquote(locale), unquote(path), bindings) do
            unquote(interpolate(val))
          end
        end
      end
    end
  end

  defp interpolate(string) do
    # TBD interpolate bindings within string
    string
  end

  defp append_path("", next), do: to_string(next)
  defp append_path(current, next), do: "#{current}.#{next}"
end
3 Likes
2

I built an Ecto-ish query builder (not the whole ORM library) for PostgreSQL using metaprogramming. TBH, it’s full of sh*tty code. The good thing is that it’s well-tested.

Why do I need yet another Ecto-ish query builder? Because my team was building something like an OLAP platform. The tables are created by the end users, not by the developers, so there’s no compile-time schemas, and everything needs to go run-time. Use of atoms for naming things is also strictly forbidden because it can cause memory leak.

Then why use metaprogramming? Because I love Ecto’s syntax.

3 Likes
3
  • GenServer is a nice example of how to build a process code around a behaviour, hiding all the barebone actor model behind a client’s callbacks
  • my implementation of stream/1 comprehension is completely written in Elixir AST
  • Telemetría uses metaprogramming to modify existing code injecting :telemetry calls based on user-defined module attributes
  • Finitomata is the same technique of exporting callbacks as GenServer, for distributed FSM
7 Likes
4

I used Elixir meta programming to create a DSL for defining executable business processes. See Mozart and Opera.

2 Likes
5

  1. For a private project I wrote a macro to write boilerplate for me, deriving a lot from schema’s. I did abandon the project long ago but now I am looking at Ash; as it is my dream come true.

  2. An experiment to have call stack based authorization, by decorating function calls. How to get name of 'calling' module and/or function?.

  3. PLR Uses metaprogramming to do all kinds of black magic by rewriting Phoenix routes and creating alternative helpers.

  4. PLR’s successor is Routex which also uses macro’s but aims at being less black magic.

Somewhere between 3 and 4 I read the Metaprogramming book from Chris McCord and some nice blog posts from @sasajuric.

Have fun with macro’s; escape into functions as soon as you possibly can and do not start writing macro’s that write macro’s :wink:

4 Likes
6

You mean to say that its library for generating DSLs is doing what yours did (and maybe better)? Because if not, yours can be valuable to open-source.

7

writing macros that write macros seems like a perilous idea :joy: :skull:

8

LISP-ers have been doing it for decades. They have meta-meta-meta-programming. :003:

2 Likes
9

Don’t get me started on the metacircular evaluator and the meta object protocol!

2 Likes
10

We really need to document this better, but all of our DSLs (which support all kinds of useful things, are type validated, have an ElixirLS extension for customized autocomplete, can accept complex values like anonymous functions, etc.) are all built using spark: Spark — spark v2.2.23

Spark takes a declarative structure, and generates a DSL from that structure.

For a very simple example:

defmodule MyApp.Dsl.Extension do
  @dsl %Spark.Dsl.Section{
    name: :dsl,
    schema: [
      foo: [
        type: :integer, 
        doc: "The amount of coolness to add", 
        default: 100
      ]
    ]
  }

  use Spark.Dsl.Extension, sections: [@dsl]
end

defmodule MyApp.Dsl do
    use Spark.Dsl, default_extensions: [extensions: [MyApp.Dsl.Extension]]
end

defmodule MyApp.Dsl.Info do
  use Spark.InfoGenerator, extension: MyApp.Dsl.Extension, sections: [:dsl]

end

Then you can use it like so:

defmodule Something do
  use MyApp.Dsl

  dsl do
    foo 10
  end
end

# and introspect it

MyApp.Dsl.Info.dsl_foo(Something) 
# => {:ok, 10}

That example is just the tip of the iceberg. It looks like “macro magic” but in reality it’s a thin veneer over a data structure that gives you elixir-specific niceties and handles the complexity that comes with building these kinds of things.

So you can write a DSL without having to write a single macro, we write the macros for you :laughing: Macros writing macros, there are cases where it makes sense :person_shrugging:

9 Likes
11

My only concern about Spark is that it lacks documentation, are you planning on writing documentation for that project or it is more of a internal project that powers Ash?

12

I second that. Spark docs would be great.

13

Spark is its own project in its own right, unrelated to Ash. In fact I think if it gets some half-decent documentation, I’d expect it to be more popular than Ash :laughing:.The only problem is one of time on my end. It is fully supported (separate from its usage in Ash) and there are other non-ash projects using it. They just had to brave its lack of docs. But it is stable and feature-full. Documenting it has always been one of those “as soon as I have some time” projects and, lo and behold the day never comes.

4 Likes
14

This is great to hear.

I don’t think the documentation needs a lot of effort, a few examples of different possibilities in readme would be more than enough to get me going, as everything else related to available macros/functions is pretty well documented.

1 Like
15

True, it’s likely an 80/20 thing, can get 80% of the value without much effort. I’ll put something on my list, will probably be next week or the week after at the earliest I can get to it.

1 Like
16

I’ll join @D4no0 here in saying that your target audience is programmers and they learn pretty well by example. Having 2-3 practical examples will increase confidence in your library, like a lot.

1 Like
17

True dat. I feel this way about man pages—the only reason people think they suck is because they lack examples (I understand it’s a storage concern but still).

18

Well, we have about 35 in the form of every DSL & extension written for Ash :laughing: And the test suite defines some more :stuck_out_tongue:

EDIT: here is a small one, for example: ash_archival/lib/ash_archival/resource/resource.ex at main · ash-project/ash_archival · GitHub

2 Likes
19

No it did not generate DSL, but functions for schema’s. So MyStuct would cause a list_my_structs/1 with field filters.

Ash does a lot more than what my macro’s did and in a composable way. Oh, and Ash has a lot more documentation!

20

I built a flexible destructuring library: