How have you used Metaprogramming in Elixir?

I built an Ecto-ish query builder (not the whole ORM library) for PostgreSQL using metaprogramming. TBH, it’s full of sh*tty code. The good thing is that it’s well-tested.

Why do I need yet another Ecto-ish query builder? Because my team was building something like an OLAP platform. The tables are created by the end users, not by the developers, so there’s no compile-time schemas, and everything needs to go run-time. Use of atoms for naming things is also strictly forbidden because it can cause memory leak.

Then why use metaprogramming? Because I love Ecto’s syntax.

• GenServer is a nice example of how to build a process code around a behaviour, hiding all the barebone actor model behind a client’s callbacks
• my implementation of stream/1 comprehension is completely written in Elixir AST
• Telemetría uses metaprogramming to modify existing code injecting :telemetry calls based on user-defined module attributes
• Finitomata is the same technique of exporting callbacks as GenServer, for distributed FSM

I used Elixir meta programming to create a DSL for defining executable business processes. See Mozart and Opera.

1. For a private project I wrote a macro to write boilerplate for me, deriving a lot from schema’s. I did abandon the project long ago but now I am looking at Ash; as it is my dream come true.

2. An experiment to have call stack based authorization, by decorating function calls. How to get name of 'calling' module and/or function?.

3. PLR Uses metaprogramming to do all kinds of black magic by rewriting Phoenix routes and creating alternative helpers.

4. PLR’s successor is Routex which also uses macro’s but aims at being less black magic.

Somewhere between 3 and 4 I read the Metaprogramming book from Chris McCord and some nice blog posts from @sasajuric.

Have fun with macro’s; escape into functions as soon as you possibly can and do not start writing macro’s that write macro’s

You mean to say that its library for generating DSLs is doing what yours did (and maybe better)? Because if not, yours can be valuable to open-source.

writing macros that write macros seems like a perilous idea

LISP-ers have been doing it for decades. They have meta-meta-meta-programming.

Don’t get me started on the metacircular evaluator and the meta object protocol!

We really need to document this better, but all of our DSLs (which support all kinds of useful things, are type validated, have an ElixirLS extension for customized autocomplete, can accept complex values like anonymous functions, etc.) are all built using spark: Spark — spark v2.7.2

Spark takes a declarative structure, and generates a DSL from that structure.

For a very simple example:

defmodule MyApp.Dsl.Extension do
  @dsl %Spark.Dsl.Section{
    name: :dsl,
    schema: [
      foo: [
        type: :integer, 
        doc: "The amount of coolness to add", 
        default: 100
      ]
    ]
  }

  use Spark.Dsl.Extension, sections: [@dsl]
end

defmodule MyApp.Dsl do
    use Spark.Dsl, default_extensions: [extensions: [MyApp.Dsl.Extension]]
end

defmodule MyApp.Dsl.Info do
  use Spark.InfoGenerator, extension: MyApp.Dsl.Extension, sections: [:dsl]

end

Then you can use it like so:

defmodule Something do
  use MyApp.Dsl

  dsl do
    foo 10
  end
end

# and introspect it

MyApp.Dsl.Info.dsl_foo(Something) 
# => {:ok, 10}

That example is just the tip of the iceberg. It looks like “macro magic” but in reality it’s a thin veneer over a data structure that gives you elixir-specific niceties and handles the complexity that comes with building these kinds of things.

So you can write a DSL without having to write a single macro, we write the macros for you Macros writing macros, there are cases where it makes sense

My only concern about Spark is that it lacks documentation, are you planning on writing documentation for that project or it is more of a internal project that powers Ash?