Exploring the limitations of macros and module attributes

Chat / Discussions
,
1

Hi!

I’ve been tinkering with bringing Clojure.spec to Elixir (I’m aware of prior art here, but want to experiment with a different syntax).

Having never dabbled in meta programming before, I’m curious to learn if I hit a wall or if I’m missing something.

Here is some simple pseudo-code

  @arg a: is_integer(), is_integer()
  @arg b: is_integer()
  @ret is_integer()
  def add(a, b) do
    a + b
  end

I discovered that @ is a macro in the Kernel module and appears to be evaluated before I get a chance to do anything (via before_compile, on_definition or my macro). Right now it fails because is_integer() isn’t valid.

Is there any way to transform the AST to replace the module attribute with my own code?

I’ve learned about the possibility of a custom defmodule implementation but that seems excessive to me. I considered using function references, like &is_integer/1 but it seems a bit ugly to me (very subjective ;). It also appears that I can’t override the @ macro easily since I’m getting “call is ambiguous” when I define my own. Are these all available options?

Thank you!
Stephan

1 Like
2

Hey, welcome to the ElixirForum, and Elixir meta-programming!

I think the conceptual nudge that will help you the most here is a little clarification in the role of various concepts during the compilation process:

  • a Module is a unit of compilation + a namespace
  • a module attribute is a constant language-level value determined at compile time
  • a Macro is a special function executed at compile time that receives AST as input and places its return value into the AST where invoked
  • macros can access module attributes, ie in compilation hooks via attribute functions.

Emphasis on value. What you place in a module attribute should be a language-level primitive value, that may be operated on later by compile-time hooks. You’ll notice the input to all documented module attributes, with the exception of type-related stuff, are just tuples, lists, strings, atoms, etc.

Typespec-related attributes get special builtin compiler-powered semantics and expressivity, that I do not think you can tap into in user-land code.

You cannot place arbitrary typespec syntax into an arbitrary module attribute today.

The easiest way to get this is to use macros, which specialize in AST transformation. You should try offering, for instance, defarg and defret macros. They could emit type-spec related module attributes AST compatible with the type system. Macros are more than capable of consuming the typespec syntax and operating on it, doing library magic with it, and re-emitting the syntax into typespec module attributes.

As an alternative, you could invent a language-level data primitive syntax that expresses what you want, like @arg a: [:is_integer, :is_integer], and compile it into a typespec via a compilation hook (and perform your library’s magic in the same pass).

The takeaway here, though, is that you’ll need to leverage macros get what you want, with module attributes as possible input to them and output from them.

One note:

You definitely don’t want to do this–even if it is possible (which would surprise me) overriding module attribute definition would have a high chance of breaking tons of stuff in caller code. @ is a macro in the Kernel module for documentation and compiler bootstrapping purposes only.

3 Likes
3

Overriding the @-macro is possible, and is in fact what Norm does to allow custom macro-expansion in the ‘@contract …’ notation. (c.f. https://github.com/keathley/norm/blob/master/lib/norm/contract.ex#L57)

I do think that this solution is a bit of a hack. For instance, it would not be possible to use multiple libraries that each try to override @ unless they pay special attention to how they do the fall-back.

4 Likes
4

@christhekeele Thank you, I really appreciate that insight - and the warm welcome!

@Qqwy ha, I guess I wasn’t aware of this Clojure.spec-like implementation - that’s pretty close to what I was imagining. The generator-side of this is what I find the most intriguing.

1 Like
5

That’s pretty cool. Still dark magicks, but now I wanna play with it!