Other languages generating code like Elixir does

So the Elixir compiler takes the data structure you mentioned before, and traverses it to generate Erlang code? Is that so?

That’s my understanding, yes. I used to think that elixir targeted a layer below that (core erlang), but it seems to just output erlang.

If you look closely, you will see macro there though :wink:

But ok, lets see:

function Example() {
  let ret = {}
  const kv = {
    foo: "called foo",
    bar: "called bar"
  }

  for (let name in kv) {
    ret[name] = () => kv[name]
  }
  
  return ret
}

const example = Example()

console.log(example.foo())
console.log(example.bar())
class Example
  {foo: "called foo", bar: "called bar"}.each do |key, value|
    define_method(key) { value }
  end
end

puts Example.new().foo
puts Example.new().bar

Unfortunately I do not have time to write more examples right now.

This “data structure” is called abstract syntax tree and it is exactly what quote will spit out. So this is tuple in form {element :: atom(), environment :: keyword(), arguments :: term()} (however I do not know how the root of the tree looks like, so maybe someone with more knowledge can say more.

Thanks for your examples, they are enough to show what Javascript and Ruby can do with respect to code generation.

This seems very much like what Elixir can do.

Do you know if Rust, or any other statically typed language that can do the same, as you only gave examples in dynamic languages?

1 Like

Well Elixir may expand everything to a big data structure, the AST, but then the part of the code corresponding to

    kv = [foo: "called foo", bar: "called bar"]
    Enum.each kv, fn {k, v} ->
      ...
    end

must be executed some how so that it generates foo and bar.

This makes Elixir compiler behave somewhat differently to a typical compiler that picks an AST and visits its node to translate into the target code.

What does Elixir do differently?

elixir evaluates what it sees first unless it’s a macro, and then builds an ast from it.

So there is initially a macro expansion phase, then Elixir executes the whole output of this phase, then it picks the resulting AST, and does the normal visit to translate to Erlang code. Is it like this?

Thanks for making me aware that Elixir is a lisp-2.

I think so, yes. I haven’t read into the compiler though, but a quick skim through the code reveled the following function https://github.com/elixir-lang/elixir/blob/725a36f14d4590972e1efade1978ab3c428585a8/lib/elixir/src/elixir.erl#L288-L291 which seems to be doing exactly what you’ve described. It might not be called on actual .ex files, but only from command line (judging by the name), but I’d guess the logic would be the same.

In languages compiled to the binary executable and without use of “macros” I think it would be only available in D. However if we allow usage of separate languages then we can use C preprocessor to generate functions in C/C++ during compilation. In Rust we can use macros to generate functions or attributes to do so (for example check Serde).

Nice link, but I do not see where Elixir executes the whole output of the macro expansion phase, which corresponds to the result of executing,

    kv = [foo: "called foo", bar: "called bar"]
    Enum.each kv, fn {k, v} ->
      ...
    end

This must be done before translating AST to Erlang code. When?

It probably happens before the function I’ve linked (or something similar) is called, but it would most likely have the same logic.

1 Like

What you’re doing there is using the unquote fragment (docs) functionality, which is technically different to what a macro does. It does not convert AST -> AST, but generates functions based on compile time values. Sadly both macros and unquote fragments use unquote(), which makes is in my opinion quite unintuitive to users that there are two different usecases for unquote, which even clash with each other if you try to use unquote fragments within a macro.

2 Likes

AFAIK Elixir is never translated to Erlang code but directly to the Erlang VM binary code. And example you have given will not work, as you need to use Each.map instead. Just think about macros as functions that return AST. Other than that, everything is interpreted as any other function.

defmodule MyDef do
  defmacro my_def(name, value) do
    {:def, [], [{name, [], []}, [do: value]]}
  end
end

defmodule Main do
  require MyDef
  
  MyDef.my_def(:foo, "bar")
end

IO.puts Main.foo
1 Like

That’s not correct: https://wandbox.org/permlink/IYH8MJnd7dUlAPhm and it probably correlates to my previous post and unquote fragments not working like macros.

1 Like

I never know when use which one, that is why I always use for which is more like Enum.map rather than Enum.each so maybe that is the source of the confusion.

The unquote function does only one thing, which is to evaluate expressions in the context of its surrounding quote. The case of the two different use cases you mention is an illusion caused by the fact that def (and the like) expand in a way that the unquote will evaluate in a context where k and v are bound.