Can someone explain `do` syntax in a non hand-holding way?

it’s actually a lisp-ish AST construct.

you can see this most clearly by doing this:

quote do
   defmodule Foo do
   end
end

which gives you this:

{:defmodule, [context: Elixir, import: Kernel],
 [{:__aliases__, [alias: false], [:Foo]}, [do: {:__block__, [], []}]]}

and

quote do
  if foo do
    :bar
  end
end

{:if, [context: Elixir, import: Kernel], [{:foo, [], Elixir}, [do: :bar]]}

that’s why sometimes most constructs let you write , do: with the colon as shorthand, because that’s exactly what was there!

haha maybe I think I should break this down more carefully. A macro in elixir is a function that emits AST + “take the AST generated, right here in the code” (a bare function will drop its value, in that location, not the AST). So macros in the first are a function. Let’s consider the “def” macro.

We’ll look at it three ways:

quote do
  def foo, do: :bar 
end

which yields:

{:def, [context: Elixir, import: Kernel],
 [{:foo, [context: Elixir], Elixir}, [do: :bar]]}

and

quote do
  def foo do
     :bar
  end
end

which yields

{:def, [context: Elixir, import: Kernel],
 [{:foo, [context: Elixir], Elixir}, [do: :bar]]}

and

quote do
  def foo do
     :throwaway_statement
     :bar
  end
end 

which yields

{:def, [context: Elixir, import: Kernel],
 [
   {:foo, [context: Elixir], Elixir},
   [do: {:__block__, [], [:throwaway_statement, :bar]}]
 ]}

Note a) forms 1 and 2 are identical
and b) form 3 groups the two lines of the function into a block.

Why is that necessary? Because according to Kernel.def/2 documentation (Kernel — Elixir v1.20.2), the function half of the def macro must take TWO AST parameters. The first parameter is the name of the function (and also a list of arguments) and the second parameter is the contents of the function. So the only arity-consistent way of representing def is by grouping the AST for the body of the function into a block and labeling it as “do”. So the do “keyword” is just elixir’s cute way of saying "hey everything until the “end” keyword winds up in the block. And if you don’t want a block, you can just pass it as do:, which builds the keyword list manually and passes it into the def function without syntactic sugar.

One more important thing to note is that in elixir, lists, atoms, and two-tuples don’t need escaping, because they are identical to their AST, so that’s how when you do a one-liner the do: part of the code makes it into the macro without being converted into these strange tuple forms.

as for why it’s a keyword list, if you look further down the documentation, you can have other parts to the function code, for example catch or rescue blocks; if you do that, for example this code which is nonsensical:

quote do
  def foo do
     :bar
   catch
     :baz
   end
end

yields:

{:def, [context: Elixir, import: Kernel],
 [{:foo, [context: Elixir], Elixir}, [do: :bar, catch: :baz]]}

so other “segments” of the function can nicely wind up stuffed into an arity-2 statement. Likewise you can arbitrarily mix-and-match catches and rescues, because a keyword-list is what you are supposed to use when you have a maybe-ordered list of arbitrary things that can’t be a map because you can maybe have duplicates.

In a nutshell, do-end blocks is a syntax sugar for a :do keyword list.

if CONDITION do
  EXPR
end

is the same as:

if CONDITION, do: (EXPR)

You can see this by calling quote as shown by @ityonemo. Since keyword lists are just data structures, do-blocks end-up being just data structures too. It doesn’t need to be given only to macros, regular functions can also use and match on it, some helpers in Phoenix.HTML do so, but their usage is indeed more common with macros.

As a syntax sugar, they also allow some specific syntactical constructs, such as left -> right and other keywords (else, rescue, etc), but all of them can be written in the literal keyword syntax format too.

Just tried it. Mind blown. I mean, of course, what was I thinking, why wouldn’t that work??!. I promise not to use this power for evil.

Thank you both for your answers!

As stated, I am well aware of it being just syntactic sugar for a :do in a keyword list, but these answers were still very helpful. I never thought to unquote them! I was also unaware that the :do syntax would work for functions too… not sure why I just didn’t try it…

Thanks especially @ityonemo for your very detailed response… there was some good insight there that is helping me grok this.

I was really mainly looking to understand what they are conceptually as related to concepts in other languages (for example how a ruby block is closure) although I’m starting to see that they really are kinda “just blocks of code” and it all depends on how the macro or function decides to deal with them. I was going to do some tests before responding buuuut I felt like responding and will play around with it later.

Thanks again!

I always think of do like progn is LISP.