Help on `?` (for getting a character's codepoint)

There is not really an implementation. This is a pure syntax feature of elixir.

The ?x syntax is handled by the tokenizer:

https://github.com/elixir-lang/elixir/blob/main/lib/elixir/src/elixir_tokenizer.erl#L233-L274

This code is also what produces the messages you get when using ? with improperly-escaped characters:

iex(1)> ? 
warning: found ? followed by code point 0x20 (space), please use ?\s instead
  iex:1:1

32

iex(2)> ?\x
warning: unknown escape sequence ?\x, use ?x instead
  iex:2:1

120

It is strange that it’s not possible to search for this as an operator. It almost feels like there should be an honorary mention in Kernel.SpecialForms (“the basic building blocks of Elixir”) that at least explains what’s going on.

There’s a bunch of syntax to define numbers, which is not documented in Kernel.SpecialForms:

1234 # decimal base (the default)
0b1010101 # binary base
0o707070 # octal base
0xF0F0F0 # hexadecimal base
?e # unicode code point

None of them have AST representations. They’re turned into their respective integers before turned into AST and are therefore part of the syntax of elixir.

quote do: 0b101010
# 42
quote do: ?e
# 101

This is similar to how [abc: :def] and [{:abc, :def}] are the same in AST (though the sugar’ed form is the default representation):

quote do: [abc: :def]
# [abc: :def]
quote do: [{:abc, :def}]
# [abc: :def]

Kernel.SpecialForms on the other hand are represented in AST, but happen to be implemented directly in the compiler, because there’s no possible macro to transform their AST to some other elixir code doing what they do (or do it efficiently).

quote do: for(i <- 1..10, do: i)
# {:for, [], […]}

I can certainly see that this is confusing though. E.g. tuple and map syntax are special forms, lists are not, as lists stay lists in AST.

Thank you all for your replies, as always, the Elixir forum delivers. This is really interesting

Cool, I’ll put the same link pinned to the latest commit SHA so others can refer to this at a later date if the file changes one day:

https://github.com/elixir-lang/elixir/blob/3b90a793fca04869cf90add0fd7144ece52f9dba/lib/elixir/src/elixir_tokenizer.erl#L233-L274