Elixir compiler and language specification

Purpose of the proposal

Elixir language is in mature state and no breaking or heavy changes are expected to happen. Therefore “now” is the right time to start thinking about creating somewhat specification of the language. Current proposal is supposed to be the place to talk about necessity, usefulness and implementation of the language specification

Language and compiler specification

What is it?

Language specification is a somewhat formal definition and description of how the language runtime must/should/can/shouldn’t behave to be valid Elixir runtime. It may also define properties of standard library (like “Kernel.+ is always a functions”), etc.
Compiler specification is also a somewhat formal definition and description

Why should we need one?

1. In case someone (like me) takes an opportunity to implement compatible compiler or language runtime or both. Anyway, for the language which is mature enough, it is only a question of time when third-party compilers and runtimes will appear, and reference-oriented implementations are almost always doomed to be incompatible in some edge cases. This leads to UBs among different compilers and runtimes, which is bad.

2. Specification prevents exploits and incorrect usage of undocumented or unexpected behavior. I can provide some examples of undocumented things, which can be exploited to hack module compilation order, create weak inter-module dependencies, detect function overrides and so on. https://www.hyrumslaw.com/

3. Specification defines more clearer path of language improvement, so that users could expect that some parts of language may change in the future. For example, marking that protocol functions (like Enumerable.reduce) may never be called if the compiler can derive the type of the first argument, can create a room for type-inference oriented compile-time optimizations.

4. Process specification of development is also a process of exploration of improvement possibilities

Why can’t we just document this stuff?

Documentation is intended for users of the language, not for compiler or runtime developers. Therefore, some topics just do not belong in documentation, while others just do not need deep explanation.

Won’t it take a ton of time?

Specifications may (and they definitely will) evolve over time. It is not necessary to define every aspect of the language beforehand, specification can be ad hoc and reactive, defining just the most important points.

Can’t we just discuss language details here?

Yes, we can, but these discussions can be lost, they can become outdated or they can just be incorrectly answered by mistake.

What do we need to do to create the specification?

1. Define a process of specification development. For example, “contributors create RFC (whose format defined here), core team approves”

2. Define specification format and rules. For example, “MUST defines that every Elixir compiler must behave this way to be considered compatible”

3. Write the specification

Let’s discuss