hst337
Elixir compiler and language specification
Purpose of the proposal
Elixir language is in mature state and no breaking or heavy changes are expected to happen. Therefore “now” is the right time to start thinking about creating somewhat specification of the language. Current proposal is supposed to be the place to talk about necessity, usefulness and implementation of the language specification
Language and compiler specification
What is it?
Language specification is a somewhat formal definition and description of how the language runtime must/should/can/shouldn’t behave to be valid Elixir runtime. It may also define properties of standard library (like “Kernel.+ is always a functions”), etc.
Compiler specification is also a somewhat formal definition and description
Why should we need one?
-
In case someone (like me) takes an opportunity to implement compatible compiler or language runtime or both. Anyway, for the language which is mature enough, it is only a question of time when third-party compilers and runtimes will appear, and reference-oriented implementations are almost always doomed to be incompatible in some edge cases. This leads to UBs among different compilers and runtimes, which is bad.
-
Specification prevents exploits and incorrect usage of undocumented or unexpected behavior. I can provide some examples of undocumented things, which can be exploited to hack module compilation order, create weak inter-module dependencies, detect function overrides and so on. https://www.hyrumslaw.com/
-
Specification defines more clearer path of language improvement, so that users could expect that some parts of language may change in the future. For example, marking that protocol functions (like
Enumerable.reduce) may never be called if the compiler can derive the type of the first argument, can create a room for type-inference oriented compile-time optimizations. -
Process specification of development is also a process of exploration of improvement possibilities
Why can’t we just document this stuff?
Documentation is intended for users of the language, not for compiler or runtime developers. Therefore, some topics just do not belong in documentation, while others just do not need deep explanation.
Won’t it take a ton of time?
Specifications may (and they definitely will) evolve over time. It is not necessary to define every aspect of the language beforehand, specification can be ad hoc and reactive, defining just the most important points.
Can’t we just discuss language details here?
Yes, we can, but these discussions can be lost, they can become outdated or they can just be incorrectly answered by mistake.
What do we need to do to create the specification?
-
Define a process of specification development. For example, “contributors create RFC (whose format defined here), core team approves”
-
Define specification format and rules. For example, “MUST defines that every Elixir compiler must behave this way to be considered compatible”
-
Write the specification
Let’s discuss
Trending in Proposals: Ideas
Other Trending Topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #blog-post
- #phoenix_html
- #iex
- #graphql
- #ai
- #genstage
- #elixirconf-us
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #security
- #hex










First 10 of 15 Posts
hst337
Who might be interested in specification
Without taking into account Elixir core team and myself there are several third-party projects who might benefit in having language specification and
sourceror(cc @dorgan) andpatch(cc @mdnowack)) could have a better understanding of how Elixir modules must work and how AST must behave, how variable contexts behave and so on.D4no0
Personally I think that this is not worth the time, you should understand that elixir without erlang is pretty much an empty shell, because some of the more advanced features like pattern matching are part of erlang, not elixir.
hst337
I understand that, but how does this go against the idea of language specification?
LostKobrakai
Shouldn’t it be the other way round? Beam code being the standard (which has defined specs for erts to follow) and elixir just like other beam languages is a consumer of that spec?
D4no0
This most probably will never happen, the only exception is lumen and who knows how many years it is from completion.
The other thing is that there are talks about elixir 2.0, so there might be a lot of breaking changes.
In general I think that having specifications is a good thing, however I don’t see how the effort invested in making a specification for elixir will be paid off.
LostKobrakai
Where did you get that impression? Yes there’s things queued for an eventual 2.0, but afaik there are no plans at all to act on those at this point.
cmo
Elixir 2.0 is coming out soon
dimitarvp
I just don’t see an already overworked core team working on this ever. And the community, while very practical and of high quality, is not big enough or with enough free bandwidth to tackle this. We seem to mostly address functional stuff that helps us do our jobs. Not too much foundational work (exceptions exist of course, but they kind of prove the rule).
I do agree with the rationale btw, I am just not seeing this idea happen.
One thing I’d explore if I had all the time and energy in the world: an alternative Beam VM implementation without hot code reloading. Maybe skipping that will allow for both more performance optimizations and a way to spec the whole thing better. Maybe.
…Or maybe we can collaboratively agree on e.g. AST format for code-generating libraries as a start. The value of that is up in the air though.
hst337
It works in both ways. ERTS developers are interested in optimizing runtime for frequently used patterns, and Elixir developers are interested in generating efficient code. The specification would help to define what kind of features Elixir relies the most and what features are not that necessary.
There is also Tria, Eir project, and a lot of companies are interested in development of statically typed elixir compilers (@josevalim gave several talks about it)
It is neither scheduled, nor expected to happen yet.
It is actually a good point, thanks. I think that there are several approaches to this problem. As I’ve said, it is not required to have 100% complete spec of the language. And I think that it should not be the job of core team to gather specification requirements, specification format, specification implementation and the specification of language itself
I would suggest something like this:
D4no0
Yeah like this talk, and the conclusion of that talk is that having static types that are solved at compile-time don’t solve much and why elixir will never be typed like languages such as java.
This seems to be a big overstatement, but it is no news that people want their classic OOP language, either because they are too lazy to learn a new paradigm or because that is the only way they know how to architect and write software, a good example is this discussion.