First impressions of Elixir's core

Per Imagining Elixir 2.0, I have decided to fork Elixir.

Additionally, I am writing gists as I go, documenting my thought process…

This is the first.

TL;DR

  1. eex should be extracted to its own package because a basic Mix project doesn’t use templates
  2. ex_unit should be extracted too, but automatically included in mix.exs

ok, so how Elixir core code would be tested?

Not sure about eex, but if it’s used somewhere (like formatting messages) then it’s also not possible to extract it.

Look that mix is used for managing dependencies. mix of course requires Elixir to work. If there is no Elixir installed then mix can’t compile. Therefore we can’t fetch ex_unit as dependency, because there is no mix installed. Finally Elixir core would depend on ex_unit which as said can’t be fetched before build mix.

Also I think that all apps are using Elixir internal (not documented) API which would force core developers to document more API than they want. Such API can’t be changed before Elixir’s version 2.0.0.

Of course I could misunderstand something and it’s possible to extract those apps. Let me know if I’m wrong.

1 Like

:ex_unit is built on top of Elixir, not coupled to it.

So to answer your question, by including it in mix.exs like any normal project… because – you misread this:mix would obviously remain in the core.

Yeah, I got that mix stays in core. Anyway fetching ex_unit as library (which needs to happen before build) is not possible, because:

  1. ex_unit would depend on core which is not compiled
  2. since core is not compiled mix is also not compiled yet

Look that first build (when you don’t have any mix file) would always fail, because dependencies must be fetched before compiling. Look how it’s working with normal libraries - you would get error which explains that you need to call mix deps.get, but you can’t do it, because you don’t have mix yet.

Sorry, don’t get why you give us those documentation links. Of course they are separate from core documentation, because they are from different apps. I mean that ex_unit code could call Elixir internal API.

eex is used by mix, and as mix is core you can’t remove eex.

1 Like

What for?

mix new at least.

Yeah, I looked at the subdirectory–mix is not only how we start a project, but how we compile it.

As a result, it is necessary to include :eex, not for :elixir but rather :mix

The majority of the code in template files is often markup, but there will also be sections of Elixir code for Phoenix to compile and evaluate.

This raises an interesting question about architecture.

:elixir is the ‘kernel’. Everything depends on it.

:mix is many things, but especially a build tool.

:eex is a lexer which converts Elixir code into HTML, and can also work with HTML within its do block.

Because :mix is an essential tool and really convenient, :eex must be included. But in practice, sometimes a project has no :eex code to compile.

I now understand why :eex is included, but there should be a way to opt out because many projects never use it.

when you build a release any applications which are not used are not included. But even if they were so what? It would just mean that there are a bunch of .beam files on disk that are probably not loaded.

I am trying to see what your objection is exactly?

2 Likes

Sure.

I think my larger question is: As :elixir uses :logger and :mix uses :eex, why are they in separate, umbrella-esque subdirectories in the core?

I.e. a bunch of OTP stuff is still in the elixir subdirectory, when it could easily be split into its own subdirectory.

They are separate OTP applications (I think) so at runtime they can be started, stopped and upgraded separately. In addition if your application does not use them a release can leave them out.

Even if this is slightly sub optimal it seems to be a pretty trivial issue

i.e. a code smell :wink:

First misconception… eex is a general purpose templating enging that directly embeds elixir code into the template, rather than using a DSL.

Because they are different (OTP) applications. This way you can compile and run your project without having to include mix and eex, just because elixir uses them…

6 Likes

Put another way:

The thing is, in Erlang, there are a bunch of module-sized packages. In this regard, I think the Elixir core takes steps in the right direction: consolidating modules with a similar purpose together.

What I don’t get is why that distinction isn’t fully done. If Elixir uses Logger, why wasn’t Logger consolidated into Elixir? Or why wasn’t Inspect consolidated into Logger? What’s the organizing principle?

Similarly, with OTP, that’s conceptually distinct from basic data structures and Enum. I get that it is included in Elixir to avoid an extra namespace. I think it would gain visibility, however, were it made more distinct in the docs than it is. I.e. I’d like to see some docs that cover the concept of OTP in the Elixir ecosystem, which to my knowledge does not exist at this high-level (think the intro page to Phoenix, which gives a high-level overview of its many parts, versus just seeing all the modules and having to piece that together… except those modules are mixed with Ecto’s…).

Heh, so notwithstanding the design weirdness, it sounds like I have nothing to worry about! :smile:

Applications in otp are not about code organisation, but about runtime abilities. The Logger application can be started/stopped as it’s own unit. You wouldn’t want to stop the elixir application just so you can stop the Logger. This is the same in erlang. Many modules of erlang are not usable if you don’t have the necessary application running. E.g. :http_uri does not work unless the :inets application is started. It’s just that imho the docs of erlang are not really explicit about that fact. If you look at the erlang source you’ll find a similar structure than elixir is using for all it’s applications.

I can think of a few reasons, mostly logger may have its own lifecycle in OTP. You may want to manage it independently of the Elixir core. It may also be a case of that some of these applications have supervision trees and others do not.

1 Like

To add onto that. You need to differenciate between elixir the programming language and elixir the otp application. Elixir the programming language does include the otp applications eex, elixir, ex_unit, iex, logger and mix. Like if you start iex and type :observer.start you’ll see iex, elixir and logger in the applications tab. The elixir otp app even starts a superivision tree at runtime to do some of what it needs to do to be able to use elixir on the beam. What part of Elixir belongs to which otp app depends on if things need to be able to have their own lifecycle as independant application. Logger might be not required for people, so it has it’s own otp application, ex_unit is usually only needed in tests, so it does as well. Same reasoning applies for eex, mix and iex. In erlang the stdlib is split up in even more otp applications, but that’s because a lot of them are stateful applications. Usually you only notice that if you try to use a module and the application is not started (like my prev. :inets example). The elixir stdlib can happily live inside the elixir otp application as those modules are not stateful by themselves, but it seems to be preferred to let users integrate processes into their own supervision trees (e.g. Task.Supervisor). If this wasn’t to be the case there would probably be more elixir otp applications as well.

This also aligns with elixirs goal to be extensible instead of relying on core development. In the core are only otp applications, which are essential for the Elixir language. So either things essential like having a compiler, stdlib, build system or test suite; or things, which are needed to run those things. If there would be an otp application in the core, which would fulfill neither of those things, then it should be extracted.

2 Likes