Elixir v1.15.0 released

Official announcement: Elixir v1.15 released - The Elixir programming language

This release requires Erlang/OTP 24 and later.

Elixir v1.15 is a smaller release with focused improvements
on compilation and boot times. This release also completes
our integration process with Erlang/OTP logger, bringing new
features such as log rotation and compaction out of the box.

You will also find additional convenience functions in Code,
Map, Keyword, all Calendar modules, and others.

Compile and boot-time improvements

The last several releases brought improvements to compilation
time and this version is no different. In particular, Elixir
now caches and prunes load paths before compilation, ensuring your
project (and dependencies!) compile faster and in an environment
closer to production.

In a nutshell the Erlang VM loads modules from code paths. Each
application that ships with Erlang and Elixir plus each dependency
become an entry in your code path. The larger the code path, the
more work Erlang has to do in order to find a module.

In previous versions, Mix would only add entries to the load paths.
Therefore, if you compiled 20 dependencies and you went to compile
the 21st, the code path would have 21 entries (plus all Erlang and
Elixir apps). This allowed modules from unrelated dependencies to
be seen and made compilation slower the more dependencies you had.
With this release, we will now prune the code paths to only the ones
listed as dependencies, bringing the behaviour closer to mix release.

Furthermore, Erlang/OTP 26 allows us to start applications
concurrently and cache the code path lookups, decreasing the cost of
booting applications. The combination of Elixir v1.15 and Erlang/OTP 26
should reduce the boot time of applications, such as when starting
iex -S mix or running a single test with mix test, from 5% to 30%.

The compiler is also smarter in several ways: @behaviour declarations
no longer add compile-time dependencies and aliases in patterns and
guards add no dependency whatsoever, as no dispatching happens. Furthermore,
Mix now tracks the digests of @external_resource files, reducing the
amount of recompilation when swapping branches. Finally, dependencies
are automatically recompiled when their compile-time configuration changes.

Potential incompatibilities

Due to the code path pruning, if you have an application or dependency
that does not specify its dependencies on Erlang and Elixir application,
it may no longer compile successfully in Elixir v1.15. You can temporarily
disable code path pruning by setting prune_code_paths: false in your
mix.exs, although doing so may lead to runtime bugs that are only
manifested inside a mix release.

Compiler warnings and errors

The Elixir compiler can now emit many errors for a single file, making
sure more feedback is reported to developers before compilation is aborted.

In Elixir v1.14, an undefined function would be reported as:

** (CompileError) undefined function foo/0 (there is no such import)
    my_file.exs:1

In Elixir v1.15, the new reports will look like:

error: undefined function foo/0 (there is no such import)
  my_file.exs:1

** (CompileError) my_file.exs: cannot compile file (errors have been logged)

A new function, called Code.with_diagnostics/2, has been added so this
information can be leveraged by editors, allowing them to point to several
errors at once.

Potential incompatibilities

As part of this effort, the behaviour where undefined variables were
transformed into nullary function calls, often leading to confusing error
reports, has been disabled during project compilation. You can invoke
Code.compiler_options(on_undefined_variable: :warn)
at the top of your mix.exs to bring the old behaviour back.

Integration with Erlang/OTP logger

This release provides additional features such as global logger
metadata and file logging (with rotation and compaction) out-of-the-box!

This release also soft-deprecates Elixir’s Logger Backends in
favor of Erlang’s Logger handlers. Elixir will automatically
convert your :console backend configuration into the new
configuration. Previously, you would set:

config :logger, :console,
  level: :error,
  format: "$time $message $metadata"

Which is now translated to the equivalent:

config :logger, :default_handler,
  level: :error

config :logger, :default_formatter,
  format: "$time $message $metadata"

If you use Logger.Backends.Console with a custom device or other
backends, they are still fully supported and functional. If you
implement your own backends, you want to consider migrating to
:logger_backends
in the long term.

See the new Logger documentation for more information on the
new features and on compatibility.

50 Likes

one issue I see is that we have :hackney dependency which in turns relies on :ssl_verify_fun and it now doesn’t compile for me, which is weird because it’s an Erlang lib.

Generated poison app
==> ssl_verify_fun
Compiling 7 files (.erl)
src/ssl_verify_pk.erl:14:14: can't find include lib "public_key/include/public_key.hrl"
%   14| -include_lib("public_key/include/public_key.hrl").
%     |              ^

src/ssl_verify_fun_cert_helpers.erl:13:14: can't find include lib "public_key/include/public_key.hrl"
%   13| -include_lib("public_key/include/public_key.hrl").
%     |              ^

src/ssl_verify_hostname.erl:16:14: can't find include lib "public_key/include/public_key.hrl"
%   16| -include_lib("public_key/include/public_key.hrl").
%     |              ^

src/ssl_verify_fingerprint.erl:15:14: can't find include lib "public_key/include/public_key.hrl"
%   15| -include_lib("public_key/include/public_key.hrl").
%     |              ^

src/ssl_verify_fun_cert_helpers.erl:23:34: undefined macro 'id-ce-subjectAltName'
%   23|   AltSubject = select_extension(?'id-ce-subjectAltName', Extensions),
%     |                                  ^

src/ssl_verify_pk.erl:26:26: record 'OTPCertificate' undefined
%   26| -spec verify_fun(Cert :: #'OTPCertificate'{},
%     |                          ^

src/ssl_verify_hostname.erl:28:26: record 'OTPCertificate' undefined
%   28| -spec verify_fun(Cert :: #'OTPCertificate'{},
%     |                          ^

src/ssl_verify_fingerprint.erl:27:26: record 'OTPCertificate' undefined
%   27| -spec verify_fun(Cert :: #'OTPCertificate'{},
%     |                          ^

src/ssl_verify_pk.erl:28:39: record 'Extension' undefined
%   28|                           {extension, #'Extension'{}}, InitialUserState :: term()) ->
%     |                                       ^

src/ssl_verify_fun_cert_helpers.erl:9:2: function extract_dns_names/1 undefined
%    9| -export([extract_dns_names/1,
%     |  ^

src/ssl_verify_hostname.erl:30:39: record 'Extension' undefined
%   30|                           {extension, #'Extension'{}}, InitialUserState :: term()) ->
%     |                                       ^

src/ssl_verify_fingerprint.erl:29:39: record 'Extension' undefined
%   29|                           {extension, #'Extension'{}}, InitialUserState :: term()) ->
%     |                                       ^

src/ssl_verify_pk.erl:51:30: record 'OTPCertificate' undefined
%   51| -spec verify_cert_pk(Cert :: #'OTPCertificate'{}, Pk :: pk()) ->
%     |                              ^

src/ssl_verify_fun_cert_helpers.erl:19:2: spec for undefined function extract_dns_names/1
%   19| -spec extract_dns_names(Cert :: #'OTPCertificate'{}) -> [] | [string()].
%     |  ^

src/ssl_verify_hostname.erl:46:36: record 'OTPCertificate' undefined
%   46| -spec verify_cert_hostname(Cert :: #'OTPCertificate'{}, Hostname :: hostname()) ->
%     |                                    ^

src/ssl_verify_fingerprint.erl:52:39: record 'OTPCertificate' undefined
%   52| -spec verify_cert_fingerprint(Cert :: #'OTPCertificate'{}, Fingerprint :: fingerprint()) ->
%     |                                       ^

src/ssl_verify_fun_cert_helpers.erl:19:33: record 'OTPCertificate' undefined
%   19| -spec extract_dns_names(Cert :: #'OTPCertificate'{}) -> [] | [string()].
%     |                                 ^

src/ssl_verify_fun_cert_helpers.erl:32:26: record 'OTPCertificate' undefined
%   32| -spec extract_cn(Cert :: #'OTPCertificate'{}) -> {error, no_common_name} | {ok, string()} | {error, invalid}.
%     |                          ^

src/ssl_verify_hostname.erl:76:38: record 'OTPCertificate' undefined
%   76|                              Cert :: #'OTPCertificate'{},
%     |                                      ^

src/ssl_verify_fun_cert_helpers.erl:34:17: record 'OTPCertificate' undefined
%   34|   TBSCert = Cert#'OTPCertificate'.tbsCertificate,
%     |                 ^

src/ssl_verify_fun_cert_helpers.erl:35:32: record 'OTPTBSCertificate' undefined
%   35|   {rdnSequence, List} = TBSCert#'OTPTBSCertificate'.subject,
%     |                                ^

src/ssl_verify_fun_cert_helpers.erl:38:26: record 'OTPCertificate' undefined
%   38| -spec extract_pk(Cert :: #'OTPCertificate'{}) -> {error, no_common_name} | #'SubjectPublicKeyInfo'{}.
%     |                          ^

src/ssl_verify_fun_cert_helpers.erl:38:76: record 'SubjectPublicKeyInfo' undefined
%   38| -spec extract_pk(Cert :: #'OTPCertificate'{}) -> {error, no_common_name} | #'SubjectPublicKeyInfo'{}.
%     |                                                                            ^

src/ssl_verify_fun_cert_helpers.erl:40:17: record 'OTPCertificate' undefined
%   40|   TBSCert = Cert#'OTPCertificate'.tbsCertificate,
%     |                 ^

src/ssl_verify_fun_cert_helpers.erl:41:26: record 'OTPTBSCertificate' undefined
%   41|   PublicKeyInfo = TBSCert#'OTPTBSCertificate'.subjectPublicKeyInfo,
%     |                          ^

src/ssl_verify_fun_cert_helpers.erl:42:16: record 'OTPSubjectPublicKeyInfo' undefined
%   42|   PublicKeyInfo#'OTPSubjectPublicKeyInfo'.subjectPublicKey.
%     |                ^

src/ssl_verify_fun_cert_helpers.erl:48:24: record 'Extension' undefined
%   48| -spec extensions_list([#'Extension'{}] | asn1_NOVALUE) -> [] | [#'Extension'{}].
%     |                        ^

src/ssl_verify_fun_cert_helpers.erl:48:65: record 'Extension' undefined
%   48| -spec extensions_list([#'Extension'{}] | asn1_NOVALUE) -> [] | [#'Extension'{}].
%     |                                                                 ^

src/ssl_verify_fun_cert_helpers.erl:55:39: record 'Extension' undefined
%   55| -spec select_extension(Id :: term(), [#'Extension'{}]) -> undefined | #'Extension'{}.
%     |                                       ^

src/ssl_verify_fun_cert_helpers.erl:55:71: record 'Extension' undefined
%   55| -spec select_extension(Id :: term(), [#'Extension'{}]) -> undefined | #'Extension'{}.
%     |                                                                       ^

src/ssl_verify_fun_cert_helpers.erl:57:28: record 'Extension' undefined
%   57|   Matching = [Extension || #'Extension'{extnID = ExtId} = Extension <- Extensions, ExtId =:= Id],
%     |                            ^

src/ssl_verify_fun_cert_helpers.erl:57:84: variable 'ExtId' is unbound
%   57|   Matching = [Extension || #'Extension'{extnID = ExtId} = Extension <- Extensions, ExtId =:= Id],
%     |                                                                                    ^

src/ssl_verify_fun_cert_helpers.erl:75:15: record 'AttributeTypeAndValue' undefined
%   75| extract_cn2([[#'AttributeTypeAndValue'{type={2, 5, 4, 3},
%     |               ^

src/ssl_verify_fun_cert_helpers.erl:77:39: variable 'CN' is unbound
%   77|   ssl_verify_fun_encodings:get_string(CN);
%     |                                       ^

src/ssl_verify_fun_cert_helpers.erl:49:1: Warning: function extensions_list/1 is unused
%   49| extensions_list(E) ->
%     | ^

src/ssl_verify_fun_cert_helpers.erl:56:1: Warning: function select_extension/2 is unused
%   56| select_extension(Id, Extensions) ->
%     | ^

src/ssl_verify_fun_cert_helpers.erl:64:1: Warning: function extract_dns_names_from_alt_names/2 is unused
%   64| extract_dns_names_from_alt_names([ExtValue | Rest], Acc) ->
%     | ^

could not compile dependency :ssl_verify_fun, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile ssl_verify_fun --force", update it with "mix deps.update ssl_verify_fun" or clean it with "mix deps.clean ssl_verify_fun"
1 Like

Please see: Include application specification for mix.exs by josevalim · Pull Request #27 · deadtrickster/ssl_verify_fun.erl · GitHub

Meanwhile users can work around this by specifying {:ssl_verify_fun, ..., manager: :rebar3} in their def deps do.

7 Likes

That didn’t work for me the other day. I ended up pointing it to your fork.

I have added this line:

{:ssl_verify_fun, "~> 1.1.6", manager: :rebar3, only: [:test], runtime: false, override: true}

then I removed deps/ and _build directories, ran mix deps.get and it compiles now. I suspect one of the extra steps I did was required too, i.e. maybe your system figured out it already has version 1.1.6 and didn’t actually update the code. Try removing deps/ and _build dirs to see if this helps.

@josevalim fixed for now, thank you!

Ref. the compilation performance, indeed there is an improvement, which is great:

Elixir 1.14.3: mix compile 658.72s user 71.35s system 417% cpu 2:54.69 total
Elixir 1.15.0: mix compile 582.92s user 56.97s system 445% cpu 2:23.57 total

This is on a real-life and fairly big project.

3 Likes

Nice @hubertlepicki! Which Erlang/OTP versions for the before and after? And is that only the current project itself or also for dependencies?

This is on Erlang 24.3.3 (I know, it’s also due to an update) and the compilation is a project + dependencies. That is on my laptop (Ryzen 5 PRO 4650U). The only changes I made were the Elixir version upgrade and the above ssl_verify_fun. Good job!

I had the same issue and this works for me. :grinning:

1 Like

Nice! You should get considerable more speedups on Erlang/OTP 26 (and also on Erlang/OTP 25 if you are on Apple Silicon).

3 Likes

Let’s see then, installing 26.0.1

Okay so after Erlang 26.0.1 update:

mix compile  828.54s user 66.14s system 529% cpu 2:48.84 total

And I ran it several times, it’s consistently around (+/- 10s) of the compilation time I saw on Elixir 1.14.3 and OTP 24. So, it’s a regression in performance for me. Trying 1.15.0-otp-26 now.

mix compile  829.23s user 66.23s system 529% cpu 2:49.11 total

So yeah, nope, Erlang 26.0.1 does not improve compilation speed over 24 for me.

Thank you for trying! So it seems the Elixir upgrade gave you a 22% gain but upgrading Erlang/OTP 26 takes that back. But I also suspect that, because you are compiling the whole project, the gains of running Erlang/OTP 26 are muted because all modules are already loaded.

If you are interested, you could also try mix run -e 1 and mix compile --force, both after the project is fully compiled, since the gains from Erlang/OTP 26 comes from module loading. But I understand if you have already compiled the project enough times today. :slight_smile:

I haven’t tried Erlang 26.0.1 but I did get this for the Elixir update:

For Elixir 1.14.3-otp-25:

  • User time: 115.08 seconds
  • System time: 46.20 seconds
  • CPU percentage: 257%
  • Total time: 1 minute and 2.63 seconds

For Elixir 1.15.0-otp-25:

  • User time: 113.54 seconds
  • System time: 46.45 seconds
  • CPU percentage: 367%
  • Total time: 43.487 seconds

So, 1.44 times faster. It took 70% of the time to compile on 1.15.0 vs 1.14.3.
This was with mix compile --force

What is a fix release (see 1.15.0-fix*)?

$ asdf list-all elixir
# …
1.15.0
1.15.0-fix
1.15.0-fix-otp-24
1.15.0-fix-otp-25
1.15.0-fix-otp-26
1.15.0-otp-24
1.15.0-otp-25
1.15.0-otp-26
1.15.0-rc.0
1.15.0-rc.0-otp-24
1.15.0-rc.0-otp-25
1.15.0-rc.0-otp-26
1.15.0-rc.1
1.15.0-rc.1-otp-24
1.15.0-rc.1-otp-25
1.15.0-rc.1-otp-26
1.15.0-rc.2
1.15.0-rc.2-otp-24
1.15.0-rc.2-otp-25
1.15.0-rc.2-otp-26
# …

Edit: Following bob and builds.txt I found the commit for this fix release. It’s just a one-line change and it’s parent is directly the 1.15.0 release. Since only *.bat file is changed it looks like the fix release is useful for Windows users only and all the change is doing is to optional support for space characters in some paths (smells like C:\Program Files). :smiling_imp:

1 Like

We are running into an issue with this workaround in an umbrella app – here’s a minimal repro: GitHub - rubysolo/deps-repro: Reproduce issue with ssl_verify_fun override in umbrella app

(This error happens under 1.14 or 1.15, btw) Am I specifying the override correctly?

You have to specify it under app2 as well (because app2 brings hackney which brings it).

2 Likes

Ah, got it – thanks! (seems obvious in hindsight)

FWIW, going from 1.14.4-otp-25 to 1.15.0-otp-25 takes our compilation from ~163 sec to ~51 sec! (M1 Max / 64GB)

3 Likes

Adding OTP 26

For Elixir 1.14.3-otp-25:
Total time: 62.63 seconds

For Elixir 1.15.0-otp-25:
Total time: 43.487 seconds
This represents a ~30.6% decrease in time compared to Elixir 1.14.3-otp-25.

For Elixir 1.15.0-otp-26:
Total time: 26.758 seconds
This represents a ~57.3% decrease in time compared to Elixir 1.14.3-otp-25.

CPU utliization for for OTP 26:
CPU percentage: 581% (367% for OTP 25, 257% for Elixir 1.14.3)
(Apple M1 Pro + 16Gb memory)

11 Likes

Also ran into the ssl_verify_func problem but was able to work through it thanks to these posts.

However, I am now seeing an issue that I wasn’t seeing before with my RabbitMQ connections via Lapin and BroadwayRabbitMQ. Haven’t been able to find the root cause yet but this is what I am seeing.

  1. Connections closing with an error.
module=BroadwayRabbitMQ.Producer [error] Cannot connect to RabbitMQ broker: {:auth_failure, ~c"Disconnected"}
  1. Logs in my RabbitMQ container
[error] <0.818.0> {handshake_error,starting,26721,
[error] <0.818.0>                  {amqp_error,frame_error,
[error] <0.818.0>                              "type 99, all octets = <<>>: {frame_too_large,1852269919,4088}",
[error] <0.818.0>                              none}}
  1. Reverting back to old versions of OTP/Elixir works.

Old

elixir 1.14.3-otp-25
erlang 25.3

New

elixir 1.15.0-otp-26
erlang 26.0.1

The only change to my dependencies I made was the ssl_verify_fun but it came out to be the same version. Nothing else in my environment changed between so it is something with elixir/erlang versions. Also worth noting that the RabbitMQ cluster itself is unaffected by these changes as it is running in docker. I also believe all of my deps for RMQ/Broadway/Lapin are up to date.

Once I solve this I plan on doing a follow-up with compile time comparisons. Really excited about those. Hopefully it shows decent gains on compiling dependencies since that’s where a lot of my time is spent currently. We have an awesome ecosystem of great tools and it’s so easy to bring them in but the compile times on them are rough.

@mackeyja92 can you please try upgrading only Elixir first and see if the issue persists or not. My suspicion is that this is an Erlang/OTP issue, so you will have to isolate and report (or maybe it was already reported on amqp).