Elixir v1.9.0-rc.0 released

Elixir News & Announcements Elixir News
1

Release: https://github.com/elixir-lang/elixir/releases/tag/v1.9.0-rc.0

Releases

The main feature in Elixir v1.9 is the addition of releases. A release is a self-contained directory that consists of your application code, all of its dependencies, plus the whole Erlang Virtual Machine (VM) and runtime. Once a release is assembled, it can be packaged and deployed to a target as long as the target runs on the same operating system (OS) distribution and version as the machine running the mix release command.

You can start a new project and assemble a release for it in three easy steps:

$ mix new my_app
$ cd my_app
$ MIX_ENV=prod mix release

A release will be assembled in _build/prod/rel/my_app. Inside the release, there will be a bin/my_app file which is the entry point to your system. It supports multiple commands, such as:

  • bin/my_app start, bin/my_app start_iex, bin/my_app restart, and bin/my_app stop - for general management of the release

  • bin/my_app rpc COMMAND and bin/my_app remote - for running commands on the running system or to connect to the running system

  • bin/my_app eval COMMAND - to start a fresh system that runs a single command and then shuts down

  • bin/my_app daemon and bin/my_app daemon_iex - to start the system as a daemon on Unix-like systems

  • bin/my_app install - to install the system as a service on Windows machines

Why releases?

Releases allow developers to precompile and package all of their code and the runtime into a single unit. The benefits of releases are:

  • Code preloading. The VM has two mechanisms for loading code: interactive and embedded. By default, it runs in the interactive mode which dynamically loads modules when they are used for the first time. The first time your application calls Enum.map/2, the VM will find the Enum module and load it. There’s a downside. When you start a new server in production, it may need to load many other modules, causing the first requests to have an unusual spike in response time. Releases run in embedded mode, which loads all available modules upfront, guaranteeing your system is ready to handle requests after booting.

  • Configuration and customization. Releases give developers fine grained control over system configuration and the VM flags used to start the system.

  • Self-contained. A release does not require the source code to be included in your production artifacts. All of the code is precompiled and packaged. Releases do not even require Erlang or Elixir in your servers, as they include the Erlang VM and its runtime by default. Furthermore, both Erlang and Elixir standard libraries are stripped to bring only the parts you are actually using.

  • Multiple releases. You can assemble different releases with different configuration per application or even with different applications altogether.

Hooks and Configuration

Releases also provide built-in hooks for configuring almost every need of the production system:

  • config/config.exs (and config/prod.exs) - provides build-time application configuration, which is executed when the release is assembled

  • config/releases.exs - provides runtime application configuration. It is executed every time the release boots and is further extensible via config providers

  • rel/vm.args.eex - a template file that is copied into every release and provides static configuration of the Erlang Virtual Machine and other runtime flags

  • rel/env.sh.eex and rel/env.bat.eex - template files that are copied into every release and executed on every command to set up environment variables, including ones specific to the VM, and the general environment

We have written extensive documentation on releases, so we recommend checking it out for more information.

Configuration overhaul

A new Config module has been added to Elixir. The previous configuration API, Mix.Config, was part of the Mix build tool. But since releases provide runtime configuration and Mix is not included in releases, we ported the Mix.Config API to Elixir. In other words, use Mix.Config has been soft-deprecated in favor of import Config.

Another important change related to configuration is that mix new will no longer generate a config/config.exs file. Relying on configuration is undesired for most libraries and the generated config files pushed library authors in the wrong direction. Furthermore, mix new --umbrella will no longer generate a configuration for each child app, instead all configuration should be declared in the umbrella root. That’s how it has always behaved, we are now making it explicit.

Other enhancements

There are many other enhancements. The Elixir CLI got a handful of new options in order to best support releases. Logger now computes its sync/async/discard thresholds in a decentralized fashion, reducing contention. EEx templates support more complex expressions than before. Finally, there is a new ~U sigil for working with UTC DateTimes as well as new functions in the File, Registry, and System modules.

v1.9.0-rc.0 (2019-06-04)

1. Enhancements

EEx

  • [EEx] Allow more complex mixed expressions when tokenizing

Elixir

  • [Access] Allow Access.at/1 to handle negative index
  • [CLI] Add support for --boot, --boot-var, --erl-config, --pipe-to, --rpc-eval, and --vm-args options
  • [Code] Add static_atom_encoder option to Code.string_to_quoted/2
  • [Code] Support :force_do_end_blocks on Code.format_string!/2 and Code.format_file!/2
  • [Code] Do not raise on deadlocks on Code.ensure_compiled/1
  • [Config] Add Config, Config.Reader, and Config.Provider modules for working with configuration
  • [File] Add File.rename!/2
  • [Inspect] Add :inspect_fun and :custom_options to Inspect.Opts
  • [Kernel] Add ~U sigil for UTC date times
  • [Kernel] Optimize &super/arity and &super(&1)
  • [Kernel] Optimize generated code for with with a catch-all clause
  • [Kernel] Validate __struct__ key in map returned by __struct__/0,1
  • [Module] Add Module.get_attribute/3
  • [Protocol] Improve Protocol.UndefinedError messages to also include the type that was attempted to dispatch on
  • [Protocol] Optimize performance of dynamic dispatching for non-consolidated protocols
  • [Record] Include field names in generated type for records
  • [Regex] Automatically recompile regexes
  • [Registry] Add Registry.select/2
  • [System] Add System.restart/0, System.pid/0 and System.no_halt/1
  • [System] Add System.get_env/2, System.fetch_env/1, and System.fetch_env!/1
  • [System] Support SOURCE_DATE_EPOCH for reproducible builds

ExUnit

  • [ExUnit] Allow multiple :exclude on configuration/CLI
  • [ExUnit.DocTest] No longer wrap doctest errors in custom exceptions. They ended-up hiding more information than showing
  • [ExUnit.DocTest] Display the actual doctest code when doctest fails

IEx

  • [IEx.CLI] Copy ticktime from remote node on IEx --remsh
  • [IEx.CLI] Automatically add a host on node given to --remsh

Logger

  • [Logger] Use a decentralized mode computation for Logger which allows overloads to be detected more quickly
  • [Logger] Use persistent_term to store configuration whenever available for performance

Mix

  • [Mix] Follow XDG base dir specification in Mix for temporary and configuration files
  • [Mix.Generator] Add copy_file/3, copy_template/4, and overwite?/2
  • [mix archive.uninstall] Allow mix archive.uninstall APP to uninstall any installed version of APP
  • [mix new] No longer generate a config/ directory for mix new
  • [mix release] Add support for releases
  • [mix release.init] Add templates for release configuration
  • [mix test] Allow running tests for a given umbrella app from the umbrella root with mix test apps/APP/test. Test failures also include the apps/APP prefix in the test location

2. Bug fixes

EEx

  • [EEx] Consistently trim newlines when you have a single EEx expression per line on multiple lines

Elixir

  • [Code] Quote :: in Code.format_string!/1 to avoid ambiguity
  • [Enum] Ensure the first equal entry is returned by Enum.min/2 and Enum.max/2
  • [Kernel] Improve error message when string interpolation is used in a guard
  • [Kernel] Properly merge and handle docs for callbacks with multiple clauses
  • [Kernel] Guarantee reproducible builds on modules with dozens of specs
  • [Kernel] Resolve __MODULE__ accordingly in nested defmodule to avoid double nesting
  • [Kernel] Type variables starting with an underscore (_foo) should not raise compile error
  • [Kernel] Keep order of elements when macro in/2 is expanded with a literal list on the right-hand side
  • [Kernel] Print proper location on undefined function error from dynamically generated functions
  • [System] Make sure :init.get_status/0 is set to {:started, :started} once the system starts
  • [Path] Do not expand ~ in Path.expand/2 when not followed by a path separator
  • [Protocol] Ensure debug_info is kept in protocols
  • [Regex] Ensure inspect returns valid ~r// expressions when they are manually compiled with backslashes
  • [Registry] Fix ETS leak in Registry.register/2 for already registered calls in unique registries while the process is still alive

ExUnit

  • [ExUnit] Raise error if attempting to run single line tests on multiple files
  • [ExUnit] Return proper error on duplicate child IDs on start_supervised

IEx

  • [IEx] Automatically shut down IEx if we receive EOF

Logger

  • [Logger] Don’t discard Logger messages from other nodes as to leave a trail on both systems

Mix

  • [mix compile] Ensure Erlang-based Mix compilers (erlang, leex, yecc) set valid position on diagnostics
  • [mix compile] Ensure compilation halts in an umbrella project if one of the siblings fail to compile
  • [mix deps] Raise an error if the umbrella app’s dir name and mix.exs app name don’t match
  • [mix test] Do not consider modules that are no longer cover compiled when computing coverage report, which could lead to flawed reports

3. Soft-deprecations (no warnings emitted)

Mix

  • [Mix.Config] Mix.Config has been deprecated in favor of the Config module that now ships as part of Elixir itself. Reading configuration files should now be done by the Config.Reader module

4. Hard-deprecations

Elixir

  • [CLI] Deprecate --detached option, use --erl "-detached" instead
  • [Map] Deprecate Enumerable keys in Map.drop/2, Map.split/2, and Map.take/2
  • [String] The :insert_replaced option in String.replace/4 has been deprecated. Instead you may pass a function as a replacement or use :binary.replace/4 if you need to support earlier Elixir versions

Mix

  • [Mix.Project] Deprecate Mix.Project.load_paths/1 in favor of Mix.Project.compile_path/1

Checksums

  • Precompiled.zip SHA1: 775e9c39c0bf680ecbab0dcbb93589b3fadea773
  • Precompiled.zip SHA512: bc50c7f766196d961e26c912ead53257b95c421cfab81c6cf58d5ae4f3e3b646ea262f75e31d23de66761e8e1aacaa42fca42b110869f08428881b748095e2d6
  • Docs.zip SHA1: 19a34ddd89f6e5e0efc04fdc080a79acf48e0f7b
  • Docs.zip SHA512: 2c604884fdb1653b641a7d240a3533293ee763a085f14ba4df3cd883bd1e12d42ac8c69eaf694ad211115c7a1acb424036c9ebcc160510d39d8c9dfd7d2374a9

Have fun!

62 Likes
2

Shouldn’t Code.load_file/2 be in that list as well?

4

I really like that the config situation is starting to get cleared up :slight_smile: I do wonder if this still means that it is necessary to create a custom command to run migrations? I’ve always kind of hated having to do stuff like this distillery/docs/guides/running_migrations.md at master · bitwalker/distillery · GitHub in release images, instead of using the standard mix ecto.migrate which seems perfectly suited for the job.

6 Likes
5

There seems to be more than only Code.load_file/2:

$ git describe --tags
v1.9.0-rc.0
$ git grep -n 'TODO: Deprecate' '*.ex' | grep v1.9
lib/elixir/lib/code.ex:57:  # TODO: Deprecate on v1.9
lib/elixir/lib/code.ex:89:  # TODO: Deprecate on v1.9
lib/elixir/lib/code.ex:762:  # TODO: Deprecate on v1.9
lib/elixir/lib/supervisor/spec.ex:112:  # TODO: Deprecate all functions in this module on Elixir v1.9.
lib/logger/lib/logger.ex:841:  # TODO: Deprecate compile_time_purge_level on v1.9
lib/mix/lib/mix/task.compiler.ex:113:  # TODO: Deprecate :ok and :noop on v1.9
6

Nice!
I’m curious, what does mix release mean to distillery+edeliver combo? I’m asking, because I’ve used it so far, and I don’t know what are potential steps to replace distillery. I know, it’s early to ask it, but maybe someone has thought about it already :smiley:

7

Elixir’s releases are basically a subset of what distillery does (it doesn’t include anything for hot code updates for example). Distillery already seems to have moved it’s mix tasks to a distillery.* namespace and should continue working there.

1 Like
8

If anyone is looking for the documentation on releases:

https://hexdocs.pm/mix/1.9.0-rc.0/Mix.Tasks.Release.html

4 Likes
9

Those are the minimum versions to deprecate them. In this case, we have postponed them as we are increasing the gap between soft-deprecation and hard-deprecation, but they will be eventually deprecated.

1 Like
10

We’ve published a blog post Updating Hex.pm to use Elixir Releases about exactly that, hope you’ll find it helpful.

5 Likes
11

Woah! mix release is quite exciting! Congratulations to the Elixir team and thank you for all the hard work. You make our lives easier.

9 Likes
12

Are config providers only available to releases or is it possible to use them when running in dev?

1 Like
13

All of my projects compile and test with no errors on 1.9. I haven’t tried releases yet. Is there a migration guide from distillery?

1 Like
14

Try out

2 Likes
15

Is there more detailed documentation of config/releases.exs available somewhere?

16

The docs linked above talk about it: https://hexdocs.pm/mix/1.9.0-rc.0/Mix.Tasks.Release.html#module-runtime-configuration

4 Likes
17

Once a release is assembled, it can be packaged and deployed to a target as long as the target runs on the same operating system (OS) distribution and version as the machine running the mix release command.

Are there plans to remove those limitations in the future?

18

I’ve never seen the BEAM have any sort of cross-compilation to date, in addition the NIF’s would have to be cross-compiled in whatever language they are written and so forth as well. I.E. it’s not possible in any easy timeframe.

2 Likes
19

Really glad to see releases included as part of the language – hats off to you guys!

1 Like
20

http://erlang.org/doc/installation_guide/INSTALL-CROSS.html

Quite a few products that I know of uses the cross compilation support to build Erlang/OTP.

However as you say, cross compiling nifs is a problem as many nifs writers have not done what needs to be done in order to make cross compilation work. I put together syslogger as an example/experiment of how one could achieve cross-compilation for nifs (if you ignore cross compiling to windows).

6 Likes
21

Can anyone comment on how much the new release process and configuration tools have improved in terms of deploying Phoenix?

A while ago I wrote up a guide for myself since the previous process was pretty complicated and it was the Distillery docs that were the only ones that even had the setup for Phoenix.

The pain points for me were:

  1. Having to go through the Distillery docs in order to deploy a Phoenix application (they were pretty good but lacking in some areas and had bugs in others). There was also a general lack of guidance on how to do more advanced things.
  2. Having to figure out how to use environment variables for configuration at runtime in a release or figure out how to properly set the configuration with environment variables when building a release.
  3. Having to create special code and special release tasks just to run migrations when those could already be run from the command line with mix. This included having to setup shell scripts.
  4. Figuring out how to load the secret key from an environment variable at runtime.
  5. Little to no information regarding Postgres/Ecto in general (the new version of the docs has almost no information at all).
1 Like