Nix vs asdf for Elixir version management

I am actually in the (very slow) process if making the BEAM work better within Nix (I have posted about this elsewhere on the forum before, maybe look for that for some more information). I am writing a tool right now that will convert a rebar3 or mix project into a default.nix file similar to tools for other languages.

Automatic context switching

Like you said, there is none. You will just need to get used to calling nix-shell or some other tool to get where you want. But this isn’t necessarily a bad thing either. You are being explicit about what you want.

Limited versions

This is also true. But I don’t really see having only the latest patch for each minor version available. Everything should be backward compatible (assuming you are not relying on internal functionalty, but there is already a post about that elsewhere). So it is just bug fixes and such.

Time for a new version

Again, true. This is mostly true about any OS package manager. Nix needs to make sure that all of the packages in the ecosystem play nicely together. With that said, I did make it super easy to make your own version of Elixir if it is not found in nixpkgs. Just take a look at how we define v1.7 and the corresponding entry to actually build it. If you need a version that is not listed or has not yet been updated. Feel free to add your own for your project!

Contributions

Just like you do not need asdf installed to contribute to a package that has asdf, you do not NEED to install Nix to contribute to a package that uses it. Just make sure you outline your requirements in your contribution guide.

For the vast majority of uses, I would currently say to stay away from Nix and NixOS for BEAM specific work. Or if you want to use NixOS, use it like a regular linux for the time being (I am doing this for a couple things).

For anyone else that REALLY wants to use Nix and NixOS right now, send me a message and we will see if we can get the BEAM integration moving a little faster.

I’m a relatively happy Nix user in general, including on OSX and as a Linux desktop via NixOS, but I would be very unhappy teaching most people to use it, even those who are able to effectively adapt to both Elixir and ASDF. It’s a lot to ask for little return unless you already find it academically interesting. It has a lot of parallels in my mind to my use of Emacs - I personally find it very powerful and compelling, but struggle to articulate well to others who are skeptical, or who have different levels of tenacity in the face of ~user-unfriendly~ complex software.

I will also say that very specifically the abstractions used in the BEAM packaging within nixpkgs are over-abstracted from my POV. They are probably very useful/powerful to the internal/official maintainers, but are sort of a nightmare to consume as someone who is literate with the syntax but not an expert. If you’re not sure what I mean, try to figure out how to use a Nix overlay or shell.nix to install a particular patch release of Erlang and of Elixir, without it having been packaged already. You can’t use a simple override, nor have I found an easy way to otherwise “patch” the URL and checksum of the target download like I can with other languages’ Nix packaging.

I ultimately fell away from using Nix specifically for the BEAM ecosystem and back to ASDF, even when I persist with Nix elsewhere.

For those unfamiliar, here’s an example of shell.nix content:

# 2018-06-26 11:30
# https://d3g5gsiof5omrk.cloudfront.net/nixpkgs/nixpkgs-18.09pre143801.5ac6ab091a4/nixexprs.tar.xz
with import <nixpkgs>{};

let
  erlang = beam.packages.erlangR21.erlang;
  elixir = beam.packages.erlangR21.elixir;
  nodejs = pkgs.nodejs-10_x;
  yarn = pkgs.lib.overrideDerivation pkgs.yarn (attrs: rec {
    buildInputs = [pkgs.makeWrapper nodejs];
  });
in

stdenv.mkDerivation rec {
  name = "my_elixir_project";
  version = "0.1.0";
  buildInputs = [
    erlang
    elixir
    nodejs
    yarn

    # phoenix_live_reload
    pkgs.darwin.apple_sdk.frameworks.CoreFoundation
    pkgs.darwin.apple_sdk.frameworks.CoreServices

    # postgrex
    pkgs.postgresql100

    # developer tools
    pkgs.direnv
    pkgs.gitMinimal
    pkgs.hex2nix
  ];
}

There are some natively supported but non-obvious ways to pin back to a historical version in a specific shell.nix context. This is a sample based on the mailing list, and you’re ultimately required to track down a GitHub SHA for the timestamp within nixpkgs that has the exact release you want. This can be used concurrently with other packages from different timestamps within nixpkgs history, but you run the likelihood of maintaining several concurrent “trees” of the underlying dependencies that take up some multiple of the space consumed by only 1.

{ pkgs ? import <nixpkgs> { } }: let

  myPkgs = import (pkgs.fetchFromGitHub {
  owner = "nixos";
  repo = "nixpkgs";
  rev = "f607771";
  sha256 = "1icphqpdcl8akqhfij2pxkfr7wfn86z5sr3jdjh88p9vv1550dx7";
}) {};

in pkgs.stdenv.mkDerivation {
  name = "pandoc";
  buildInputs = [
    myPkgs.pandoc
  ];
}

Regarding space consumption, you can minimize this somewhat with nix-store --optimise -v to perform hard-linking, if the contents of those dependencies hasn’t changed much or at all. It’s really valuable to learn how the nix store and garbage collection work, because I often find I have several gigabytes of redundant files stored, even with directives like this in my /etc/nix/nix.conf:

gc-keep-outputs = true
gc-keep-derivations = true
env-keep-derivations = true

auto-optimise-store = true

Thank you for your replies!

I have effectively seen some tools like this for languages like Rust or Haskell. What is the point compared to just set up the environment in a shell.nix and using mix?

How do you manage to use asdf on NixOS to build Erlang? Do you install build dependencies globally? Do you have a list of such dependencies somewhere?

I’ve used something comparable from the Go ecosystem to allow me to quickly package a Golang project I don’t manage. Were I a developer on the project, I wouldn’t tackle it that way, but I actually just want to use its included CLI binary without a bunch of non-reproducible go get ... commands, not hack on the project. That may be the motive in other scenarios too, or it could also be handy if you’re deploying your project from source directly via Nix and only Nix, rather than using Nix → Mix, or something like Distillery.

I’ll try to test this tonight and let you know what I have in /etc/nixos that appears to be making it work. It’s not my primary machine.

I’m still gathering my thoughts, but based on @Ankhers post above I did get cranky enough to work my way through nixpkgs source and nix repl to figure out how to call the same functions from the source files he linked. This should allow us to get a minimal shell environment of any arbitrary Erlang and Elixir versions with a shell.nix and two secondary .nix files. I’ll try to share that here too. And if there’s a way to do it one single shell.nix, I’m all ears! Here’s what I’m working with so far:

Oh, regarding your custom versions draft, this seems really neat. I was just playing around with nix-shell, editing my .zshrc not to import again PATH-modifying scripts so that I can keep some useful aliases and override my environment. I will definitely give a try to Nix to setup the environment, using custom derivations as needed.

Thank you very much!

Ok, I better understand like this. In a build / deployment process it is easyer just to have to nix-build indeed. I was thinking about the development process where one typically prefers to use tooling from the language ecosystem.

I have played around a bit and come to this shell.nix for Elixir 1.7.0-rc.1 using Erlang 21 from nixpkgs:

I’ve passed rebar from beam.packages.erlangR21 to the Elixir builder to avoid depending on Erlang 19—on which rebar depends otherwise.

This seems very promising. My next step is trying to setup a full environment for Nerves.

For those interested, I’ve set up a an environment using custom versions for Elixir and Erlang and some dependencies for Nerves:

Again, I’ve defined a custom rebar using the compiled Erlang instead of the one from nixpkgs.

To avoid the shell dependencies being garbage-collected, I’ve found you can do:

$ nix-instantiate shell.nix --indirect --add-root $PWD/shell.drv

With keep-outputs = true in your nix.conf, this prevents such long to build derivations to be collected. When you don’t work anymore on the project and want to get rid of your shell dependencies, you just have to delete shell.drv and run the garbage collector. Hope this will help

By the way, I don’t find how to edit my previous post to avoid this kind of double-post.

An update for future readers: I have completely switched from asdf to Nix lately. It is a joy to use to setup a complete environment, including non-language dependencies you can need to build different kind of projects. I’ve set up standard shell.nix for both standard, Phoenix and Nerves projects. I have also written an article about it which I hope will be useful to Nix newcomers.