Nix, the package manager

General Prog (Archived)
1

I know I said I would make this thread “in the next day or two” about a month ago. I am finally getting around to it. Sorry for the delay!

The Confusion

Lets get a bit of terminology out of the way. It is rather unfortunate, but nix can refer to a couple things within the ecosystem and they are sometimes used interchangibly.

Nix the package manager

As you have probably guessed, this is going to be mostly what my post is about. I will be giving some basic examples of how to actually use nix, and hopefully a handful of other people will chime in with their workflows on how to use it.

Nixpkgs

This is the default repository that nix runs against. You can find a lot of different packages here, and if there is something that you want that is not currently packaged, you have the ability to add it yourself!

NixOS

This is a linux based operating system built from the ground up to support nix.

Nix the programming language

This is the language used to describe and build nix packages.

The basics

For anyone that has not yet heard of nix, it is a purely functional package manager. It aims to be able to give reliable and reproducible builds. As mentioned previously, nixpkgs runs from a git repository, so it is possible to “pin” to the git ref that you want to build against. This is very useful if you have a team building a project with nix so that your entire team can be sure to be using the same version of all software.

You can use it along side your current installation of any other package manager(s) you may be using. It works by creating a /nix/store directory where it puts everything. An example in the directory will look like

cq8qrvg8s1jwcli4w2rnd4nlays35ikf-elixir-1.8.2

This is an installation for Elixir 1.8.2 that I have. cq8qrvg8s1jwcli4w2rnd4nlays35ikf is a hash of all of that packages inputs (including its dependency graph) in order to uniquely identify that package. I will come back to why this is important a little later.

Installation

Installation is very simple. On any Linux or MacOS machine, just run

$ curl https://nixos.org/nix/install | sh

as a user that is not root. You can find these instructions on the main page for Nix.

Installing and removing packages

You can install a package using the nix-env command line utility.

$ nix-env -i hello

With the above, you will install GNU Hello. To remove a packge, simply

$ nix-env --uninstall hello

The previous command will only unlink the package from your $PATH. To actually remove it, and all unused packages, you will need to run

$ nix-collect-garbage -d

The -d option will make it actually delete the packages and free your disk space.

If you would like to search for available packages, you can find most listed here.

Multiple of the same package

This is one of the killer features of Nix in my opinion. The ability to have multiple of the same package, even multiples of the same version, installed simultaneously. This is useful for when you have different projects that are configured differently (think C project with different switches to enable other features). This is the reason for the hash that prefixed the package name and version in the /nix/store path.

The hash also has a second function, in that it is a unique identifier. When you try to install a package, nix will calculate that hash and then ask nix’s cache whether or not it has something with that identifier. If it does, it will just download it and throw it into your /nix/store path. If it does not, it will just follow the build instructions in order to build your package. This is great because it means you will generally not have to wait for long compile times on larger packages.

Nix shell

Nix shell is a really handy tool if you want to test a package without bringing it into your global package set, or if you are working on a project that supports nix.

In order to test a package, you would simply

$ nix-shell -p elixir

If this is your first time running the command, it will download the current version of Elixir (current is dependent on the nix release channel you are currently following) and all of it’s dependencies. It will then drop you into a bash shell where you have access to elixir. You can test by running iex or any other binary that Elixir ships with.

If you were to run that command again, it will immediately drop you into the bash shell with Elixir because it is already downloaded.

Using it in a project

The easiest way to get started using nix in a simple elixir project would be to use the previous nix-shell command. It pulls in the basics required for building your project. This may work well enough for a while if you are the only person doing this. However, a better solution would be to build a basic shell.nix file. If you put that file in the root of your project, you can run $ nix-shell and it will automatically download any of the inputs it needs (in this case, just elixir) and drop you into a shell to start working on your project.

Choosing your Elixir version

The previous example will just use the current (based on the release channel you are following) version of Elixir available to you. What if you wanted to use a specific release version, or even a custom version (master, specific branch, etc)

Using a specific version

This is rather easy. In the example shell.nix file I linked, you can see that we depend on pkgs.elixir. If you needed to stay at say, Elixir 1.4, you would just use pkgs.elixir_1_4 instead. This should set you up using a specific version of Elixir instead of rolling with whichever version becomes the default.

Using a custom version

This becomes a bit more involved, but still fairly easy.

This gist describes the two files needed in order to use a custom version of Elixir. In this case, I just grabbed a recent random commit and plugged it into the file. Since we are using a non-standard version, it will need to compile Elixir, so it may take a bit of time, depending on your computer.

How does Erlang fit into this?

Right now, the default version of Erlang across all of nixpkgs is R20. This can of course be changed depending on your needs. In the case of using a specific version of Elixir, you will just need to replace pkgs.elixir with pkgs.beam.packages.erlangR21.elixir_1_4. Replace erlangR21 and elixir_1_4 with whichever versions of Erlang and Elixir you want.

To use a different version of Erlang with a custom Elixir version, just replace inherit rebar erlang with inherit rebar erlang = erlangR21, or whichever version of Erlang you want to use.

Keep in mind with the way we have nix building things, the version of Elixir that you are running, will ALWAYS be compiled with the version of Erlang you are running. So you should never run into issues with compatibility.

Wrapping up

This took a bit longer than I expected to put together, but here it is. If you want to look into nix a bit more, I will post a handful of external links to various blog articles, videos, communities, etc that will hopefully get you off the ground. Feel free to ask any questions about anything nix related. I am by no means an expert, but I will do my best to answer or find someone who can.

External links

Nixos Forums
#nixos on freenode
Using nix with Haskell (I know this is not BEAM related, but he does some really cool things that I hope we can get going with the BEAM community at some point)
Nix manual
Nixpkgs manual
Nix pills

19 Likes
2

Adding some relevant resources:

Elixir/Erlang-specific:

General Nix dev workflow:

shell.nix gists:

Tools to build Mix projects with Nix:

mkShell vs mkDerivation

Older (< 2018) posts tend to use mkDerivation instead of mkShell, but the latter is only a convenience wrapper around the former when writing Nix expressions for nix-shell. See mkShell needs more documentation in the manual #58624 Nixpkgs issue for more info and links.

Advantages of mkShell

(For creating dev environments, that is.)

  • accepts an inputsFrom list, to “to pull all the dependencies together” from a large project, for example. From the initial PR for mkShell:

    […] you have multiple projects each with their own shell.nix , and you could create 1 master shell.nix that merges all subproject’s shell.nix in case you wanted to work on all the subprojects together. (CMCDragonkai’s comment)

    Imagine you have a monorepo with multiple services, each in their own folder with their own default.nix that you can nix-shell and nix-build. On the top-level you can then add a shell.nix with, mkShell { inputsFrom = [ serviceA serviceB ]; to pull all the dependencies together. (zimbatm’s comment)

  • shellHook s are also composed form input expressions. See PR #63701 for examples.

7 Likes
3

Some ancillary resources I would recommend reviewing, and following the development in the case of nix-flakes especially:

  • home-manager - declarative management of your userspace tools and configuration files and services/daemons :heart:
  • nix-darwin - emulate NixOS’ declarative management for OSX users specifically :heart:
  • NUR - Nix User Repository, a specification for user-maintained Nix packages that can be incorporated fairly cleanly into your own Nix efforts, albeit with user-to-user trust being a strong operating requirement
  • niv lets you pin and bump the versions of any contributing Nix channels that you use in your default.nix and shell.nix to have point-in-time reproducible builds without requiring users to alter their own Nix usage habits (but does cause redundant storage of derivations and packages that co-exist in your niv definitions as well as their other system-wide Nix use.
  • nix flakes (original gist) - seed for a reusable modular definition for referring to external Nix code in a way that is slightly different from either NUR or configured overlays and potentially gets upstreamed:
4 Likes
4

I will also throw out a link to Hercules CI which is currently in pre-release invite-only mode, which I haven’t made up my mind about yet - but a glance, it seems mutually exclusive with Hydra jobs for the same self-authored package repository I’m working on, because it stipulates that you cannot use channel-based imports with i.e. <nixpkgs> syntax. Very snappy to respond to git pushes, though.

2 Likes
5

A minimal shell.nix example for a Phoenix project

Take with a grain of salt, first try, but it works on NixOS. Would be happy to hear any suggestions or criticisms.

The PostgreSQL database will be initialized with the current user, therefore you may have to change config/<env>.exs connection setup from postgres user to your username (the password remained postgres by default). To connect from the terminal, use psql -h $PGDATA -U your_username -d postgres.

####################################################################
# Importing a cloned Nixpkgs repo  (from my home directory), because
# the latest channels don't have Elixir 1.9.
# See https://nixos.org/nix/manual/#idm140737317975776 for the meaning
# of `<nixpkgs>` and `~` in Nix expressions (towards the end of that
# section).
####################################################################

{ pkgs ? import ~/clones/nixpkgs {} }:

pkgs.mkShell {

  buildInputs = with pkgs; [
    beam.packages.erlangR21.elixir_1_9
    postgresql_11
    nodejs-12_x
    git
    inotify-tools
  ];

  shellHook = ''

    ####################################################################
    # Create a diretory for the generated artifacts
    ####################################################################

    mkdir .nix-shell
    export NIX_SHELL_DIR=$PWD/.nix-shell

    ####################################################################
    # Put the PostgreSQL databases in the project diretory.
    ####################################################################

    export PGDATA=$NIX_SHELL_DIR/db

    ####################################################################
    # Put any Mix-related data in the project directory
    ####################################################################

    export MIX_HOME="$NIX_SHELL_DIR/.mix"
    export MIX_ARCHIVES="$MIX_HOME/archives"

    ####################################################################
    # Clean up after exiting the Nix shell using `trap`.
    # ------------------------------------------------------------------
    # Idea taken from
    # https://unix.stackexchange.com/questions/464106/killing-background-processes-started-in-nix-shell
    # and the answer provides a way more sophisticated solution.
    #
    # The main syntax is `trap ARG SIGNAL` where ARG are the commands to
    # be executed when SIGNAL crops up. See `trap --help` for more.
    ####################################################################

    trap \
      "
        ######################################################
        # Stop PostgreSQL
        ######################################################

        pg_ctl -D $PGDATA stop

        ######################################################
        # Delete `.nix-shell` directory
        # ----------------------------------
        # The first  step is going  back to the  project root,
        # otherwise `.nix-shell`  won't get deleted.  At least
        # it didn't for me when exiting in a subdirectory.
        ######################################################

        cd $PWD
        rm -rf $NIX_SHELL_DIR
      " \
      EXIT

    ####################################################################
    # If database is  not initialized (i.e., $PGDATA  directory does not
    # exist), then set  it up. Seems superfulous given  the cleanup step
    # above, but handy when one gets to force reboot the iron.
    ####################################################################

    if ! test -d $PGDATA
    then

      ######################################################
      # Init PostgreSQL
      ######################################################

      initdb $PGDATA

      ######################################################
      # PostgreSQL  will  attempt  to create  a  pidfile  in
      # `/run/postgresql` by default, but it will fail as it
      # doesn't exist. By  changing the configuration option
      # below, it will get created in $PGDATA.
      ######################################################

      OPT="unix_socket_directories"
      sed -i "s|^#$OPT.*$|$OPT = '$PGDATA'|" $PGDATA/postgresql.conf
    fi

    ####################################################################
    # Start PostgreSQL
    ####################################################################

    pg_ctl -D $PGDATA -l $PGDATA/postgres.log  start

    ####################################################################
    # If $MIX_HOME doesn't exist, set it up.
    ####################################################################

    if ! test -d $MIX_HOME
    then
      ######################################################
      # Install Hex and Phoenix
      ######################################################

      yes | mix local.hex
      yes | mix archive.install hex phx_new

      ######################################################
      # `ecto.setup` is defined in `mix.exs` by default when
      # Phoenix  project  is  generated via  `mix  phx.new`.
      # It  does  `ecto.create`,   `ecto.migrate`,  and  run
      # `priv/seeds`.
      ######################################################

      mix ecto.setup

      mix deps.get
    fi
  '';

  ####################################################################
  # Without  this, almost  everything  fails with  locale issues  when
  # using `nix-shell --pure` (at least on NixOS).
  # See
  # + https://github.com/NixOS/nix/issues/318#issuecomment-52986702
  # + http://lists.linuxfromscratch.org/pipermail/lfs-support/2004-June/023900.html
  ####################################################################

  LOCALE_ARCHIVE = if pkgs.stdenv.isLinux then "${pkgs.glibcLocales}/lib/locale/locale-archive" else "";
}
4 Likes
6

I want to show off how I’m using Nix as a development environment for Elixir Projects.
-> https://github.com/cw789/elixir_nix_seed

The big feature for me is, in a team I can make sure everyone is able to use an identical development environment.

6 Likes
7

A very minimalist approach: shell.nixes

3 Likes
8

Truly minimal shell.nix:

{ pkgs ? import <nixpkgs> {} }:

with pkgs.beam.packages.erlang;
pkgs.mkShell {
    buildInputs = [ elixir ];
}
2 Likes
9

I’ve updated my educational repository to show off how I’m using Nix right now: Github

  • PostgreSQL was not working correctly with v11 (adapted some steps from @toraritte)
  • mix deps.get won’t work if Nix is running with sandbox enabled (added --option sandbox relaxed)
  • Added the cool short shell.nix from @hauleth
1 Like
10

A related thread from the Nix discourse:
How to configure PostgreSQL declaratively (NixOS and non-NixOS)?

11

9 posts were split to a new topic: Guix package manager