Building an Elixir app with Nix and git dependencies

Hello friends,

I’m desperately trying to have a nix build for one my app. I think I’m on the right track but I have a git dependency on the way that just doesn’t seem to be fetchable.

Please note that this is my first actual Nix build, I’ve been running NixOS for a few months now and only been using Nix shells using flakes, but no actual build.

The dependency is a public one: GitHub - nicklayb/box_ex: My Elixir toolbox but the project I’m trying to build is currently private. I tried to follow a few blog posts/forum posts/documentation, but here’s how my flake looks at the moment

Relevant files

flake.nix

{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils";
  };
  outputs =
    {
      self,
      nixpkgs,
      flake-utils,
    }:
    flake-utils.lib.eachDefaultSystem (
      system:
      let
        pkgs = nixpkgs.legacyPackages.${system};
        erl = pkgs.beam.interpreters.erlang_27;
        erlangPackages = pkgs.beam.packagesWith erl;
        elixir = erlangPackages.elixir;
      in
      {
        devShells.default = pkgs.mkShell {
          buildInputs = with pkgs; [
            elixir
            erlang_26
            direnv
            inotify-tools
            just
            gphoto2
            v4l-utils
            jq
            ffmpeg
          ];

          shellHook = ''
            eval "$(direnv hook bash)"
            direnv allow
            mix deps.get
          '';
        };
        packages = let
          version = "0.1.0";
          src = ./.;
          mixFodDeps = erlangPackages.fetchMixDeps {
            inherit version src;
            pname = "photo-boite-deps";
            sha256 = "sha256-njEjczedx4mRDZrIzqXXtwr7OoDroOG7hcqzU4WXu6U=";
          };
          translatedPlatform = {
            aarch64-darwin = "macos-arm64";
            aarch64-linux = "linux-arm64";
            armv7l-linux = "linux-armv7";
            x86_64-darwin = "macos-x64";
            x86_64-linux = "linux-x64";
          }
          .${system};
        in rec {
          default = erlangPackages.mixRelease {
            inherit version src mixFodDeps;
            pname = "photo-boite-web";
            postBuild = ''
              mix do deps.loadpaths --no-deps-check, phx.digest
            '';
            preInstall = ''
              ln -s ${pkgs.tailwindcss}/bin/tailwindcss _build/tailwind-${translatedPlatform}
              ln -s ${pkgs.esbuild}/bin/esbuild _build/esbuild-${translatedPlatform}

              ${elixir}/bin/mix assets.deploy
              ${elixir}/bin/mix phx.gen.release
            '';
          };
        };
      }
    );
}

mix.exs

defmodule PhotoBoite.MixProject do
  use Mix.Project

  def project do
    [
      app: :photo_boite,
      version: "0.1.0",
      elixir: "~> 1.14",
      elixirc_paths: elixirc_paths(Mix.env()),
      start_permanent: Mix.env() == :prod,
      aliases: aliases(),
      deps: deps()
    ]
  end

  def application do
    [
      mod: {PhotoBoite.Application, []},
      extra_applications: [:logger, :runtime_tools]
    ]
  end

  defp elixirc_paths(:test), do: ["lib", "test/support"]
  defp elixirc_paths(:dev), do: ["lib", "test/support"]
  defp elixirc_paths(_), do: ["lib"]

  defp deps do
    [
      {:bandit, "~> 1.5"},
      # here's my git dep
      {:box, git: "https://github.com/nicklayb/box_ex.git", tag: "0.14.0"},
      {:credo, "~> 1.7.11", runtime: false, only: ~w(dev test)a},
      {:ecto_sql, "~> 3.10"},
      {:esbuild, "~> 0.8", runtime: Mix.env() == :dev},
      {:gettext, "~> 0.26"},
      {:jason, "~> 1.2"},
      {:postgrex, ">= 0.0.0"},
      {:phoenix, "~> 1.7.19"},
      {:phoenix_ecto, "~> 4.5"},
      {:phoenix_live_view, "~> 1.0.4"},
      {:tailwind, "~> 0.2", runtime: Mix.env() == :dev},
      {:tz, "~> 0.28"},
      {:qr_code, "~> 3.2.0"}
    ]
  end

  defp aliases do
    [
      setup: ["deps.get", "ecto.setup", "assets.setup", "assets.build"],
      "ecto.setup": ["ecto.create", "ecto.migrate", "run priv/repo/seeds.exs"],
      "ecto.reset": ["ecto.drop", "ecto.setup"],
      test: ["ecto.create --quiet", "ecto.migrate --quiet", "test"],
      "assets.setup": ["tailwind.install --if-missing", "esbuild.install --if-missing"],
      "assets.build": ["tailwind photo_boite", "esbuild photo_boite"],
      "assets.deploy": [
        "tailwind photo_boite --minify",
        "esbuild photo_boite --minify",
        "phx.digest"
      ],
      gettext: [
        "gettext.extract",
        "gettext.merge priv/gettext"
      ]
    ]
  end
end

The error I’m getting is (only in nix build, works fine locally in a shell):

       last 25 log lines:
       > Generated bandit app
       > ==> websock_adapter
       > Compiling 4 files (.ex)
       > Generated websock_adapter app
       > ==> phoenix
       > Compiling 71 files (.ex)
       > Generated phoenix app
       > ==> phoenix_live_view
       > Compiling 39 files (.ex)
       > Generated phoenix_live_view app
       > ==> box
       > Compiling 37 files (.ex)
       > Generated box app
       > ==> phoenix_ecto
       > Compiling 7 files (.ex)
       > Generated phoenix_ecto app
       > Running phase: buildPhase
       > Compiling 20 files (.ex)
       > Generated photo_boite app
       > Running phase: installPhase
       > Unchecked dependencies for environment prod:
       > * box (https://github.com/nicklayb/box_ex.git - 0.14.0)
       >   lock mismatch: the dependency is out of date. To fetch locked version run "mix deps.get"
       > ** (Mix) Can't continue due to errors on dependencies

Where I am right now:

  • I see mentions of mix2nix (instead of erlangPackages.fetchMixDeps), I don’t fully understand the difference, but mix2nix explicitly says
    • Currently, only public packages from Hex.pm are supported. If you have any dependencies from git, private repositories, or local sources, you will need to manually specify those.

    • I tried to see how to add it manually, but box_ex has its own dependency, do I need to generate a deps.nix on the git dependency too?
  • I saw people saying that there might be build artifact in the nix env so making sure that _build as well deps is gitignore, that’s the case.
  • I noticed that the deps SHA has influence, I had tried to update all my deps and they all had the lock mismatch error, once I updated the SHA, I went back to having only box_ex causing error.

I’m a little lost right now, I love the principle of Nix but the documentation isn’t quite optimal, neither is the tooling.

Any help would be appreciated. Thank you all!

2 Likes

I’ve built a library for this exact purpose: flakify. It requires you to install igniter first, and then you probably need to remove your own flake.nix and flake.lock for it to work, it’ll generate new ones for you.

3 Likes

The link doesn’t seem to work, it doesn’t point to anything.

However, I would likely understand what I am doing, and not rely on something that “dumps” file for me, if you can guide me through what the tool does to make git deps work, I’d be more interested in that

2 Likes

I apologize for my terseness in the previous response, I was on the train. I’ve corrected the link.

To be clear, this “library”, is actually just a tool that writes a flake.nix file and alters a few things in the default Phoenix setup to make it work. It uses deps.nix to handle dependencies, so (contrary to mix2nix) it works just fine with github dependencies. You could even run flakify in a different test Phoenix repo and just copy the changes over manually.

I have plans to extend the functionality of the script so that it also can produce a NixOS module, systemd service and perhaps a deploy-script, but that’s not done yet.

3 Likes

Try setting the mixFodDeps sha256 to lib.fakeHash. Then build.

I have found that’s necessary to get the actual hash mismatch error. Otherwise it presents exactly as you’re seeing.

1 Like

I’m getting

warning: Git tree '/home/nboisvert/dev/photo_boite' is dirty
error: hash mismatch in fixed-output derivation '/nix/store/78w5q84ik8dnif7x6r9zp1j1908vhz44-photo-boite-deps-0.1.0.drv':
         specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
            got:    sha256-njEjczedx4mRDZrIzqXXtwr7OoDroOG7hcqzU4WXu6U=
error: 1 dependencies of derivation '/nix/store/yyph7wwwvjib4a9f1p5jr41klrwj5nmf-photo-boite-web-0.1.0.drv' failed to build

My understanding of fakeHash was that is just a placeholder until running it gives you the actual hash. I might have been wrong though.

flakify didn’t worked. It fails with:

flakify.install` x
** (Protocol.UndefinedError) protocol Enumerable not implemented for type Rewrite (a struct). This protocol is implemented for the following type(s): DBConnection.PrepareStream, DBConnection.Stream, Date.Range, Ecto.Adapters.SQL.Stream, File.Stream, Function, GenEvent.Stream, HashDict, HashSet, IO.Stream, Jason.OrderedObject, List, Map, MapSet, Phoenix.LiveView.LiveStream, Postgrex.Stream, Range, Stream

Got value:

    %Rewrite{
      sources: %{
        ".formatter.exs" => %Rewrite.Source{
          from: :file,
          path: ".formatter.exs",
# ... All that follows seems to be ASTs for my files

But I saw the deps_nix portion. I’ll give that a try, that’s pretty much the only bit I’m interested it

Yeah… my hypothesis was it would give you a hash different than the one in your flake. I was wrong.

Check out my recent thread about deploying on NixOS. The overlays.nix has methods that worked for me. FWIW I spent a lot of time trying mix2nix and deps.nix approaches, but FOD style was what worked in the end. It was github dependencies that kept gumming things up, among other things.

I will take a closer look later and try to sus out the difference here.

I did managed to build using deps-nix. I can now run the binary without any problem. Just gotta add the systemd unit at this point but overall seems fine.

What did it to me was:

  • Use deps-nix to generate a deps.nix file, unlike mix2nix, this one does work with git deps
  • Make sure all my mix calls were doing mix do deps.loadpaths --no-check-deps, [the command]

Thanks @munksgaard for pointing me to flakify which then pointed to deps-nix!

3 Likes

A bit of update, I reached a point where the build runs without any hiccups but fails as a nixosmodule. I’m really not sure why. I end up with

ln: failed to create symbolic link 'deps/expo/src': File exists

When I try to enable the nixosModule in a system. Running nix build works 100% but as a module it doesn’t. I assumed the whole point of nix was to make sure that if it builds as a package, it’s gonna build when used as a module.

The module doesn’t do any overlay and stuff, it’s really just doing environment.systemPackages = [default] (where default is the build derivation).

Getting there! The reason why I had this ln issue was because I was referring to my flake locally (as path:/path/to/my/flake in my inputs).

I assumed this would follow the same rule as to ignore what’s in .gitignore but that’s not the case, my deps folder was also copied over to the store while trying to build. I’m guessing this is not a problem for anyone using a flake in repository but I’m currently testing the flake and wanna avoid pushing everytime.

Have you been able to implement a checkPhase (using mix test)? I can’t seem to get mix to ignore the dependency check for the test task, even though I’ve included --no-check-deps.

  checkPhase = ''
    MIX_ENV=test ${elixir}/bin/mix test --no-deps-check
  '';

The above fails with “Unchecked dependencies for environment test”, even though the dependencies are available in ./deps and other tasks work just fine (ie. assets.deploy).