devenv_new – Mix task wrapper to create projects in a new devenv.sh Nix environment

Your Libraries & OS Mentoring Libraries
, , , ,
1

A Mix task that wraps any Elixir Mix project generator in a devenv.sh environment.

Note that you can install Nix on top of macOS or most Linux distros without requiring a full NixOS setup.

Installation

Install the archive directly from Hex:

mix archive.install hex devenv_new

Usage

Use mix devenv.new to wrap any Mix project generator:

# Create a Phoenix project with devenv
mix devenv.new phx.new my_app --devenv postgres,redis

# Create an Igniter project with devenv
mix devenv.new igniter.new my_project --devenv elixir=1.17,postgres --install ash,ash_postgres

# Create a basic Elixir project with devenv
mix devenv.new new my_lib --devenv elixir=1.17,minio --sup

# To run Mix in a temporary Elixir Nix environment
nix-shell -p elixir --run 'mix devenv.new igniter.new demo_app '\
'--devenv postgres,bun '\
'--install ash,ash_postgres,ash_authentication_phoenix,ash_graphql '\
'--auth-strategy magic_link '\
'--with phx.new'

cd demo_app
devenv up
MIX_ENV=test mix ash.reset
mix test

See story.html for a full log of a generator run.

Devenv Features

Available devenv features:

  • elixir - Elixir runtime (supports version specification, e.g., elixir=1.17)
  • postgres - PostgreSQL database with project-specific databases
  • redis - Redis cache/session store
  • minio - MinIO object storage (S3-compatible)
  • npm - Node.js runtime with npm
  • bun - Bun runtime/package manager

The generator is built to be easily extendable, by creating an .eex template file and adding it to the look-up table in devenv.new.ex.

6 Likes
2

Super cool! I will try it out later.

One question, did you also setup the variable envs so all created files are encapsuled inside the project directory? For example, in my devenv config, I setup it so all postgres data will live inside the project directory (I think inside .env/.devenv/state/postgres), I also made that all elixir config files like hex, mix, erlang-history, etc are all inside the .env/.devenv/state too.

In other words, running elixir, postgres, erlang, etc never leaks anything outside the scope of the project.

1 Like
3

No special env setup, so far. PostgreSQL lives under .devenv/state/postgres by default. All state is contained within the project folder, but not necessarily under .devenv.

If you have a good env setup to share, let me know. So far it only knows about languages and services, but other classes could be added.

4

For Elixir, I define these variables:

  state_dir = config.env.DEVENV_STATE;

  mix_dir = "${state_dir}/mix";
  hex_dir = "${state_dir}/hex";

And then I set them like this:

    env.ERL_AFLAGS = "-kernel shell_history enabled -kernel shell_history_path '\"${state_dir}/erlang-history\"'";

    tasks."elixir:setup" = {
      exec = ''
        mkdir -p ${mix_dir}
        mkdir -p ${hex_dir}

        export PATH=${mix_dir}/bin:$PATH
        export PATH=${mix_dir}/escripts:$PATH
        export PATH=${hex_dir}/bin:$PATH
      '';
    };

    env.MIX_HOME = mix_dir;
    env.HEX_HOME = hex_dir;

    env.ERL_LIBS = "${hex_dir}/lib/erlang/lib";

For postgres I set this:

    env.PSQL_HISTORY = "${state_dir}/psql_history";

For node, these ones:

    env.NPM_CONFIG_CACHE = "${state_dir}/npm";
    env.NODE_REPL_HISTORY = "${state_dir}/node_repl_history";

And, if you plan to add support for rust, I set this:

    env.CARGO_HOME = "${state_dir}/cargo";
5

Thank you for the inspiration! I was considering isolating Hex archives, actually.

1 Like
6

This is pretty neat! I’ve been planning on getting into nix & devenv at some point, especially considering how important it is getting to be able to run containerized local development. Its on my short list to checkout :person_bowing:

1 Like
7

Why would you want nix to run containerized stuff?

(Genuine question, the benefits seem overlapping)

8

It allows you to have full development dependencies per project and scoped only for them. It gives you access to basically all packages from Nix so you can setup it the way you want and do that in a declarative and consistent way.

IMO it is the best way to setup you development infrastructure without having to handle that at the OS level, specially if you work with multiple projects with different dependencies.

The only downside, IMO is that the nix language is kinda hard to work with if you don’t want to spend a lot of time understanding it.

PS. i’m talking here about using devenv, NixOS and Home Manager are not required to work with devenv at all (but I do like them a lot too)

1 Like
9

I know what nix does, what is confusing to me is how zach is going to get benefit from nix AND containers at the same time. Unless he meant «comtainerized» abstractly achieved via nix and not oci combined with nix :laughing:

1 Like
10

Sorry, I kind of conflated a lot of things in one post, but my real point is that there are a whole host of tools, like nix and devenv, that I’ve been meaning to look into, specifically for agentic workflows (which may involve containerization, repeatable builds, sandboxing etc. TBD).

11

Container from “to contain”, Docker, LXC etc. offer containers, Devenv achieves “containerisation” in a different way, without strict file system boundaries.

That being said, Nix in general has good support for bundling a config in an OCI container, see here for Devenv.

Devenv is a good way to get a feeling for Nix: You can treat your devenv.nix as a config data file without knowing the Nix language, some of the more complicated stuff can be setup via devenv.yaml, like adding additional package sources.

Without Devenv you would typically create a Flake providing a dev shell – for me that typically meant looking for and starting with a template. Always felt a bit cumbersome when I just required an ad-hoc environment to play around with some piece of software written in obscure language xy…

12

I’ve been using nix for the past year for my dev environment and our server setup.
Does anyone here have first hand experience with what benefit devenv offers on top of nix?

Project looks neat!

13

Have any of you tried making flakes with claude code? The combination of verbose syntax with what seems to be a bit trial and error as a learning path seems to be a nice fit for letting an agent loose on the docs and a template as prior art, let it spin a while and see what comes out the other end.

I have had good success making oci stuff with CC at least. It made me a nice test psql container for github that had both postgis and pgvector in it. It took a few unattended iterations, but it worked well.

Been meaning to look at nix more closely, but it seems so big to dip toe in.

14

devenv provides community-driven abstractions for developer environments—much like NixOS does for system configuration, and home-manager does for user configuration of the home directory.

You could just take a Nix shell, and define your PostgreSQL configuration with various ENV vars (PGDATA, …) to make a PostgreSQL installation “local”, but with devenv as a minimum you just need to declare:

  services.postgres = {
    enable = true;
  }

In addition to the above, devenv also provides abstractions for:

  • background process management
  • composable tasks (kind of make replacement)
  • building OCI containers
  • … and more

In my company we have been using devenv for developer environments for around 2 years now. We have employees on Linux, Windows WSL and MacOS. Each time a new OS came up, there were some env specifics that had to be resolved / improved, but the problems were not devenv related per se. Overall the experience has been overwhelmingly positive, with new employees being able to go from 0 to running a monolith with a large number of binary dependencies and source modified PostgreSQL plugins in minutes.

This is how the setup instructions roughly look like for a new MacOS user with a brand-new laptop:

  • Install direnv and setup shell hooks
  • Clone repo
  • Run a few first time setup scripts to get Nix installed and setup
  • direnv allow to allow automatic env var loading / unloading in the directory of the repo
  • devenv up to start background processes
  • make ci to confirm the application passes various static analyses and tests
  • Install Zed (as Zed requires no plugin setup - Zed supports elixir-ls and direnv out of the box. We install elixir-ls via devenv so that all devs have identical version of the language server)
  • Write prod code
3 Likes
15

Nix is fantastic for Docker image building, see:

With nix, you can generate Docker containers that:

  • are built deterministically
  • can be smaller
  • can build faster
  • are more composable (well, the source code written in nix is more composable, which allows you to be more flexible in how you generate Docker containers. No Dockerfile can merge multiple base images into a single layer, but you can achieve this with nix.)
16

Getting to this level sounds like several hundred hours of work tho :sweat_smile:

1 Like
17

It might feel like a lot of work, but in general, the amount of time spent is dependent on:

  • prior knowledge.
  • environment complexity. For a greenfield project, it could be 2 mins of LLM generated devenv config.

Purely on devenv configuration itself, I’ve spent at most a few full-time work days in the last two years, as lot of the time gets spent on things you’d have to do anyway, regardless of devenv:

  • PostgreSQL config shows up in devenv, and it’s a big chunk by size, but you’d need that setup no matter how you manage your dev environments.
  • If I spend time packaging something with nix, that time would still have to be spent even if there was no nix - trying to get everyone on the same package.
  • …

Nix is one of those technologies that have a large learning curve. But that’s the same with many other technologies that are worth learning - you are not going to become proficient in Haskell, Clojure, Prolog, … in a weekend. What I can say, having learned a bunch of Nix/NixOS before the advent of LLMs, is that LLMs make it way faster to develop the fundamental understanding of the Nix ecosystem. But one has to be careful with prompting as the purpose of prompting should be building understanding as opposed to hoping that LLMs can generate some black-box that works.

4 Likes
18

Thanks I will give it a try!

At work we are using Nix both for our dev environment and setting up our infrastructure for our server. It has been great but also a lot of manual setup for the development environment.

1 Like