Nerves development environment with Docker (and VS Code)

How do you feel that you can acquire the development environment for Nerves without affecting the host environment?:thinking: The Docker can do it with minimal steps!!:tada::tada::tada:

We want to offer the Docker solution mainly for beginners and Windows users.

Dockerfile:

Docker Hub (pre-built image)
https://hub.docker.com/r/nervesjp/nerves

VS Code dev-container:

Main contributions and use cases

  • If you are new to Nerves, you can easily try out its development without any affect on your host PC.
  • It will be convenient for hands-on training to provide a unified environment for all participants.
  • Even if you are already using Nerves, you can quickly try out the latest development environment. Docker does not affect your host environment.
  • The Docker Hub repository here publishes the pre-built image. You can try it right away by just pulling it (but it may not always be up to date).
  • You can also experience a better development environment by using the Visual Studio Code and its extension.

How to use with Docker alone

You need to install Docker Desktop
Docker has some requirements for host PC to be operated. See details: Windows / macOS

from Docker Hub repository

You can try the Nerves development with pre-built image.

$ docker pull nervesjp/nerves
$ docker run -it -w /workspace nervesjp/nerves
root@6e304327bd2e:/workspace# 

from GitHub repository

You can build Docker image locally, and customize it to your needs.

$ git clone https://github.com/NervesJP/docker-nerves
$ cd docker-nerves

$ docker build -t docker-nerves .
$ docker run -it -w /workspace docker-nerves 
root@9bc88d0fc7b8:/workspace# pwd
/workspace
root@9bc88d0fc7b8:/workspace# echo ${HOME}
/root
root@9bc88d0fc7b8:/workspace# ls ~/.mix/*
/root/.mix/rebar  /root/.mix/rebar3

/root/.mix/archives:
hex-0.20.6  nerves_bootstrap-1.10.0

Tips: Mount volumes on host

Since a filesystem into Docker image will disappear when an image rebuild/execute, it is useful to mount a volume on host to keep files of Nerves project. -v ${PWD}:/workspace can mount current directory on host to Docker image.

$ docker run -it -w /workspace -v ${PWD}:/workspace docker-nerves 

It is also efficient to mount Nerves related setting files, such as ~/.ssh/ and ~/.nerves. Following is an example to share setting files between host and image.

$ docker run -it -w /workspace -v ${PWD}:/workspace  -v ~/.ssh:/root/.ssh -v ~/.nerves:/root/.nerves docker-nerves 

More convenient with VS Code

Visual Studio Code is one of the most popular editor for engineers.
Remote - Containers extensions provides an awesome experience with Docker development containers.

See details:

How to use

  1. Clone the repository
git clone https://github.com/NervesJP/nerves-devcontainer
  1. Start VS Code, run the Remote-Containers: Open Folder in Container… command from the Command Palette (F1) or quick actions Status bar item, and select the project folder you cloned at Step 1.

  2. After a while, you can enjoy Nerves/Elixir development with Bash on Docker image and VS Code like as follows.

root@ebc5e1a19ae3:/workspaces/nerves-devcontainer# ls 
LICENSE.txt  README.md
root@ebc5e1a19ae3:/workspaces/nerves-devcontainer# ls ~/.mix/*
/root/.mix/rebar  /root/.mix/rebar3

/root/.mix/archives:
hex-0.20.6  nerves_bootstrap-1.10.0
root@ebc5e1a19ae3:/workspaces/nerves-devcontainer# elixir --version 
Erlang/OTP 23 [erts-11.1.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Elixir 1.11.2 (compiled with Erlang/OTP 23)
root@ebc5e1a19ae3:/workspaces/nerves-devcontainer# iex 
Erlang/OTP 23 [erts-11.1.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Interactive Elixir (1.11.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> 
  1. Note that you need to generate your SSH keys, only for the first time to login Docker image.
root@ebc5e1a19ae3:/workspaces/nerves-devcontainer# ssh-keygen -t rsa -N "" -f .ssh/id_rsa`

Tips

Manual build of Docker image locally

This repository will clone pre-built image on Docker Hub.

You can also build Docker image locally by locating above Dockerfile to ./devcontainer and uncommenting //"dockerFile": "Dockerfile", on ./devcontainer/devcontainer.json.
It means you can customize your own Nerves development environment as you want.

Mounted files about Nerves settings

Each time an image is execute, a filesystem into Docker image will disappear. So Nerves related setting directories are mounted in the current (this) directory on the host.

First directory is ${PWD}/.ssh that is expected to contain id_rsa and id_rsa.pub SSH key pairs.
They are used for the construction of Nerves firmware and secure connection to Nerves devices.
It is mounted to ~/.ssh/ on Docker image.
To generate your SSH keys, you need to operate ssh-keygen -t rsa -N "" -f /workspaces/nerves-devcontainer/.ssh/id_rsa after logging into Docker image for the first time.

Second is ${PWD}./nerves that is mounted to ~/.nerves. It keeps archives of Nerves toolchains and nerves_system_br.
If it disappeared when Docker container was restarted, you need to download archives every time you operate mix deps.get.
So we decided to mount it on this directory.
Although mix deps.get at first will take a while to unzip archives to ./nerves/artifacts due to poor performance about bindings between host and container filesystems, we think it is acceptable compared that we always have to wait when running mix deps.get for a long time.

In addition, your Nerves project can be kept on the current directory by the VS Code automatic feature.

Current limitation(s)

burn Nerves firmware to microSD card

Docker has restrict policies to avoid effecting host environment. Therefore, mix burn cannot be operated from Docker image because there is no right to access /dev to on host as a root user.

One way to burn Nerves firmware is just operating fwup on the host. fwup is an utility for constructing/burning Nerves firmware.

After installing fwup on the host according to this step, please do following command on the host terminal (e.g., PowerShell as Administrator, Terminal.app).

$ cd <your_nerves_project_dir>
$ fwup _build/${MIX_TARGET}_dev/nerves/images/<project_name>.fw

Please let us know if you have a cool solution! (issue)

Any questions, suggestions and comments (including English quality :smile: ) are welcome!!

14 Likes

About the VS Code dev-container, I decided to add .hex/ as the bind directory to keep the Hex packages on the host.

Maybe it seems that I can no longer modify the original post. So please check README on GitHub for the latest updates.


1 Like

Hi, thanks so much for this write up!

I wanted to share a similar setup I have developed for Nerves development in docker-compose:

In this repo, I’ve included a volume mapping of /dev/sdb in the docker-compose config along with the privileged option to ensure accessibility for writing to the the microSD card. You may need to use a different /dev/ address for your purposes.

In order to allow for updating the project over the air, I’ve used a script that copies ssh keys from the home folder on the host into the container and fixes permissions so that they’re ready for use with fwup inside the container.

I tried to include a bare bones description of how to use this repo in the readme, but it could certainly use some improvement. Please feel free to ask clarifying questions or comment on what I’ve got in the repo.

2 Likes

Jason, Thanks for sharing your knowledge!!
I wanna refer to your contribution in my repository :smile:

I found a volume mapping of /dev/sdX works well only on Linux as the host. Both on Windows and macOS could not pass through a device to a container as it requires support at the hypervisor level (See: https://docs.docker.com/docker-for-mac/faqs/#can-i-pass-through-a-usb-device-to-a-container)

I believe that technology will evolve well🙏

Both on Windows and macOS could not pass through a device to a container as it requires support at the hypervisor level

That’s a major bummer. I had initially created my docker-compose nerves starter to work with my nephews (who have a Mac) in an effort to make it easier for them to get up and running. I think I may go back to trying nix package manager and nix-shell to achieve a starter repo that works for everyone.

I got pretty far along there too, but kept running into issues with fwup.

I wanna refer to your contribution in my repository

Sure, thanks! I would love to improve the on boarding process for new developers by providing even easier tooling to get started. Nerves has so much going for it already, it’s exciting to see more work emerging that reduces the barrier to entry. Cheers!

1 Like

Are you able to share your nix-shell?

@elay1 I did have a nix-shell working at one point, but unfortunately, newer versions of nerves required a newer version of fwup, that did not work well with nix (failing tests prevented compilation). I ended up going back to docker-compose for nerves work. If you do get something working with nix-shell, I would definitely be interested to hear about it.

@elay1 Good news, I gave nix-shell another try for nerves development. This time it worked without a hitch. I plan to publish the repo shortly so you can give it a try.

1 Like

Here’s the starter repo I just created for nix-shell:
https://github.com/jasonmj/nerves-nix-shell

3 Likes

Hello Jason, Just got round to trying this. Are you running this on an M1 mac?
I am running this on a 2015 mac with Catalina.
I installed nix with

sh <(curl -L https://nixos.org/nix/install) --darwin-use-unencrypted-nix-store-volume --daemon

I am also cross referencing instructions from nix docs and this detailed blog https://www.mathiaspolligkeit.com/dev/exploring-nix-on-macos/ and https://ejpcmac.net/blog/about-using-nix-in-my-development-workflow/ (Thankyou to those blogs))
then copied your shell.nix contents to my home shell.nix file.

$ nix --version
nix (Nix) 2.6.0

but I got this error…

$ nix-shell
error: Package ‘lxqt-openssh-askpass-1.0.0’ in /nix/store/ryk5y6b34vxmzp4v9cgr8aga9g2icxmh-nixpkgs/nixpkgs/pkgs/desktops/lxqt/lxqt-openssh-askpass/default.nix:46 is not supported on ‘x86_64-darwin’, refusing to evaluate.

       a) To temporarily allow packages that are unsupported for this system, you can use an environment variable
          for a single invocation of the nix tools.

            $ export NIXPKGS_ALLOW_UNSUPPORTED_SYSTEM=1

       b) For `nixos-rebuild` you can set
         { nixpkgs.config.allowUnsupportedSystem = true; }
       in configuration.nix to override this.

       c) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
         { allowUnsupportedSystem = true; }
       to ~/.config/nixpkgs/config.nix.
(use '--show-trace' to show detailed location information)
16:06:01~

I was going to set option c but just wondered what you thought of the problem…
Still playing with both nerves in my local home/projects directory and played with Nix a few years ago.

Nix documentation has improved a lot but its still a lot to get your head around.
I did a search on NixOS Discourse for elixir and found this conversation…

https://discourse.nixos.org/t/whats-the-current-best-in-class-approach-to-packaging-elixir-erlang-beam-applications-using-nix-releases-as-of-july-2020/8205/3

Some good links in there too.

@elay1 Ah, yeah… I am not on MacOS. I’m using NixOS on a Thinkpad.

I used lxqt-openssh-askpass as a way of prompting for authentication before burning firmware to an SD card. You might want to test and see if MacOS can handle this for you by simply excluding that package from the list.

That’s worked! I am surprised we have the latest elixir version available. NixOS used to always be behind on elixir versions. Thanks - this has got me on the path again to experimenting with nix with elixir and nerves.

[nix-shell:~]$ elixir -v
Erlang/OTP 24 [erts-12.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit]

Elixir 1.13.2 (compiled with Erlang/OTP 24)

[nix-shell:~]$ iex
Erlang/OTP 24 [erts-12.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit]

Interactive Elixir (1.13.2) - press Ctrl+C to exit (type h() ENTER for help)
iex 14:43 -1 >

I manually installed

mix local.hex --force
mix local.rebar --force
mix archive.install --force hex nerves_bootstrap

$ mix nerves.info (edited output)
Nerves:           1.7.12
Nerves Bootstrap: 1.10.5
Elixir:           1.12.3
|nerves_bootstrap| Info End

And then tested burning nerves project to sdcard following getting started instructions.
https://hexdocs.pm/nerves/getting-started.html

$ mix firmware.burn

I got prompted for password by macos
thanks

1 Like