Best way to pull new firmware OTA into remote devices

Hi there :slight_smile:
I was wondering, what Nerves experts would suggest as a solution for OTA firmware updates to a fleet of remote devices. Of course there is nerves_firmware_ssh (and nerves_firmware_http), but it assumes one can SSH (or HTTP PUT) to the device, which means the device needs to have a known address. That’s easy on local networks, but harder when the device is deployed remotely on other networks.

What I think the best way would be, is deploying a release server and letting all the devices check periodically for updates, securely downloading them, and then updating themselves. This seems the most reasonable option to me, but I didn’t find a package for pulling a new firmware (only pushing it via SSH or HTTP). I can write something like that, but I wanted to ask before I re-invent the wheel.

What’s your opinion?

3 Likes

Definitely hop over to the #Nerves Slack channel, https://elixir-lang.slack.com. We are currently working on a server platform for deploying firmware updates. Currently most people have a Phoenix Channel or MQTT channel open to their server, and push update notifications. That push notification will usually have a secure URL in which to download the updated firmware.

The platform that is currently being built will work similar to this as well, but will be open source and built by the Nerves community.

3 Likes

That’s amazing news @entone, thanks! Joining the Slack channel now

Is there a repo for this project?

So, I ended up creating a simple update manager GenServer that subscribes on an MQTT topic and downloads/validate/apply firmware updates when notified. It works very well, even though it’s quite specific to my use case. I’d be happy to switch to the open-source solution mentioned here when possible, and possibly contribute to it.

Any update on this project?

I have my check with server portion written, to the point where I’m ready to download and apply the update. How do you actually go about doing this?

Any update on a firmware update server platform?

Replying myself : I stumbled upon a blog post about NervesHub. It’s not ready for production, but it seems that there is serious work under progress :slight_smile:

2 Likes

Thanks for pinging us on NervesHub. It’s technically private beta right now and we have a couple companies regularly using it in production. At the moment, our focus is on improving device authentication and first-time device registration. That has turned out to be more work than expected so we have been quieter on announcing progress. I expect that we’ll have something working fairly well by end-of-year, but with very rough docs.

If you’re building custom hardware right now and anticipate using NervesHub, I’d add an ATECC508A (~60-70 cent adder) to your board. That chip securely stores the private key used for authentication and locks down a couple other important pieces of device information in one-time-programmable memory. The chip can also be used with Amazon IoT, etc., so you’re not locked into NervesHub.

6 Likes

Hi @fhunleth,
is it in the plans to eventually open NervesHub publicly? I would definitely be interested in becoming a customer with my company, also in the private beta phase if feasible.

I have two questions, also for @fhunleth.

The first is very similar to @lucaong’s question: How stable would you say that NervesHub is now, seven months after your last status update?

As for the second: What are the current SSL-settings that NervesHub uses by default? Is it using the sane secure defaults outlined in @voltone 's SSL client shootout (which has also been distilled into this talk)?

1 Like

Hey - I somehow missed these, so sorry for the late replies.

Here’s the status of NervesHub:

  1. It’s currently in production use by several companies and has been working really well for the past year. There are details since we don’t have all of the features that all of us want for large numbers of devices, but it is possible to work around.
  2. The main programmatic APIs to NervesHub are pretty stable, but we’ve made some large changes to the UI this summer and we plan a few more large ones to improve usability.
  3. Our docs are slowly getting better at https://docs.nerves-hub.org/.
  4. We recently added some opt-in operational support features - like remote login. I.e., click on the device and get an IEx prompt on it.
  5. Work to support billing is in progress. This won’t affect low volume hobbyist/maker use. We have funding to support that use case. This is intended for companies who would prefer to not run their own instance of NervesHub so that we can sustain the service.

One could say that NervesHub is public already - every line of code used by it is open-source. It’s a fairly large Apache 2-licensed Phoenix/LiveView project. Our hosted NervesHub lets anyone register an account (albeit via using mix commands rather than via the web UI). The plan is still to remove the “beta” tag and make our hosted NervesHub look public.

I’d say that if you want to use NervesHub for a commercial device and are comfortable with Elixir tooling that there’s no reason to wait. If your deployment has special needs from the firmware update server that aren’t supported, please check with us. Our hosted NervesHub enforces quotas to keep our costs in line. We’ve been bumping quotas pretty liberally for those who have asked, and once billing is available, you’ll not have to talk with us.

If you’re at a company that is uncomfortable using our hosted NervesHub for firmware updates (no offense taken - my company is one of these), you can setup NervesHub on your own AWS servers. I’m not an AWS person and this doesn’t seem easy to me, but I’m told that if you’re familiar with AWS that it’s ok. We still need to add how to do this to our docs.

Finally, with regard to the security question and SSL settings, both Justin Schneck and I talk with voltone periodically regarding security. I’m sure that we can do better and we’re due for another review. Please DM us or email support@nerves-hub.com, or if it’s proactive/best practice kind of thing, then PRs/GitHub issues would be most appreciated.

6 Likes

Awesome! That sounds great. Thanks for the answer and for the outstanding work on Nerves and its ecosystem!

3 Likes

There is any update on access to nerves hub servers, or any advance with the documentation to deploy into AWS?

We are currently working on additional documentation for this. You can find the PR here: https://github.com/nerves-hub/documentation/pull/6

4 Likes