Ensuring drivers are loaded before application boots

Nerves Forum Nerves Questions / Help
1

I’m building a custom Nerves system for a CM4-based board and running into a timing issue where the main Elixir application starts before all kernel drivers and hardware are fully initialized.

My firmware uses NervesTime with an RTC (PCF8563) on an I2C bus that’s created by a device tree overlay. During boot, NervesTime tries to initialize the RTC, but the I2C bus doesn’t exist yet:

22:43:22.173 [info] i2c i2c-22: Added multiplexed i2c bus 10

22:43:22.218 [debug] [NervesTime] starting /usr/sbin/ntpd with: ["-n", "-S", "/srv/erlang/lib/nerves_time-0.4.9/priv/ntpd_script", "-p", "0.pool.ntp.org", "-p", "1.pool.ntp.org", "-p", "2.pool.ntp.org", "-p", "3.pool.ntp.org"]

22:43:22.237 [error] [NervesTime] Cannot initialize rtc '{NervesTime.RTC.NXP.PCF8563, [bus_name: "i2c-10", address: 81]}': :bus_not_found

When I try to initialize the RTC later via iex, it works with no issues:

iex(2)> {:ok, ref} = NervesTime.RTC.NXP.PCF8563.init(bus_name: "i2c-10", address: 0x51)
{:ok,
 %{
   address: 81,
   bus_name: "i2c-10",
   i2c: %Circuits.I2C.I2CDev{
     ref: #Reference<0.3964485155.1160904719.209822>,
     retries: 0,
     flags: [:supports_empty_write]
   }
 }}

I am considering adding an erlinit pre-run script to explicitly wait for the the hardware to be fully initialized, but am curious if there is a recommended Nerves pattern for waiting on hardware initialization? Ideally this would be baked into the Nerves system.

1 Like
2

Hi @hikabab,

You’re running into a common issue where libraries assume that hardware exists on initialization when it really shouldn’t. I make this mistake too and I’m on the maintainers list for that library, so I’m not blaming those involved. The usual answer is that the library needs to either retry or hook into the proper notification for when its requirements are met. I prefer retrying for these things since it’s simpler. Retries can also recover from I2C issues like the bus being temporarily hung.

If you’re looking for a pragmatic answer that does involve changing that library, then I think you have it. For other things, I’d suggest adding init code to an Application.start callback that gets run before the code that needs it. However, an RTC is special and affects the clock, so you’ll benefit by getting it initialized as early as possible.

If you already have a custom Nerves system, then the way I’d solve the problem is to use the Linux device driver for the PCF8563. If it’s a built-in device driver and you’re loading the overlay via the config.txt, then I’d expect system time to get set by the RTC before any Elixir code runs. I know others who prefer handling the RTC in Elixir, so if you’re interested in updating NervesTime.RTC.NXP.PCF8563 to retry and are open to sending PRs, then I’d work with the other maintainers to get that merged.

1 Like
3

I actually did explore having Linux manage the RTC and ntpd directly, but that seemed contrary to the “Nerves philosophy” of handling as much as possible in Elixir rather than relying on traditional Linux tools.

I noticed that nerves_time is included as a dependency in nerves_hub_link, and there was a PR (now closed) to check for time synchronization before attempting to connect. That made me think the intended direction was to have Elixir manage the RTC.

I’m curious about the broader architectural direction here: should libraries like nerves_time become more resilient with retries/timeouts to handle these timing issues? Or is it actually fine (or even preferable) to delegate responsibilities like RTC management to Linux and keep the Nerves system more self-contained?

4

You can configure nerves_time to wait if time is really the only blocker: Add process to await system time adjustment up to configurable limit by LostKobrakai · Pull Request #98 · nerves-time/nerves_time · GitHub

1 Like
5

When using the mostly-for-development SharedSecret method with NervesHub the time can really screw with the signing of the secret so that’s why that was considered. I’m not sure we ever merged that. It was also about this time I noticed just how long it can be between OS time getting NTP sync and Erlang actually warping time to match. Fun stuff. Not painful.. at .. all.

I wouldn’t necessarily consider nerves_hub_link to be a reference for the Nerves philosophy or best practice. It is well proven but has been worked on for a long time by many hands and contains more than one philosophy :smiley:

It very much tries and retries the way that Frank is talking about. Because internet access is not a guarantee. Access to the hardware crypto chip is hopefully stable but if it blips, best try again later.

I think improving the RTC library sounds good. As that benefits us all.

But for something like an RTC using the Linux kernel driver is reasonably pragmatic as well. There is no purity :slight_smile: always a balancing act. You will se a mix in most production Nerves projects.