Tz - time zone support for Elixir (alternative to Tzdata that comes with a lot of bugfixes)

Hello

Allow me to introduce you to Tz, an alternative time zone database support to Tzdata.

Why another library?

First and foremost, it comes with a lot of bugfixes. At its current state, Tzdata has many bugs, some of which have already been reported for some time, but left unfixed for the moment.

The Tz library has been tested against nearly 10 million past dates, which includes most of all possible imaginable edge cases. You will find below 10 random examples of bugs using Tzdata, that my tests allow me to detect.

Time zone periods are computed and made available in Elixir maps during compilation time (by “period” it is meant a period of time where a certain offset is observed, for example from March 31 until October 27 2019 clock went forward by 1 hour in Belgium).

I would like to reduce the compilation time as it currently takes over 15 seconds. For example, Tzdata ships with a dump of the computed periods in an ETS file; consequently the periods no longer have to be computed when compiling the dependency. However I do not want to use ETS for querying, but the idea of a dump to avoid all the period computations is interesting.
As I’m writing this post, I realized that the compilation time just got much faster; no idea what happened though and honestly I do not understand what really takes time during the compilation process; I reduced the data inside the maps that represent the periods, could it be that? It seems then that it might not be the computations that take the most time, but rather related to the size of the maps.

The period lookup can be optimized. Note however that without any kind of optimization yet written for Tz, querying the periods and writing the result into a file for 9.866.112 million dates takes around 6.5 minutes, whereas Tzdata takes almost 10 minutes (same code is used for querying, using the DateTime.from_naive/3 and DateTime.shift_zone/3 functions once with Tz.TimeZoneDatabase, once with Tzdata.TimeZoneDatabase). Note that in Java it takes less than 15 seconds to generate and write these nearly 10 millions dates into a file with a similar code logic… how do you think that’s manageable?
Tzdata comes however with dynamic tz data updates; I have no plans to integrate that in Tz (for every iana tz database update, the Tz dependency will have to be updated). In order to keep your time zone database updated, you can “watch” the project on github for releases and I also plan to provide with an optional small utility that logs on your server when a new iana tz database update is detected.

If you happen to be part of a profitable company relying on time zone support in Elixir, and the company wouldn’t mind supporting a little for continuous work, I have set up GitHub Sponsors for this particular project as I have been working on it for a long time, full-time, without a source of income; just in case I’d have saved you months of work. and there’s still a lot of work to be done:

• the library is tested against nearly 10 million dates; this code for testing is currently in a private separate package that needs to be reworked and open-sourced;
• the library lacks code documentation for now;
• I’d like to do some continuous refactoring and renaming;
• decrease the compilation time;
• decrease the tz periods lookup time;
• provide different utilities (in separate packages; I want to keep Tz minimal to provide the time zone support for Elixir’s DateTime module) to extract other useful data from the iana tz database, watch for iana tz database updates, etc.
• …

Bugfixes

Here are 10 example bugs with Tzdata that my testing code detects:

Example bug 1:

DateTime.from_naive(~N[1912-01-01 00:00:00], "Africa/Abidjan", Tzdata.TimeZoneDatabase)

** (UndefinedFunctionError) function nil.utc_off/0 is undefined

Bug has been reported here: https://github.com/lau/tzdata/issues/90

Using Tz:

DateTime.from_naive(~N[1912-01-01 00:00:00], "Africa/Abidjan", Tz.TimeZoneDatabase)

{:gap, #DateTime<1911-12-31 23:59:59.999999-00:16 LMT Africa/Abidjan>,
#DateTime<1912-01-01 00:16:08+00:00 GMT Africa/Abidjan>}

Example bug 2:

DateTime.from_naive(~N[1920-09-01 00:00:00], "Africa/Accra", Tzdata.TimeZoneDatabase)

{:gap, #DateTime<1917-12-31 23:59:59.999999-00:00 LMT Africa/Accra>,
 #DateTime<1920-09-01 00:20:00+00:20 +0020 Africa/Accra>}

The documentation says

When there is a gap in wall time - for instance in spring when the clocks are turned forward - the latest valid datetime just before the gap and the first valid datetime just after the gap.

But the first date returned by Tzdata happens nearly 3 years earlier, that’s definitely not the “latest valid datetime just before the gap”.

Using Tz:

DateTime.from_naive(~N[1920-09-01 00:00:00], "Africa/Accra", Tz.TimeZoneDatabase)

{:gap, #DateTime<1920-08-31 23:59:59.999999+00:00 GMT Africa/Accra>,
 #DateTime<1920-09-01 00:20:00+00:20 +0020 Africa/Accra>}

Example bug 3:

DateTime.from_naive(~N[1891-03-15 00:00:00], "Africa/Algiers", Tzdata.TimeZoneDatabase)

{:ambiguous, #DateTime<1891-03-15 00:00:00+00:09 PMT Africa/Algiers>,
 #DateTime<1891-03-15 00:00:00+00:12 LMT Africa/Algiers>}

According to tzdata, “1891-03-15 00:00:00+00:09 PMT” happens first, but that is wrong;
“1891-03-15 00:00:00+00:12 LMT” happens first, then clock went backwards and the time zone abbreviation changes from LMT to PMT.

Using Tz:

DateTime.from_naive(~N[1891-03-15 00:00:00], "Africa/Algiers", Tz.TimeZoneDatabase)

{:ambiguous, #DateTime<1891-03-15 00:00:00+00:12 LMT Africa/Algiers>,
 #DateTime<1891-03-15 00:00:00+00:09 PMT Africa/Algiers>}

Example bug 4:

DateTime.from_naive(~N[1977-05-06 01:00:00], "Africa/Algiers", Tzdata.TimeZoneDatabase)

{:ambiguous, #DateTime<1977-05-06 01:00:00+02:00 CEST Africa/Algiers>,
 #DateTime<1977-05-06 01:00:00+01:00 WEST Africa/Algiers>}

For tzdata, 1977-05-06 01:00:00 at Africa/Algiers is ambiguous;
however, a DST change happened at 1977-05-06 00:00:00, where clock jumped for 1 hour (gap between 00:00 and 01:00); so from 01:00 there are no ambiguous dates or gaps.

Using Tz:

DateTime.from_naive(~N[1977-05-06 01:00:00], "Africa/Algiers", Tz.TimeZoneDatabase)

{:ok, #DateTime<1977-05-06 01:00:00+01:00 WEST Africa/Algiers>}

Example bug 5:

DateTime.from_naive(~N[2062-01-07 00:00:00], "Africa/Casablanca", Tzdata.TimeZoneDatabase)

** (RuntimeError) dynamic periods assume 2 rules per year

Using Tz:

DateTime.from_naive(~N[2062-01-07 00:00:00], "Africa/Casablanca", Tz.TimeZoneDatabase)

{:ok, #DateTime<2062-01-07 00:00:00+01:00 +01 Africa/Casablanca>}

Example bug 6:

DateTime.from_naive(~N[2064-01-20 02:00:00], "Africa/Casablanca", Tzdata.TimeZoneDatabase)

** (MatchError) no match of right hand side value: :min

Using Tz:

DateTime.from_naive(~N[2064-01-20 02:00:00], "Africa/Casablanca", Tz.TimeZoneDatabase)

{:gap, #DateTime<2064-01-20 01:59:59.999999+00:00 +00 Africa/Casablanca>,
 #DateTime<2064-01-20 03:00:00+01:00 +01 Africa/Casablanca>}

Example bug 7:

DateTime.from_naive(~N[2013-10-25 01:00:00], "Africa/Tripoli", Tzdata.TimeZoneDatabase)

{:ambiguous, #DateTime<2013-10-25 01:00:00+02:00 CEST Africa/Tripoli>,
 #DateTime<2013-10-25 01:00:00+01:00 CET Africa/Tripoli>}

There was no DST change on 2013-10-25 01:00:00 at Tripoli.

Using Tz:

DateTime.from_naive(~N[2013-10-25T01:00:00], "Africa/Tripoli", Tz.TimeZoneDatabase)

{:ok, #DateTime<2013-10-25 01:00:00+02:00 CEST Africa/Tripoli>}

Example bug 8:

DateTime.from_naive(~N[2013-10-25 02:00:00], "Africa/Tripoli", Tzdata.TimeZoneDatabase)

{:gap, #DateTime<2013-03-29 00:59:59.999999+01:00 CET Africa/Tripoli>,
 #DateTime<2013-10-25 03:00:00+02:00 EET Africa/Tripoli>}

This one is tricky. There was a DST change according to the following iana rule:

Rule | Libya | 2013 | only | - | Oct | lastFri | 2:00 | 0 | -

the local offset from standard time changed from 1 hour to 0.

However, the standard offset from UTC time changed as well: one hour was added.
That leads to a total offset difference of 0. Hence, there is no gap.

Using Tz:

DateTime.from_naive(~N[2013-10-25 02:00:00], "Africa/Tripoli", Tz.TimeZoneDatabase)

{:ok, #DateTime<2013-10-25 02:00:00+02:00 EET Africa/Tripoli>}

Example bug 9:

DateTime.from_naive(~N[1941-04-18 23:00:00], "Europe/Belgrade", Tzdata.TimeZoneDatabase)

{:ok, #DateTime<1941-04-18 23:00:00+01:00 CET Europe/Belgrade>}

There is a gap at that time. The local offset from standard time moved from 0 to 1 hour.

Using Tz:

DateTime.from_naive(~N[1941-04-18 23:00:00], "Europe/Belgrade", Tz.TimeZoneDatabase)

{:gap, #DateTime<1941-04-18 22:59:59.999999+01:00 CET Europe/Belgrade>,
 #DateTime<1941-04-19 00:00:00+02:00 CEST Europe/Belgrade>}

Example bug 10:

DateTime.shift_zone(~U[2010-03-27 14:00:00Z], "Asia/Kamchatka", Tzdata.TimeZoneDatabase)

{:ok, #DateTime<2010-03-28 01:00:00+11:00 +11 Asia/Kamchatka>}

According to iana’s records, the standard offset from the UTC time changed from 12 hours to 11 hours at
2010-03-28 02:00 standard time, which was 2010-03-28 14:00 UTC time. That’s why Tzdata shows +11 above.
However, there is another rule that says, at 2010-03-28 02:00 standard time, the local offset from the standard time changed from 0 to 1 hour. So all in all, it is not +11 but should stay at +12.

Using Tz:

DateTime.shift_zone(~U[2010-03-27 14:00:00Z], "Asia/Kamchatka", Tz.TimeZoneDatabase)

{:ok, #DateTime<2010-03-28 02:00:00+12:00 +12 Asia/Kamchatka>}

https://github.com/mathieuprog/tz