jj1bdx

jj1bdx

Primary resources for leap seconds:

I think having an API to check out how the operating system handles the leap seconds will be useful, such as accessing the tzdata or timezone database (and the current timezone environment).
OTOH, having something else than the operating system definition might be implementing the same things twice with different interpretation and might be troublesome.

Also, UTC is affected by the leap seconds, and not absolute. If you want a time sequence of monotonic increase in the same pace at all times without any human society intervention, TAI should be a choice. OTOH, the operating system clocks of most computers in the world are synchronized with UTC.

1
Post #13
jj1bdx

jj1bdx

On FreeBSD, tzdata should be recompiled if the operating system configuration of being aware of handling or not handling leap seconds is changed.
So tzdata is not orthogonal to the timezone database, and rather tzdata is dependent on the leap second handling definition in the operating system.

1
Post #14
josevalim

josevalim

Creator of Elixir

This is interesting. However, we already keep the offsets in the DateTime struct in ISO seconds and we wouldn’t be able to change it due to backwards compatibility. Thoughts?

2
Post #15
michalmuskala

michalmuskala OP

I would say that this will be up to the concrete time zone info provider library. In this way it’s outside the scope of the proposal. One possible implementation I can imagine is reading the time zone info from the operating system.
That said, I agree that the leap second issue is parallel, but closely related to time zones, so the proposals could try to address both of them.

0
Post #16
Lau

Lau

I am looking forward to having this behaviour in the Elixir standard library - something that has been planned for a long time, and I think there was even talk about having it in Elixir version 1.4 or 1.5.

This sounds like it is based on a misunderstanding.

The Tzdata library has a bunch of different public functions. However they are not all needed to simply do timezone calculations. For instance you can get a list of all time zones, which is useful in some situations. But for the core calculations you mostly need one function.

Example:

gregorian_seconds = {{2018,01,01},{00,00,00}} |> :calendar.datetime_to_gregorian_seconds()
periods = Tzdata.periods_for_time("Europe/Copenhagen", gregorian_seconds, :wall)

Since there is no ambiguity for that wall time this returns just one period which has the abbreviation, standard offset and UTC offset needed to make a DateTime struct.

Here a behaviour that is not released, but one that the Tzdata library implements:

  @type gregorian_seconds :: non_neg_integer()
  @type time_zone_period_limit :: gregorian_seconds() | :min | :max
  @type time_zone_period :: %{
          utc_off: Calendar.utc_offset(),
          std_off: Calendar.standard_offset(),
          zone_abbr: Calendar.zone_abbr(),
          from: %{
            standard: time_zone_period_limit,
            utc: time_zone_period_limit,
            wall: time_zone_period_limit
          },
          until: %{
            standard: time_zone_period_limit,
            utc: time_zone_period_limit,
            wall: time_zone_period_limit
          }
        }

  @callback periods_for_time(Calendar.time_zone(), gregorian_seconds, :wall | :utc) :: [
              time_zone_period
            ] | {:error, :not_found}

Both Tzdata 0.1.x and 0.5.x versions implement that behaviour. So you can either have the newer version that can update automatically or the 0.1.x versions that require manual updates.

The “from” and “until” part are useful when there is a gap in wall time during e.g. “spring forward”. The “standard” part is not needed for most calculations, and I would lean towards excluding the “standard” part of from/until from a native Elixir behaviour.

0.1.x versions of Tzdata are using compilation of the data using macros. Once compiled and loaded, it is quite fast. The negative part of that is memory usage during compilation, but more importantly it requires compiling again whenever new data is needed. Akin to recompiling Postgrex everytime someone writes to a Postgres database.

About leap seconds: I also think there should be an interface/behaviour for that. Either as the same behaviour or one separate for one for time zones. Leap seconds are needed in order to verify UTC datetimes. So in order to fully support ISO8601 and UTC they are needed. And the tz database from IANA has leap second information included, so if we want to support both UTC and timezones, it makes sense to not ignore the leap second information provided.

4
Post #17
Qqwy

Qqwy

TypeCheck Core Team

@Lau thank you for that information!

Does anyone know what the story is with regard to checking if the OS clock already compensates for leap seconds (and how)?

@josevalim The fact that current datetimes already store the offset in an iso8601-format indeed means that it is a lot harder to write backwards-compatible extensions that are also calendar-agnostic. I will give this matter some thought, but it will probably difficult to keep changes both general, compatible and clean.

0
Post #18
josevalim

josevalim

Creator of Elixir

Well, to be fair, we could assume that they are always part of 86400. We just need to keep it in mind when doing computations to other timezones.

0
Post #19
Qqwy

Qqwy

TypeCheck Core Team

Yes. To be exact, this is already part of the implicit contract of DateTime. I remember that we talked about the notion of day-subdivision systems in the calendar discussion topic, were multiple subdivisions (like the Hebrew and Arabic systems) are just not compatible with 86400 seconds. But since these systems both see limited use, and I presume that especially for timezones this is fairly uncommon (Because AFAIK – do prove me wrong – all common timezones are offset from ISO8061/UTC), it might not at all be unreasonable to keep the Elixir standard library implementation limited to ISO8601-‘ish’ timezones.

Side Note: The only backward-compatible way to alter the structure that is represented that I can currently think of, would be to introduce a new struct with a new name (like ‘ProperDateTime’) that therefore no library could have written code against, which means that its implementation might be very different than the current DateTime struct. Immediate drawback: All current libraries do not work at all with this new structure.

0
Post #20
Lau

Lau

I ran some benchmarks on the compiled version of tzdata (version 0.1.201805)

Operating System: macOS
CPU Information: Intel(R) Core™ i7-7920HQ CPU @ 3.10GHz
Number of Available Cores: 8
Available memory: 16 GB
Elixir 1.6.4
Erlang 20.3.4

Name                                        ips        average  deviation         median         99th %
periods_for_time_for_reykjavik_tz       12.51 M      0.0800 μs   ±357.97%      0.0700 μs        0.21 μs
periods_for_time_by_utc_london           5.85 M       0.171 μs  ±2071.47%        0.20 μs        0.30 μs
periods_for_time_by_utc_cph              4.20 M        0.24 μs  ±1601.73%           0 μs           1 μs
periods_for_time_by_wall_cph             3.66 M        0.27 μs  ±1575.91%           0 μs           1 μs

The benchmarking tool says a median of 0 μs for some of them :thinking:

About the system clock and leap seconds: Different systems handle them differently. Unix time does not really support UTC. So there are different strategies to handle them: skip back a second to essentially repeat the 60th second (xx:59:59 happening twice). Or make each second slightly longer on a leap second day to compensate for the extra second.

I have read about some systems running on TAI instead of UTC. But from an Erlang point of view I haven’t seen anything about it handling TAI - it also uses UTC. I’m not sure but maybe if you set your system clock to run TAI instead of UTC, Erlang would think that it was UTC.

5
Post #21
ConnorRigby

ConnorRigby

Nerves Core Team

I only skimmed this so pardon if this has been said before, or if it isn’t exactly a problem.
also I use timex in a Nerves application, which pulls in the tzdata application. These are my only experiences with time libraries in Elixir other than the stdlib.

Firstly, i would love for stdlib to have timezone support, because that is the only reason i use/need timex. (I try not to use deps unless i really need them due to various constraints).

However, syncing the timezone database is something a little weird. tzdata downloads the package on a timer of some sort and writes it to disk. From what i know about the elixir stdlib, and a quick poke thru :observer, i don’t think there is anything else that does anything to this extent right now.

obviously defining a @behaviour as described above would alleviate this to some extent, even so i think implementing this at the stdlib level would be a little difficult and messy to cover all situations. (thinking from a Nerves point of view where the filesystem is read only, and network is not guaranteed.)

anyway sorry for that longwinded post, just wanted to express that blurb while this is being discussed.

0
Post #22