Calendars & Calendar Conversions

@Eiji I said assuming if this anomaly exists every year until year 86336 no problem will be there with dates.

@Qqwy thank you for explaining :blush: i have closed my PR on conversion using UNIX.

I have the feeling that some of my posts might have been too long, so to reformulate:

@kip, @josevalim I propose we use a {RD/JD, fraction} approach, as this would mean that calendars do not have to care about their internal details: For instance, when converting from the Islamic to the Hebrew calendar, leap seconds would not need to be considered at all!

The fraction part could either be an arbitrary-precision Decimal, a Rational number, or a fixed-precision integer representation that is precise enough to handle very small differences between day subdivisions. Using floats is of course out because of their rounding errors. Using decimals or rationals is out if we want to include the Calendarium in the core (or minimize dependencies for another reason).

If we want therefore to use a fixed-precision representation, we need to choose something that is precise enough to handle very small differences. The leap second differences in the UTC calendar are the smallest ones that we will need to handle for the forseeable future.
We know that we want to represent all of 1 / 86400 (one SI-second on a normal day) 1 / 86401 (one SI-second on a leap-second-day) and 1 / 86399 (one SI-second on a leap-second-removal-day). Milliseconds are not precise enough. Attoseconds are.

To whomever is confused by the concept of leap seconds:

Consider a straw.
This straw has a certain length.
I can put my ruler against it, and measure that it is ‘10.25’ centimeter.
I can also put a ruler in inches against it, and measure that it is ‘4.035’ inches.

Neither observation has altered the length of the straw.

If we want to count multiples of straws lengths in fixed, centimeter-size steps, we could say ‘well… lets assume they are exactly 10 centimeter long’.
But now here’s the problem: After counting four straws, we end up with 40 centimeter. But in reality, four straws are 41 centimeter. Oh snap! The more straws we count, the more our calculation is off.
The solution? Proclaim that every fourth straw is a leap straw which is not ten, but 11 centimeter long!

These proclamations have not altered the length of the actual straw in any way either!

The straw in this analogy is of course the day, whose length is the same across all calendars, because that’s simply how the earth rotates.

When we convert from Super Strawcounting With Leapcentimeters to Inches or vice-versa, it is useful to choose an intermediate representation that is based upon the length of the straw itself: This representation will never need leap straws, as ‘a quarter straw’ is always exactly a quarter of a straw, regardless of with what ruler it is measured.

And therefore, if IERS decides that another leap second is needed, this does not alter that ‘a day is a day’ (one full rotation of the earth remains one full rotation of the earth), but only how many SI-seconds fit in that given day. And therefore, counting with fractions of a day is more suited to calendrical conversions than choosing e.g. milliseconds as in-day unit.

1 Like

@Qqwy i think there is still the need to take leap seconds into account if we are talking about a diff function for datetimes? there is such a function now for naivedatetime, which doesn’t consider leap seconds and i think probably that needs an update.

For purely conversion i can seee it’s not a consideration.

We need to take leap seconds into account when writing a function such as seconds_between.
We will not need to take leap seconds into account when writing a function such as period_between, where ‘period’ will be returned as full days + final day fraction.

I missed that you are proposing fraction-of-day instead of microseconds.

Fraction of day is how RD/JD are usually defined as you know so there is familiarity and a lot of literature to support this approach.

I recognise better now why you were emphasising that precision is important - in order to preserve round trip conversions of at least microsecond resolution i assume, since that’s the contract described by Calendar.

I’ve done most of the work now to update my PR - pending whether there’s a consensus on whether to go the microseconds-since-midnight or the fraction-of-a-day route.

@Qqwy, @josévalim any additional thoughts to arrive at a conclusion before i finish up the next version?

1 Like

A tangentially related question to @josevalim: the current Date struct defines fields foryear, month, day. The current DateTime struct adds hours, minutes, seconds, milliseconds, timezone_info to that. For quite a few calendars, most of these fields will be unused (for instance, the hebrew callendar subdivides the day in many so-called ‘parts’ instead of hms). For other calendars, more fields might need to be added (or certain properties recomputed each time it is used).

What is the rationale behind requiring all custom calendar structs to have these fields? Why was this chosen over a simple %Date{calendar: calendar_module, content: calendar_specific_representation} representation?

Hmm, probably for pattern matching, but that does beg the question, why does the calendar type not supply the Date/DateTime types instead of the other way around?

We will need to balance those trade-offs because hiding it behind the calendar would make it more flexible for multiple calendars but make it harder for 99% of the developers who don’t care about those.

1 Like

I totally understand and agree with this, although I am not entirely sure what advantage it has to specifying these fields directly in the struct. :slight_smile:

What would be the best way for calendars that have different kinds of subdivisions/supervidisions to fit in the current Date and Time structs? (ab)use one of the existing struct fields?

Can you please post some samples of calendars that don’t use year, month, day?

I do not know the answer to this question. :slight_smile: Someone would have to try different approaches and let us know how it goes.

1 Like

The Mayan and Balinese calendars have a different structure than year, month, day. As far as I can tell, other calendars fit year, month, day quite well,

Hmmm, i don’t think it is necessary to support them into Date structs. Elixir is becomming pretty good at calendars and i like it. But its ok not supporting 2 of nearly obsoleted type of calendars natively i think. They can have libraries defining their own structs.

Non of languages i’ve worked with support that kind of calendars in their nature.

By the way, it is just my opinion.

I can easily think of two other very commonly-used calendars that do not fit {year, month, day}:

  • The Julian Day and Rata Die calendars we talked about in this topic, which only count days.
  • The ISO Week date calendar, that is used a lot in business and fiscal environments, which counts {year, week_number, day}.

Besides this, there are many calendars that do not fit the DateTime subdivision in hours/minutes/seconds/milliseconds, such as (from the top of my head; there are more):

  • Rata Die, which counts a fraction of a day and is therefore precise.
  • The afore-mentioned Hebrew calendar, which uses 12 hours that are each subdivided in 1080 ‘parts’, a day thus contains 25920 parts.
  • The Hindu calendar, which subdivides a day into 60 Ghati and a Ghati into 60 Pala.
  • TAI, which only counts seconds.
  • POSIX time, which only counts seconds but gets corrected for leap seconds.
1 Like

Supporting them in Date struct will add so much complexity. the only two ways are either changing Date structure or Hacking other calendars in it.

One complexity i can think of is the presentation. how would as a developer present a Date that has more than 3 factors. using Date will become tricky. Date should be represented as a format for calendars which follow year, month, day rules. as i said, it is more convenient for them to have their own structs. I think we should not handle them in Elixir native Date struct.

And i have to say i’m still not happy with overheads and complexity added to Elixir to achieve calendar conversions using JD/RD. although that seems to be only my concern in this trade-off. :pensive:

I do not think Elixir’s standard library will become more complex because of this change. Rather, it will be a simple and well-defined base that libraries implementing the different calendars can build on, and allow them to be interoptable. This was the whole reason for Calendar, Date, Time, DateTime and NaiveDateTime to be added to Elixir in the first place.

@josevalim @kip: I’ll create a simple library that implements some of the calendars described in the Calendrical Calculations and Calendrical Calculations II papers while attempting to use Elixir’s existing Calendar, Date, DateTime etc. structs, and see what issues arise.

1 Like

I didn’t mean the ability to convert calendars made it complex. the strict usage of JD/RD made it complex. even in Go and .NET where they use JD/RD, they use it is inside the Calendars implementation and they can choose either to use Integer date or any other formats they want. (In .NET the base timestamp is ISO DateTime which everyone knows about and there are many different algorithms for conversion from/to it)

But i agree JD/RD it is a good approach and does not have many downsides of other timestamps and of course it is simple.

But changing or making Date struct Dynamic will make very larger complexities especially in integrations of Elixir and other APIs. don’t you think?

Seems like a good idea. :heart:

Right now, Elixir does not work with Julian Day or Rata Die yet; this topic is the discussion about adding it.
Using JD/RD and converting to and from it results in algorithms that are considerably less complex than when UTC is used as intermediate format. Furthermore, as @kip has noted, there exist implementations of conversions from and to JD/RD for at least 23 existing calendars.

The idea is to hide this internal representation and conversion from the end users. Only people that implement new calendars have to know about it.

I would see downside to having made the Date, Time, DateTime etc. structs dynamic, as the internal representation could still have been pattern-matched on if required. However, as these structs now have become part of the core language in their current form, they should not be altered as this would be a backwards-incompatible change. So no: they will not and should not become dynamic.

I could see an :extra field being useful for Date, Time, DateTime etc. to allow calendar implementations to store information that is not part of the normal fields. This makes more sense to me than e.g. the ISO Week Date calendar using :month to store the week number.

I really like the combination of {RD, microseconds} format. and i think @kip is implementing it.
The only concern i have is about developers who is going to implement another calendar for Elixir. if their algorithm does not work with JD they should change it. JD is really good, i can’t think of a situation that it will fail. but it would be a good idea if Elixir could give developers freedom to choose either to follow this rule or not.

Sure, using :month as week number is not a good hack. it may work for some people but surely it is not going to be used officially. :extra field may be a good idea. I also think another third-party implementation of Date would be a good idea for those scenarios.