Calendars & Calendar Conversions

Chat / Discussions
, ,
#33

I do not think Elixir’s standard library will become more complex because of this change. Rather, it will be a simple and well-defined base that libraries implementing the different calendars can build on, and allow them to be interoptable. This was the whole reason for Calendar, Date, Time, DateTime and NaiveDateTime to be added to Elixir in the first place.

@josevalim @kip: I’ll create a simple library that implements some of the calendars described in the Calendrical Calculations and Calendrical Calculations II papers while attempting to use Elixir’s existing Calendar, Date, DateTime etc. structs, and see what issues arise.

1 Like
#34

I didn’t mean the ability to convert calendars made it complex. the strict usage of JD/RD made it complex. even in Go and .NET where they use JD/RD, they use it is inside the Calendars implementation and they can choose either to use Integer date or any other formats they want. (In .NET the base timestamp is ISO DateTime which everyone knows about and there are many different algorithms for conversion from/to it)

But i agree JD/RD it is a good approach and does not have many downsides of other timestamps and of course it is simple.

But changing or making Date struct Dynamic will make very larger complexities especially in integrations of Elixir and other APIs. don’t you think?

Seems like a good idea. :heart:

#35

Right now, Elixir does not work with Julian Day or Rata Die yet; this topic is the discussion about adding it.
Using JD/RD and converting to and from it results in algorithms that are considerably less complex than when UTC is used as intermediate format. Furthermore, as @kip has noted, there exist implementations of conversions from and to JD/RD for at least 23 existing calendars.

The idea is to hide this internal representation and conversion from the end users. Only people that implement new calendars have to know about it.

I would see downside to having made the Date, Time, DateTime etc. structs dynamic, as the internal representation could still have been pattern-matched on if required. However, as these structs now have become part of the core language in their current form, they should not be altered as this would be a backwards-incompatible change. So no: they will not and should not become dynamic.

I could see an :extra field being useful for Date, Time, DateTime etc. to allow calendar implementations to store information that is not part of the normal fields. This makes more sense to me than e.g. the ISO Week Date calendar using :month to store the week number.

#36

I really like the combination of {RD, microseconds} format. and i think @kip is implementing it.
The only concern i have is about developers who is going to implement another calendar for Elixir. if their algorithm does not work with JD they should change it. JD is really good, i can’t think of a situation that it will fail. but it would be a good idea if Elixir could give developers freedom to choose either to follow this rule or not.

Sure, using :month as week number is not a good hack. it may work for some people but surely it is not going to be used officially. :extra field may be a good idea. I also think another third-party implementation of Date would be a good idea for those scenarios.

#37

@alisinabh: @kip is working on a Pull Request right now that uses Rata Die as internal representation. The current version uses integers, but he is planning on writing a version using either the millisecond format or a fractional format. This was stated by him a few posts ago.

As previously stated, I strongly advise the use of a fractional format as a millisecond representation is:

  1. imprecise.
  2. decoupled from the length of one earth rotation and therefore suffers from leap seconds.

I am also very interested about the opinion of @Lau, as he is the author of the original Calendar library that the structs that are now part of Elixir Core got split off from.

1 Like
#38

9 posts were split to a new topic: Ex_calendar

#39

@OvermindDL1 Oh, sorry to hear that, i hope you get better soon :frowning:

This is a pretty good idea. It is beautiful but unfortunately i don’t think it is currently achievable due to its non-backward compatible nature without getting our hands dirty.

And not to mention, the year, month, day structure will handle 99.9% of use cases. So, not supporting it in core data structures would not be a big deal! I personally don’t know any languages that does support this. even Joda-Time is strict about year, month, day structure in BaseDateTimes. although you can implement your own version of a DateTime and use Joda calendars (Chronologies).

#40

You ‘do’ support it built-in, just via a Calendar.Gregorian or Calendar.ISO or something, those would be the most used ones by far. :wink:

#41

But instead of Date or DateTime everyone should change to something like Calendar.ISO.Date or Calendar.ISO.DateTime. Am i wrong?

#42

Or alias it in of course. ^.^

#43

That was what i meant when i said getting hands dirty :slight_smile:

1 Like
#44

Heh. One nice side-effect, it clearly tells you what format your Date/DateTime is in. ^.^

2 Likes
#45

I like it too, and since Elixir is a newborn language, maybe this change wont get anybody mad because it is really useful and really Elixir like. It is a nice feeling if Elixir handles it in such a way that there will be no calendar that doesn’t fit into its core structure.
I really like your idea. :thumbsup:

2 Likes
#46

From a developers perspective:

We already had a lot of different Date/Time structures. Each library had it’s own - timex had it’s own, calendar had it’s own, ecto had it’s own, different calendar implementations had it’s own… And it was a horrorshow. The usability of those solutions was extremely poor. If you needed to do anything with dates, you found yourself in an endless pipeline of conversions from one format to another. Please, please, please let’s not go back.

1 Like
#47

Well spoken.

I think it is important to realize that there are two separate problems that Elixir Core attempts to solve:

  1. Ensure that different date/time libraries can communicate, by specifying a single underlying data type.
  2. Ensure that different calendars can be represented and calculated with; that Elixir is not tied to a single calendar.

The first problem was solved by introducing the Date, Time, DateTime and NaiveDateTime structs. Since Timex, Calendar and Ecto all only worked with the ISO 8601 proleptic Gregorian calendar, it was very natural at that time to include struct fields that matched that calendar.

I think the choice of splitting off the NaiveDateTime was a very smart one that follows the explicit nature of the language.

I also want to praise the Elixir Core team that they had the foresight at that time to think about (2.), included a :calendar field in these structs and alter the way the structs are represented by dispatching to the Calendar behaviour. :thumbsup:!

This was what has left room open for the work @kip is doing right now: creating a very nice way of converting between different calendars by using the Rata Die as intermediate representation. This will allow:

  • Comparing between two possibly different calendars, even if they do not follow the :year, :month, :day, :hours, :minutes, :seconds fields in the way that Calendar.ISO does.
  • Obtaining an interval in {days, fraction_of_day} between two dates in possibly different calendars.
  • Adding a {days, fraction_of_day} interval to a date in any calendar.
  • Enumerating over an interval between two timestamps (in days, and possibly also any other subdivision or superdivision days)

So: I am really happy with the changes that have been introduced in 1.3, and I am also very happy with the new changes that are being worked on! :smiley:

#48

What would be a really nice addition in the future by the way, is to add a %Period{} struct (Or maybe a %Date.Period{}, %TIme.Period{}, %DateTime.Period{} and %NaiveDateTime.Period{}?) whose fields have the same names as those of the normal structs, and that can be used to specify periodical events, such as ‘1 month and 3 days’.
These could then be combined with a starting date(time) to return the times of periodic events in the future. Other than with an exact interval (a {day, fraction_of_day}), this period is added per-field: in above example, first one month is added to the current date, ending up at the same day next month (or possibly the last day of the month if that month has less days). And then three days are added (maybe rolling over to the next month).

This idea is not new; Joda time uses it as well.

And of course, details of how this would work exactly is something to discuss :slight_smile: .

#49

There is absolutely nothing stopping this from happening today. Otherwise it is like saying you cannot implement another key-value data structure because Elixir already has a Map module. :slight_smile: If you want key-value genericity, you can define the protocol and you can define new data types. The same for the calendars data structures.

Given the Gregorian Calendar is the international de facto and using it as a base is what makes sense for the huge majority of developers and the huge majority of times, it makes sense to prioritize its representation.

So while it is very interesting to see how far we can take the calendar structures we have today, I have absolutely no problem with saying we won’t support calendars X, Y and Z since the problem can be solved by a third party. In fact, we may even arrive to the conclusion that the :calendar field is unnecessary and any other calendar should be integrated via a protocol. All it takes is someone to define the protocols in a separate library, call it Zoolendar or whatever, which everyone can depend on when building new implementations.

4 Likes
#50

Hehe. ^.^

2 Likes
#51

I agree with protocols and separate time representations idea. Also agree with you that Gregorian should be default.
I can create simple demo structure that combines both notes today. I wws already think about it and want to ask too.

#52

Is there a need to add new data formats to represent dates and times? There are a bunch already: Date, NaiveDateTime, DateTime structs in the ISO calendar. Additionally Erlang already has an integer representation of dates. E.g. {2017,1,8} |> :calendar.date_to_gregorian_days and also functions for “gregorian seconds”.

If we want to add a behaviour for alternative calendar modules, could it simply require functions for conversions to and from ISO structs? E.g. %DateTime{calendar: Calendar.ISO}, and the same for Date and NaiveDateTime.

ISO DateTime structs already support leap seconds. Although there are reasons for not always taking leap seconds into account when doing conversions.

You can know about leap seconds up to ~6 months into the future. The Tzdata library provides this information. The Calendar library (calendar on hex) uses the tzdata to make this list native Elixir 1.3+ DateTime structs. Example:

 iex> Calendar.TimeZoneData.leap_seconds |> List.last
 %DateTime{calendar: Calendar.ISO, day: 31, hour: 23, microsecond: 0,
    minute: 59, month: 12, second: 60, std_offset: 0, time_zone: "Etc/UTC",
    utc_offset: 0, year: 2016, zone_abbr: "UTC"}

This functionality is also used to validate DateTime structs where the second field has a value of 60.

Calendar.DateTime.from_erl({{2015, 12, 31}, {23, 59, 60}}, "Europe/London") returns an error because there was no leap second at 2015 December 31st.

Calendar.DateTime.from_erl({{2015, 12, 31}, {23, 59, 60}}, "Europe/London")
{:error, :invalid_datetime}

But in 2016 there was, so this is allowed provided the newest tzdata is available:

Calendar.DateTime.from_erl({{2016, 12, 31}, {23, 59, 60}}, "Europe/London")
{:ok, %DateTime{calendar: Calendar.ISO, day: 31, hour: 23, microsecond: {0, 0}, minute: 59, month: 12, second: 60, std_offset: 0, time_zone: "Europe/London", utc_offset: 0, year: 2016, zone_abbr: "GMT"}}
1 Like