Proposal: strftime-based calendar/datetime formatting

josevalim · 2018-12-16T18:19:39.394Z

kip:

Although I started thinking this would be best done as configuration passed around in each structurally compatible Date , Time or DateTime struct or map, the variations are too many. I now feel it is better that this detail be backed into a module that implements the Calendar behaviour. This doesn’t make implementing a calendar any more complex. It does potentially have the side effect of having to generate calendar modules at runtime for some use cases.

I am not sure I agree with this.

The instants are the same and they are represented by the same year, month, day, hour, minute and second, so it does not feel to me they should be different calendars. To me, it is the same calendar, but it wants to be traversed or viewed in different ways. My suggestion would be to explore a WeekTraverser or WeekBuilder or some sort of behaviour that focuses on exploring a given calendar through different interpretations of its weeks. It may be even something that can be implemented on top of iso days (and therefore calendar agnostic)?

kip:

Formatting for strftime should not be the calendars concern (in my opinion).

I agree with this but the calendar has to provide a default behaviour. So the recent proposal has a default week_of_year but you can customize it via the week_of_year option. The calendar also provides the default values for preferred time and so on and I don’t think we can escape this.

kip:

Format encoding: %<flags><width><conversion> .

While I really like this idea, then we are no longer implementing strftime, so it needs to be a very conscious choice. The biggest benefit IMO is that it gets rid of all the padding options and, as you said, removes formatting concerns from the calendar.

kip:

Flags (based upon the Ruby implementation)

Those flags are all officially supported by strftime but I decided to go only with three related to padding for now for simplicity. We can always add more later.

kip:

I have included an options argument to strftime above because for localisation a locale name needs to be specified. In Python a locale is part of the stdlib and runtime environment but not so Elixir.

Isn’t this then forcing localization and therefore formatting concern into the calendar? I would prefer if the calendar does not receive a locale, ever. With the latest proposal, you can localize everything like this:

Calendar.format(datetime, "%format", MyApp.Calendar.format_config(:pt_br))

So I believe we are going with the loose contract you mentioned.

@kip assuming that we will merge the flags and width syntax that you proposed, are there any other changes you would like to see based on my reply?

Thank you.

kip · 2018-12-16T18:47:15.156Z

@josevalim Looks good. I understand why the MyApp.Calendar.format_config(:pt_br) approach. And its easy to integrate with my Cldr project so that Cldr users can adopt Calendar.format easily.

josevalim:

I am not sure I agree with this.

Totally ok with not agreeing. I just can’t reconcile how to do what you suggest and still use the Date, Time and DateTime standard structs which have a :calendar field only. Also every calendar is basically just a different representation of the same moments in time so by extension aren’t you saying there is only one calendar and multiple different representations that model year, month, day in alternate ways? Anyway, off topic for this thread.

I have nothing else to add to the proposal and I think a lot of users will benefit from it.

josevalim · 2018-12-16T18:52:16.405Z

kip:

Anyway, off topic for this thread.

Good call, I copied the discussion to a new thread so we can talk more about it: How to support multiple week calendars in Elixir?

rudolfb · 2018-12-17T17:18:15.977Z

josevalim:

week_of_year - a function that receives calendar, year, month, day and returns the week_of_year tuple

In german speaking countries we have the concept of a calender week, which differs to a simple week_of_year number. I am unsure what value the week_of_year number would return in the current planned implementation.

A week_of_year can be based on the 1JAN of the year, where any date from 1JAN to the 7JAN (both inclusive) is interpreted as WeekNr=1, and date between 8JAN to 14JAN (inclusive) is interpreted as WeekNr=2, etc. This could be called week_of_year.
Another way to define a week_of_year is to use a relevant first_day_of_week, e.g. MON, and then look to see how many days of the start of the year are in the first week, and if the number of days >=4 (for a 7 day based calender) then this is the first week, if there are three or less, then the week is calculated as being a part of the previous year’s last year number:
https://en.wikipedia.org/wiki/Week#Week_numbering
This could be called week_of_year_iso.

The calculation of the week_of_year might thus depend not only on the calender and specific date, but also on the defined first day of the week for the calender as well as the type of week_of_year number the user is looking for. Just having a week_of_year option might not suffice to differentiate what type of WeekNr ought to be returned, or the returned WeekNr might not be what a caller expects.

josevalim · 2018-12-17T18:14:50.198Z

@rudolfb the week_of_year would be customizable, so it would return whatever you want. But we have currently removed it from the proposal because we are still not sure that’s the direction we want to go

josevalim · 2018-12-17T18:20:35.725Z

kip:

Format encoding: %<flags><width><conversion> .

@kip we have decided to accept this extension but we will keep the default formatting based on ISO. The rationale is the following, if we keep the default without padding, then it means that all calendars users have to add padding.

However, if we keep the ISO convention, then the ISO users do not have to add explicit padding, but non-ISO users will have to add them exactly as before. This keeps it convenient for ISO users (the majority), it allows us to follow the strftime spec, it allows us to reduce knowledge of formatting in the calendar, while providing useful features to other calendars.

Finally, I would like to note that, by figuring out all of these trade-offs with strftime, we have actually solved the problems we had with ICU. However, we are planning to go with strftime because it has a much smaller surface API.

Do you have any final thoughts? I will update the proposal with those latest changes.

kip · 2018-12-18T01:13:32.655Z

@josevalim No final thoughts. strftime makes more sense for most users I think since its a familiar api across languages and having the format default to the ISO calendar aligns well too.

Lastly, this approach still allows ex_cldr_dates_times to serve configuration very easily to strftime which I think will be helpful for some people.

josevalim · 2018-12-19T21:26:07.736Z

Hi everyone, we are glad to say this proposal has been accepted. The next step is to develop it as a separate library for further validation. We will let you know once that happens.

kuon · 2018-12-20T15:06:35.829Z

This might be out of scope for the standard formatting, but I had to display hours with value >24h for late night schedules.

The idea is simple, instead of writing 2019-01-01T02:00 we have 2018-12-31T26:00.
This is quite common in Japan, but I’m sure it is used elsewhere. This is used mostly for event schedules (cinema, tv shows, conferences…). The idea is that it’s the continuation of the days you woke up.

I wonder how this kind of date could be formatted in elixir.

josevalim · 2018-12-20T22:19:35.063Z

@kuon in those cases, you can always interpolate the value directly on the format string, mixed with the formatting syntax.

bjunc · 2018-12-22T18:13:13.661Z

I think there is more than enough rationale to justify something like this. To be honest, when I first got into Elixir, this was something I looked around for and just assumed it already existed.

Having used datetime formatters in a few languages, I think my favorite has been Moment.js (which isn’t strftime, and is closer to LDML). I even created a version for Elixir (sorry, never shared it). I think the formatting tokens are more intuitive than strftime.

https://momentjs.com/docs/#/displaying/format/

For example, what I created looked like this:

DateTimeFormatter.format(datetime, "YYYY-MM-DD")
{:ok, "2018-12-22"}

DateTimeFormatter.format!(datetime, "M/D, h:mm a")
"12/22, 1:14 pm"

DateTimeFormatter.format(datetime, "[today is] dddd")
{:ok, "today is Saturday"}

One key difference is multiple characters for the token. Eg:

M              1 2 ... 11 12
Mo             1st 2nd ... 11th 12th
MM             01 02 ... 11 12
MMM            Jan Feb ... Nov Dec
MMMM           January February ... November December

That said, it sounds like you are proposing the ability to substitute formatters? So the default would be strftime, but someone could build a LDML formatter?

josevalim · 2018-12-22T18:21:58.942Z

bjunc:

someone could build a LDML formatter?

That someone can even be you!

But yes, a formatter is just a module, so you can just define your own module if you want to.

bjunc · 2018-12-22T18:35:47.387Z

Ha, well I’d be happy to give it a shot.

kip · 2019-06-09T12:34:48.919Z

With my CLDR-based date/time formatter now finally out the door I look forward to serving configuration for strftime whenever its ready for a road test …

josevalim · 2019-11-02T09:24:56.205Z

We have finally implemented a library based on this proposal: https://github.com/plataformatec/nimble_strftime

Everyone, please do give it a try in your application! And @kip, let us know if it provides the necessary hooks for i18n/l10n.

kip · 2019-11-03T04:48:13.928Z

@josevalim, all good. I think the only thing maybe missing is era, but I see that’s not part of the substitutions anyway.

For those looking to localise formatting with NimbleStrftime, there is now an ex_cldr function to generate the options for NimbleStrftime.format/3. ex_cldr_dates_times is another option for those looking for CLDR-based formatting.

MyApp.Cldr.Calendar.strftime_options!/2 (where MyApp is any Cldr backend module you’ve defined) provides these options. You’ll need to update to ex_cldr_calendars version 1.5.0.

Its a ! function because if the specified locale is unknown it will raise.

Examples

iex> NimbleStrftime.format(~D[2019-11-03], "%A, %b %d %Y", MyApp.Cldr.Calendar.strftime_options!())
"Sunday, Nov 03 2019"

iex> NimbleStrftime.format(~D[2019-11-03], "%A, %b %d %Y", MyApp.Cldr.Calendar.strftime_options!("fr")) ==
"dimanche, nov. 03 2019"

kip · 2020-01-22T03:02:36.211Z

With Calendar.strftime/2 being merged into Elixir master (from the NimbleStrftime lib) I’d like to open a proposal on some additional formatting options and gain community feedback.

There are three sub-proposals and any and all feedback is welcome.

Weeks (yes, again with the weeks conversation )

Linux strftime has formatting flags for weeks. These being:

   %U     The week number of the current year as a decimal number, range
          00 to 53, starting with the first Sunday as the first day of
          week 01.  See also %V and %W.  (Calculated from tm_yday and
          tm_wday.)

   %V     The ISO 8601 week number (see NOTES) of the current year as a
          decimal number, range 01 to 53, where week 1 is the first week
          that has at least 4 days in the new year.  See also %U and %W.
          (Calculated from tm_year, tm_yday, and tm_wday.)  (SU)

   %W     The week number of the current year as a decimal number, range
          00 to 53, starting with the first Monday as the first day of
          week 01.  (Calculated from tm_yday and tm_wday.)

Adding this formatting directives would require an update to the Calendar behaviour to provide support.

Options:

No need, don’t add
Just add %V for the ISO week number and add Calendar.iso_week_of_year/1 to the Calendar behaviour
Maximise compatibility, go with all three and add the callbacks

Add support for Era

Calendar.day_of_era/1 returns {day, era} but that’s an integer, not a display format. Calendar behaviour doesn’t do any display format translation today. The new Calendar.strftime/2 does display format translation for AM and PM (and variants). So one approach is to simply enhance Calendar.strftime/2 to also map Calendar.ISO eras 0 -> display_name and 1 -> display_name. And there would need to be agreement on display name (AD versus CE and BC versus BCE).

Era isn’t used in day-to-day formatting for Gregorian calendars except for dates before year 1. It is used as a standard part of formatting Japanese calendars and for some format of Chinese calendars (and derivatives on the 60 year cycle).

strftime defines the following directives:

Specifier	Meaning
%Ec	Date/time for current era.
%EC	Era name.
%Ex	Date for current era.
%EX	Time for current era.
%Ey	Era year. This is the offset from the base year.
%EY	Year for current era.

These can be implemented without a change to the calendar behaviour if Calendar.strftime/2 treats era like it treats am/pm and provides an internal translation. It would perhaps be better that both of these (ie am/pm and era translations became behaviours since that would also help in international projects using Gettext or Cldr.

Options

Era? Who cares, forget about it
Compatibility is a good idea and implementation seems simple. Do it.
I care about calendars beyond Calendar.ISO so I need this (I’m not holding my breath on this one )

Localised number systems (ie not Latin alphabet)

strftime supports localising the date format to use non-latin alphabets. These is also supported in ex_cldr and friends. The default for these directives is to use the Latin alphabet so implementing these directives is quite trivial. If no configuration is provided in options, use the fallback to the Latin characters.

The formatting directives are:

Specifier	Meaning
%Od	Represents the day of the month, using the locale’s alternative numeric symbols, filled as needed with leading 0’s if an alternative symbol for 0 exists. If an alternative symbol for 0 does not exist, the %Od modified conversion specifier uses leading space characters.
%Oe	Represents the day of the month, using the locale’s alternative numeric symbols, filled as needed with leading 0’s if an alternative symbol for 0 exists. If an alternative symbol for 0 does not exist, the %Oe modified conversion specifier uses leading space characters.
%OH	Represents the hour in 24-hour clock time, using the locale’s alternative numeric symbols.
%OI	Represents the hour in 12-hour clock time, using the locale’s alternative numeric symbols.
%Om	Represents the month, using the locale’s alternative numeric symbols.
%OM	Represents the minutes, using the locale’s alternative numeric symbols.
%OS	Represents the seconds, using the locale’s alternative numeric symbols.
%Ou	Represents the weekday as a number using the locale’s alternative numeric symbols.
%OU	Represents the week number of the year, using the locale’s alternative numeric symbols. Sunday is considered the first day of the week. Use the rules corresponding to the %U conversion specifier.
%OV	Represents the week number of the year (Monday as the first day of the week, rules corresponding to %V) using the locale’s alternative numeric symbols.
%Ow	Represents the number of the weekday (with Sunday equal to 0), using the locale’s alternative numeric symbols.
%OW	Represents the week number of the year using the locale’s alternative numeric symbols. Monday is considered the first day of the week. Use the rules corresponding to the %W conversion specifier.
%Oy	Represents the year (offset from %C) using the locale’s alternative numeric symbols.

Options

Who cares about countries and people that don’t use the Latin alphabet. Forget it.
Good idea - easy to implement, no impact if I’m not doing non-latin alphabet code. Glad to see Elixir embracing global cultures (ok, I’m pitching I know )

josevalim · 2020-01-22T11:02:28.403Z

kip:

Weeks (yes, again with the weeks conversation )

I believe we can add %V but I would make it calendar specific. So once you change calendars, it will return the week number of said calendars. The default is of course ISO. Does strftime specify a way to get the week of year? Without week of year, I don’t see this being very useful.

About %U and %W, I am honestly not sure, as they seem to be based on gregorian calendars. What would they return for a japanese calendar, for example?

kip:

Add support for Era

… So one approach is to simply enhance Calendar.strftime/2 to also map Calendar.ISO eras 0 -> display_name and 1 -> display_name .

I agree with this proposal. day_of_era works but we will probably need to use year_of_era to implement all of the modifiers (except the %EC itself).

Regarding “(AD versus CE and BC versus BCE)”, let’s just pick whatever strftime uses/returns.

kip:

Localised number systems (ie not Latin alphabet)

I like this proposal too but I wonder if it should simply be an option called :number_formatter. The number formatter is a function that receives the number as an integer and the padding, and it returns a string. You can replace the number formatter to use any numeric alphabet and padding that you want. This seems simpler overall. WDYT?

Thanks for writing these down @kip. If you want, we can proceed with a PR for era while we discuss the remaining topics.

kip · 2020-01-23T05:00:32.702Z

Looks like I have misunderstood how %EC works (at least on MAC OSX). It just returns the century years:

kip@Kips-iMac-Pro strftime % ./a.out '%EC'
Result string is "20"

Therefore I don’t see a compatible path forward for including era formatting in strftime/2.

josevalim · 2020-01-23T10:27:22.206Z

So there is no way to get the actual era name in strftime?