Proposal: DateTime.from_iso8601/3 preserves wall date time and applies Etc/GMT±n time zone ID

Chat & Discussions Proposals: Ideas
,
1

Sumary of proposal

  1. DateTime.from_iso8601/3 adjusted to:

    • set all numeric fields to the values as parsed (not shifted to UTC), preserving wall time.
    • set the :utc_offset field to the parsed offset value,
    • set the :std_offset to zero
    • set the :time_zone field to Etc/GMT±n (standard IANA canonical zone names) or Etc/Unknown (reserved by the IANA database to represent unknown time zones)
    • set the :zone_abbr field to ±n or unk

  2. Adjust the implementation of the UTC-only time zone database to include the following zones which are canonical, non-varying offsets from UTC:

    • All Etc/GMT±n canonical zones (not links) from the etcetera file, including their abbreviations ±n
    • The Etc/Unknown time zone name (and its abbreviation unk) to represent unknown time zones. This zone is reserved for this explicit purpose.

  3. Support these Etc/GMT±n time zones in DateTime.shift_zone/2 as part of the core library (ie not requiring a time zone database to be configured).

Why this proposal?

ISO8601 is a common standard representation of human-readable dates and time but it does not provide an ability to express time zone other than as an offset from UTC. Since a time zone database is not included in Elixir, the standard library can only create DateTime structs in the Etc/UTC time zone.

This leads to some unexpected results in parsing ISO8601 date time string with DateTime.from_iso8601/3. For example:

iex> {:ok, date_time, _offset} = DateTime.from_iso8601("2025-07-10T23:00:00-01:00")
{:ok, ~U[2025-07-11 00:00:00Z], -3600}

iex> Date.to_string(date_time)
"2025-07-11"

Here we can see that because Elixir has to return a date time in the Etc/UTC time zone, it basically parses the ISO8601 date time then shifts it to the Etc/UTC time zone. Information is not lost - the offset is part of the returned tuple. However this can result in the wall time of the date time changing from the original value in potentially unexpected ways.

Since the Date, Time, NaiveDateTime and DateTime modules accept any map with the relevant fields (a good thing), the change in wall time from the original ISO8601 can lead to surprising results like the above example.

Proposal benefits

  • ISO8601 parsed date times can be better represented in Elixir leading to fewer unexpected errors and a more expressive result from parsing.
  • The full range of Etc/GMT±n time zone names (and their abbreviations) can be represented in the standard library without an external time zone database and date times can be shifted amongst them.
  • An unknown timezone, Etc/Unknown can be applied when parsing an offset with no standard time zone name. This would apply for any offset that has minutes (like Australia/Adelaide).

Possible compatibility issues

  • The proposal does not change the structure or meaning of any of the fields in DateTime
  • The Etc/Unknown zone is reserved - but does not exists in the IANA time zone database. As a result it can be used with impunity but will result in an error when attempting to DateTime.shift_zone/2. This is also compatible with existing function signatures.
  • It is possible that some usage depends on DateTime.from_iso8601/3 returning a UTC date time. Although ti should be noted that this is not called out in the function documentation
7 Likes
2

A following proposal might be to add DateTime.from_rfc9557/1 following the RFC9557 standard, in particular its support of Internet Extended Date/Time Format (IXDTF).

This format allows the expression of date times with time zones (and other information). For example, the following is a valid RFC9557 date time:

2022-07-08T00:14:07Z[Europe/Paris]
3

Changing the return value of DateTime.from_iso8601/3 would be a breaking change. So I guess a better option would be DateTime.from_iso8601/4, which an additional parameter / keyword option allows for opting into the new behaviour. It would potentially also allow to switch to a {:ok, datetime} return value over {:ok, datetime, 0}, which would be confusing because there is an offset, just transported on the datetime instead of on the side.

The Etc/GMT±n timezones only support full hour offsets it seems. There are many timezones with offsets not mapping to full hours and their iso8601 values would not be covered by those. So I’m not certain this would be much of a useful addition.

Etc/Unknown generally seems like a reasonable compromise given the (named) timezone is indeed unknown – we only know the offset, but not the timezone it orginiated from. The only question I see is how we handle computations on Etc/Unknown. E.g. UTC-7 offset can be both MST or PDT(daylight saving for PST), so how would DateTime.shift(datetime_offset_minus_7, month: 2) act. It would be ambiguous if that is “supposed” to hit a daylight saving shift or not. We could decide for one or another, but it’s an argument to be had on “correctness” or we could consider this an error case for the operation, which again would be a breaking change to a function signature.

In the end this is a bit of a weird case because we technically have offsets / some notion of a timezone, but any calculations (instead of derivations) made on such datetimes would essentially map to NaiveDateTime in terms of correctness properties.

2 Likes
4

Good comments as always @LostKobrakai.

Changing the return value of DateTime.from_iso8601/3 would be a breaking change

The function signature doesn’t change, but definitely the return value of the date time would change. The offset doesn’t need to change - it can still return the millisecond offset value.

So I’m not certain this would be much of a useful addition.

Given its DateTime.from_iso8601/3 I think that any opportunity to return a date time reflecting the expected wall time is a good thing. And probably a normal expectation.

The Etc/GMT±n timezones only support full hour offsets it seems.

Yes, which is why Etc/Unknown is important.

The only question I see is how we handle computations on Etc/Unknown . E.g. UTC-7 offset can be both MST or PDT(daylight saving for PST), so how would DateTime.shift(datetime_offset_minus_7, month: 2) act.

I think that DateTime.shift/2 is still valid for Etc/Unknown, because the result is still Etc/Unknown. The date and time components of Date, Time, DateTime and NaiveDateTime are always wall time values (unless I’m mistaken - possible!). Therefore we can still shift them and still be clear we don’t know the zone.

DateTime.shift_zone/2 would not be possible because the source zone is unknown and an error would be returned.

In the end this is a bit of a weird case because we technically have offsets / some notion of a timezone, but any calculations (instead of derivations) made on such datetimes would essentially map to NaiveDateTime in terms of correctness properties.

The fact we have a wall time and offset but we don’t know the time zone name makes this different from a NaiveDateTime. The positive lack of information (a zone name) is still information. And we know the offset which is useful in many cases.

5

This “offset only” DateTime operates quite differently to ones where the timezone is known (unless it happens to be an Etc/GMT+x zone). In the current state, you can always reconstruct the offset if you need it from the time zone. (i.e. you can pull out a NaiveDateTime and a time zone name, and then put them back together reliably.) Effectively the offset is just there as an optimisation.

With this change you’d have to make sure the offset travels alongside the time zone at all times; and then you need to special case Etc/Unknown and use the offset. It feels like the sort of thing that could just keep dropping new bugs for a while.

Perhaps another module (DateTimeWithOffset or something) could encompass this idea? It’s an approach that the Java time classes take.

2 Likes