My production (hobby) system started encountering errors tonight which look to be related to the time change. Heres the error:
** (ArgumentError) cannot convert ~N[2023-03-12 02:17:24] to datetime because such instant does not exist in time zone America/New_York as there is a gap between #DateTime<2023-03-12 01:59:59.999999-05:00 EST America/New_York> and #DateTime<2023-03-12 03:00:00-04:00 EDT America/New_York>
[info] (elixir 1.14.1) lib/calendar/datetime.ex:629: DateTime.from_naive!/3
[info] (flames 0.7.0) lib/flames_web/dashboard/helpers.ex:52: Flames.Dashboard.Helpers.display_timestamp/1
[info] (flames 0.7.0) lib/flames_web/dashboard/errors_live.ex:82: anonymous fn/4 in Flames.Dashboard.ErrorsLive.render/1
[info] (elixir 1.14.1) lib/enum.ex:2468: Enum."-reduce/3-lists^foldl/2-0-"/3
[info] (flames 0.7.0) lib/flames_web/dashboard/errors_live.ex:56: anonymous fn/2 in Flames.Dashboard.ErrorsLive.render/1
[info] (phoenix_live_view 0.18.16) lib/phoenix_live_view/diff.ex:398: Phoenix.LiveView.Diff.traverse/7
[info] (phoenix_live_view 0.18.16) lib/phoenix_live_view/diff.ex:139: Phoenix.LiveView.Diff.render/3
[info] (phoenix_live_view 0.18.16) lib/phoenix_live_view/static.ex:252: Phoenix.LiveView.Static.to_rendered_content_tag/4
The code which does this is located here:
In this case, it appears that the timezone cannot be converted because it lands in between the shift. Wouldn’t the appropriate action here, rather than a failure which occurs for an hour once a year (which is incredibly difficult to predict), be to just add 1 hour since it seems to already know that 2am goes to 3am and anything between 2am and 3am is actually just 3am+.
Why does the standard library put an exception that can happen so rarely like this? My system is down for the next 35 minutes I suspect until UTC time is past 3am.
First, sorry for the production issues, that’s always unpleasant.
The thing is though, the standard library does not know that adding an hour is a valid interpretation of the data in the general case. It might be in your case, but you could just as easily be here on the forum reading a post by someone else about how the standard library silently advances inputs an hour during DST which leads to invalid duration logic (for example). Standard libraries should, in my opinion, err on the side of correctness, particularly in cases like date and time logic where there’s a lot of essential complexity.
As for how you would handle this, I would question where the datetime applied to display_timestamp comes from in the first place. Where are you getting the data for this instant in time given that, as Elixir notes, it does not exist?
I think the issue here was I was assuming the date is already in the timezone when it was actually UTC. It came from the database so I think thats the issue. I understand what you’re saying that it may not be correct in all cases. From the looks of things once I dug in, tzdata is at fault, not the standard library. It looks up to try to find this time and gets nothing back here:
I guess this is caused by my mistake to begin with since I was wrong about what the actual timezone was in this case but seeing as how these kinds of things happen very rarely its not likely something anyone will anticipate. Timezones are always hard.
Instead of assuming, a better approach is to read UTC timestamp from the DB into a DateTime struct, which carries the information that the time is in UTC. In this way you’re always dealing with a well-defined point in time that must exist in any time zone. Then you can call DateTime.shift_zone! with no fear.
Also note that if you call DateTime.from_naive instead of DateTime.from_naive! you will get an appropriate result even when dealing with times that are ambiguous in the given time zone, so you can react to it.
But the UTC-based approach is the easiest solution IMO.
I’ve gone through four iterations of storing dates, times, and timezones in my DB and finally learned about “wall time” from this great discussion. Strongly recommend reading through that and seeing if like me it would make more sense for you to store your dates as naive with a separate timezone field.
Wall time is kind of a special case though. If you’re only interested in storing a specific point in time (such as the timestamp from a past event) as opposed to “4pm in New York on a certain future date”, then you don’t need to store the time zone.
Taking a look at the OP example, it seems to me that his application is storing timestamps.
Yeah, I think you’re right. I thought because of their code dealing with the timezone that they cared specifically about it but I think I just misread it.
Nitpick: none of the functions used in the code above do any “converting” - you won’t find a single + or - anywhere. All they do is add and remove the timezone information to change the shape of data between NaiveDateTime and DateTime.
So it is actually storing it using Ecto as a :utc_datetime. But in an effort to support other cases, you can see above on line 42 of the code linked in the original post, it calls display_timestamp again. It seems this was the problem. 2:30 AM in UTC time on that day, dropping timezone info, followed by converting to a DateTime using timezone info from configuration rather than doing a shift in timezone directly from UTC. As @al2o3cr points out, nothing is added or removed:
I realize now this method is also probably the wrong time and not correctly adjusted for the configured timezone. Since this is an error dashboard used only by myself I never really verified the timestamps were off by a few hours since everything that occurs usually happens in the past. Heres what the dashboard looks like for those who are curious:
