maxs

maxs

Ecto and timezones

I’m trying to understand what is the ‘correct’ way to deal with timezones using Postgres 9.6, Ecto 2.1 and Elixir 1.4. The landscape is a little confusing, with the old Ecto types, the latest Elixir types, and libraries like Calecto and Timex.

My app needs to save events that happen at a certain ‘wall time’ in the future in a specific time zone, e.g. “9pm on January 1st 2018, in New York”. Different events can have different time zones.

How should I store this in the database? Should I save the time in UTC (:utc_datetime) along with timezone name “America/New_York” (:text) in another column? That is 2018-01-02T02:00:00Z, but what if the definition for America/New_York gets redefined to a new offset [I know it’s highly unlikely, but it can theoretically happen]? It’ll no longer be 9pm ‘wall time’ in NY.

Or should I be storing the date and time as a :naive_datetime (would I save the local time, “2018-01-01T21:00:00”)? How do I then query a list of (different time zone) events chronologically (by actual time, rather than wall clock time)?

For both options, how do I write the following queries in Ecto?
“All events that occur on the 1st of January 2018 (in their associated time zone)?”
“All events that occur within an hour of 9pm in New York on January 1st 2018?”

Thanks in advance for any pointers as to the ‘official’ ways to do this in the Ecto/Elixir world! I’d love to contribute with a blog post/docs once I figure all this out :slight_smile:

First 5 of 5 Posts Switch mode

josevalim

josevalim

Creator of Elixir

Use the Calecto library. The trouble with any datetime in a given timezone in the future is that you don’t know quite exactly when it will happen, given the timezone rules may change any time. The Calecto has proper data types to handle scenarios like that.

Qqwy

Qqwy

TypeCheck Core Team

If the definition of a timezone changes, then you should update the times that are stored in the database manually. After all, there are two equally likely possibilities:

  • The stored timestamps should be kept -5 hours relative to UTC.
  • The stored timestamps should be changed to reflect the new wall time offset in the America/New York timezone.

So there is some manual work involved when this happens. But that should not be a problem as this event is quite unlikely (so there is no reason to automate it).

I would advise against storing naive datetimes as you have all the information available to create complete datetimes (namely: you know the timezone they were made in), so you’d be throwing information away that you want to use later during searching. Storing timestamps from multiple timezones as naive datetimes in the same database column will make sorting chronologically and also querying (i.e. ‘all times before 2018-02-02T00:00:00+00’) a hassle (needing multiple or really complex queries).

So, I’d suggest storing your times as UTC-relative times; be that as the Calecto DateTime type or a UNIX timestamp + timezone name. This will also make the following easier:

  1. will require multiple queries (or a single complex query). I think the way to go would be to get the list of active timezones at that specific date (i.e. 2018-01-01) from the Tzdata library that both Calendar and Timex use, and then construct the different time offsets to compare database values against them from this list.

  2. might be simpler, as you only have a single time offset. If you work with UNIX timestamps, this can be queried very easily from within Ecto. If you work with the Calecto.DateTime values, I’m not sure if there are abstractions you can use to make this a nice query in Ecto, or if you have to resort to writing raw SQL.
    Maybe @lau, who wrote both Calendar and the Calecto wrapper, can shine some light on this.

That would be great! I think this is a very interesting problem whose solution might help many people.
We’re talking a lot about how to make the usage of calendars, dates and times more comprehensible and useful in Elixir. Questions like these are very important for this discussion, I think :smiley: .

Lau

Lau

but what if the definition for America/New_York gets redefined to a new offset [I know it’s highly unlikely, but it can theoretically happen]

It’s not that unlikely. It’s only been 10 years since there was changes to the New York time zone (along with the rest of USA). And if you look at the whole world there updates with changes coming out on an average of almost once a month.

In Calecto you can use the type Calecto.DateTime if you use Postgres. What it does is save a DateTime as a composite field so that you have both the “wall time” and the name of the time zone.

Read more about it here: How to save datetimes for future events - (when UTC is not the right answer)

For this query:
“All events that occur on the 1st of January 2018 (in their associated time zone)?”
Since the datetime is saved in wall time, you do a normal query on the wall time component of the composite field. This is relatively simple.

This query is a bit more tricky:
“All events that occur within an hour of 9pm in New York on January 1st 2018?”

Postgres does have timezone conversion functionality built in, but it’s timezone data is not updated automatically so it can become outdated and out of sync with the time zone data in Elixir, which is kept up to date automatically by default. What you can do is do a query that takes in “too many” records and then afterwards you can filter them in Elixir. E.g. do a query like above based on the wall time but include records that are well before and after (let’s say 30 hours before and after) the wall time, and then in Elixir you can use e.g. Enum.filter and provide a function that check if the DateTime is within the hour you want.

Lau

Lau

The time zones change often. Last year there was 10 new data releases! So with this solution you would have to check / update the database manually 10 times a year.

If you save only the datetime converted to UTC and not the wall time, you will use valuable information. If the time zone definition changes, it could be difficult or impossible to know what the intended wall time was.

This blog post has more information How to save datetimes for future events - (when UTC is not the right answer)

Qqwy

Qqwy

TypeCheck Core Team

Thank you very much for your reply and this in-depth blog post.
I am very happy to admit that my earlier suggestion to (only) store UTC-relative times thus is flawed, and that I have learned something today. :smiley:

— All posts loaded —

Where Next?

Trending in Questions Top

stjefim
Hello! Suppose you are building workflow (order / task / payment) processing system with the following requirements: Each workflow con...
New
jonnycharles
I’m in search of an Elixir library that offers PDF generation capabilities similar to Ruby’s Prawn. While there have been discussions abo...
New
spammy
I’m looking to build a personal workflow to quickly deploy web applications written in elixir/phoenix, for local consumption (ie not on t...
New
silverdr
Using Phoenix.LiveView.TagEngine as an EEx.Engine is deprecated! To compile HEEx, use Phoenix.LiveView.TagEngine.compile/2 instead. Sta...
New
dli
Before I dive in myself, did anyone successfully sprinkle Hologram into their existing LiveView app? Looking for hints regarding: Addi...
New
bottlenecked
Hi all, I wanted to ask how the community is dealing with post-release steps. Today we have Ecto migrations, which make sure that the db...
New
michallepicki
I am using Oban and occasionally, shortly after a deployment, a handful of jobs can fail because of dependency on other parts of the syst...
New

Other Trending Topics Top

JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve. They are GUI (Emerge) and State management (S...
New
ausimian
Emily is an Elixir library that runs Nx computations on Apple’s MLX. Install it as the default Nx backend and Nx, defn, Axon, Nx.Serving,...
New
type1fool
I just stumbled on a newly redesigned elixir-lang.org. :tada: It looks like @Software_Mansion did the work, and I think it is generally a...
New
juhalehtonen
There has been a thread to discuss the Stack Overflow Developer Survey on this forum every year since 2018, so here’s yet another one for...
New

We're in Beta

About us Mission Statement