itssasanka

itssasanka

Hi all,

Trying to get some more clarity over utc_datetime and naive_datetime for Ecto:

The documentation above suggests that while NaiveDateTime doesn’t store any
TimeZone data associated with it, DateTime seems to, however the former, when used,
Ecto seems to store timestamps (inserted_at, updated_at) in according to UTC timezone anyway.
I read that using utc_datetime “ensures” that dates are always converted to utc before getting stored/retrieved from and to the database, however I am not sure what the significance of the “ensures” aspect here is, since naive_datetime seems to do it anyway.

And I also think storing timestamptz values in the database may not be really required, since I can just store utc timestamps everywhere and user timezone preferences in another table and use these two pieces of info to convert accordingly before presenting to the users.

Can someone throw some light on what the real world significance in using utc_datetime type for timestamps is? What am I missing here?

Showing Posts 1 to 10

LostKobrakai

LostKobrakai

You’re correct. timestamp columns are actually better for ecto (alone) as with timestamptz postgres will try to convert the timestamp to the clients timezone on its own. While for timestamp you can be sure postgres just leaves timestamps as is, with ecto handling all the timezone stuff at runtime using the settings/timezone db of elixir.

naive_datetime does store timestamps as UTC datetimes – that’s true. But that’s only “accidental” because it’s using NaiveDateTime.utc_now() to generate timestamps. If you manually set such fields to a different naive datetime you build from another timezone there’s no way for ecto to detect that. NaiveDateTime structs just don’t have a timezone component, which is the reason why they’re called naive.

For utc_datetime ecto will enforce that any saved datetime uses the Etc/UTC timezone. It’ll error otherwise. If you query the record you’ll also get back only datetimes with tthe timezone set to UTC. So the fact that datetimes use UTC changed from being implicit and prone to error with naive_datetime to explicit and enforced at runtime with utc_datetime.

17
Post #1
dimitarvp

dimitarvp

This only works for automatically generated fields like inserted_at and updated_at and if you manually set other datetime fields to DateTime.utc_now(). But if you have any other Elixir code that is not timezone-aware and it fetches time in the local timezone (or have a legacy DB with non-UTC datetimes in it) then you’re in for trouble.

Better to do all of these together:

  • Use utc_datetime type for datetime fields in your Ecto schema modules.
  • When setting times manually, always start with DateTime.utc_now() and/or do calculations with Elixir’s stdlib or timex, always making sure you use UTC.
  • If you have to work with an inherited legacy database, just make a one-off script to convert any non-UTC datetimes to UTC. It will save you a world of trouble in the future.
11
Post #2
wojtekmach

wojtekmach

Hex Core Team

Definitely use :utc_datetime_*. In hindsight, we wish we made that the default in Ecto but now we can’t due to backwards compatibility (and there are no plans for Ecto 4)

Here are some resources:

35
Post #3
itssasanka

itssasanka OP

Thank you all so much! this sums it up, once and for all !
So basically it’s just great that Ecto is enforcing saving DateTime values in utc with the utc_datetime, which I feel really positive about since it’s a standard development practice. I’ve seen no other library/framework enforcing it.

I made the following global changes to my Phoenix 1.5 app to use utc_datetime instead of naive_datetime by Ecto, both in schemas and migrations for anyone who may find it useful:

In config.exs, I added this:

config :myapp, MyApp.Repo, migration_timestamps: [type: :utc_datetime]

created a new schema.ex with the following code:

defmodule MyApp.Schema do
  defmacro __using__(_) do
    quote do
      use Ecto.Schema
      @timestamps_opts [type: :utc_datetime]
    end
  end
end

And in my models everywhere, I replaced use Ecto.Schema with use MyApp.Schema

And, done!

17
Post #4
dimitarvp

dimitarvp

I do almost the same:

@timestamps_opts [type: :utc_datetime_usec]

You likely will have cases when you would need sub-second precision. Even if you never do I’d think this extra precision won’t cost you almost anything in terms of disk space.

henrik

henrik

I started with :utc_datetime but switched to :utc_datetime_usec just because it got annoying having to do DateTime.utc_now() |> DateTime.truncate(:second) every time. I don’t really need the precision but it seemed to be less friction for the stuff I’ve done so far.

10
Post #6
itssasanka

itssasanka OP

Sounds great! then I’ll make this change in my app to get rid of this friction, nothin’ much to lose here anyway.

daduam

daduam

Why was the migration_timestamps change in your config.exs necessary?

dimitarvp

dimitarvp

Because IIRC having inserted_at and updated_at being at a microsecond precision is not the default.

itssasanka

itssasanka OP

Because Ecto.SQL.Migration needs that configuration. AFAIK it is a different module from Ecto.Schema

Where Next? Top

Trending in Questions Top

RSP87
I’m working on a project that simulates the bumbl example in the programming phoenix book. It acts almost like an email client. We have a...
New
kszambelanczyk
Hello! Could someone please give me a help/sample code, how to delete a file from s3 using waffle/waffle_ecto from Phoenix app. I creat...
New
RemyXRenard
I’m seeing that a list inside a Kino.DataTable will be interpreted as a charlist, even if the Kino.configure() is set to charlists: :as_l...
New
velrest
So my question is quite simple and i have found no conclusive answer on forum, google or AI. Should we use :erlang.float for Integer to ...
New
samoloth
Hi, I’ve just set up an application with ash_authentication. There is only magic link strategy for now, so there is no confirmation add o...
New
FlyingNoodle
If a change or preparation module uses Ash.Changeset.get_argument/2 or Ash.Query.get_argument/2 (or any of the other get_argument functio...
New
ryanwinchester
apply_graft/2 doesn’t rewrite an add_many sub-workflow’s deps on an add step. Grafted jobs cancel with “upstream job was deleted” Version...
New

Other Trending Topics Top

mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
marciok
Hi there! We created Gust: A task orchestrator inspired by Airflow. For those who have never heard about Aiflow, it’s a Python-based wor...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Dmk
Xamal is a deployment tool for Elixir apps that deploys native releases to bare metal servers over SSH. It’s a port of GitHub - basecamp/...
New
Damirados
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve. They are GUI (Emerge) and State management (S...
New
netoum
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews