toraritte

toraritte

Why can't `:timestamptz` be set up as default timestamp for migrations in config?

Hi,

Trying to make :utc_datetime the global default for Ecto schemas and migrations in our app, and use timestamptz as default datetime type in PostgreSQL. Was able to figure these out, but there seems to be inconsistency around migration configurations - and because I don’t understand what’s happening, having a hard time letting it go.

Schemas

Making :utc_datetime the default at most can only be done on a module-per-module basis, but at least it’s straightforward.

defmodule ANV.Accounts.User do

  use Ecto.Schema

  # EITHER
  @timestamps_opts [type: :utc_datetime]

  schema "users" do

    field :username, :string

    # OR
    timestamps(type: :utc_datetime)
  end
end

Migrations

In migration: timestamps(type: :timestamptz)

Found the simplest solution in Time zones in PostgreSQL, Elixir and Phoenix by @hubertlepicki :

create table(:events) do
  add :title, :string

  timestamps(type: :timestamptz)
end

Which makes total sense, because Ecto.Migration.timestamps/1 will forward the type info to Ecto.Migration.add/3.

This solution also seems to automatically choose the right Elixir struct, which is DateTime, instead of the default NaiveDateTime, because if the corresponding schema timestamps are not set to :utc_datetime, any DB operations will fail.

Application-wide: migration_timestamps

From the Ecto.Migration docs:

  • :migration_timestamps - By default, Ecto uses the :naive_datetime type, but you can configure it via:

    config :app, App.Repo, migration_timestamps: [type: :utc_datetime]
    

Naively assumed that exchanging :utc_datetime to :timestamptz and simply using timestamps() in the migration would do the trick (based on how Ecto.Migration.timestamps/1 is implemented), but got an error message instead (see below).

It also seems superfluous to use a global config with :utc_datetime, because one would have to provide :timestamptz in the migration anyway, simply overwriting the global setting.

I assume that providing the type directly to timestamps/1 goes through the Postgrex adapter, setting the correct Elixir type, but using config has a side effect somewhere along the line.

** (DBConnection.EncodeError) Postgrex expected %DateTime{}, got ~N[2019-10-01 17:45:34]. Please make sure the value you are passing matches the definition in your table or in your query or convert the value accordingly.
    (postgrex) lib/postgrex/type_module.ex:897: Postgrex.DefaultTypes.encode_params/3
    (postgrex) lib/postgrex/query.ex:75: DBConnection.Query.Postgrex.Query.encode/3
    (db_connection) lib/db_connection.ex:1148: DBConnection.encode/5
    (db_connection) lib/db_connection.ex:1246: DBConnection.run_prepare_execute/5
    (db_connection) lib/db_connection.ex:1342: DBConnection.run/6
    (db_connection) lib/db_connection.ex:540: DBConnection.parsed_prepare_execute/5
    (db_connection) lib/db_connection.ex:533: DBConnection.prepare_execute/4
    (postgrex) lib/postgrex.ex:198: Postgrex.query/4
    (ecto_sql) lib/ecto/adapters/sql.ex:658: Ecto.Adapters.SQL.struct/10
    (ecto) lib/ecto/repo/schema.ex:649: Ecto.Repo.Schema.apply/4
    (ecto) lib/ecto/repo/schema.ex:262: anonymous fn/15 in Ecto.Repo.Schema.do_insert/4
    priv/repo/seeds.exs:21: anonymous fn/2 in :elixir_compiler_1.__FILE__/1
    (elixir) lib/enum.ex:1948: Enum."-reduce/3-lists^foldl/2-0-"/3
    priv/repo/seeds.exs:16: (file)
    (elixir) lib/code.ex:813: Code.require_file/2

In the end, I don’t mind the extra typing, and timestamps(type: :timestamptz) explicitly conveys type in every migration, but still curious what I am missing.

Thanks!

Most Liked

josevalim

josevalim

Creator of Elixir

Naively assumed that exchanging :utc_datetime to :timestamptz and simply using timestamps() in the migration would do the trick (based on how Ecto.Migration.timestamps/1 is implemented), but got an error message instead (see below).

This should work. However, if you update your migrations, you must update your schema. So a combination of:

config :app, App.Repo, migration_timestamps: [type: :timestamptz]

And @timestamps_opts [type: :utc_datetime] is enough for everything.

Ecto does not automatically figure out which type to use from the migration/database, exactly because a single database type may be one or more Elixir types. Case in point, utc_datetime can be used for both timestamp and timestamptz. The way timestamptz works is that the DB converts to UTC. The way utc_datetime works is that Elixir’s adapter converts to UTC before writing to the database. So since utc_datetime already guarantees the data will be normalized, you don’t strictly need timestamptz.

16
Post #2
champeric

champeric

I’ll chime in, because this is a subject that always annoyed me about how it was handled by default and I need to manually change things to my taste (I’m using PostgreSQL).

First, I want my timestamps to be timestamptz in the DB. Forget that it’s badly named (confusion about timezone), it is the correct type to represent a “point in time”. If your DB uses timestamp (without tz), it’s ambiguous and someone needing to interface with the DB outside your app won’t have the slightest clue what they represent (is it local time (ouch…) or UTC?). Using the correct type makes confusion impossible.

Anyway, most of the things I change have been mentioned in this thread.

Make sure migration timestamps are timestamptz by default and have a default, which makes it easier to insert rows in DB manually for testing. It’s important that the default timezone is set to UTC (it could be set in the DB, but I prefer to have my system work either way. It’s important because Ecto was doing some casting where where: token.inserted_at > ago(^days, "day") was converted to SQL inserted_at > $2::timestamp + (interval ...) and the only this conversion works correctly is if the default timezone is UTC.

config :my_app, MyApp.Repo,
  migration_primary_key: [type: :identity],
  migration_timestamps: [type: :timestamptz, default: {:fragment, "now()"}],
  after_connect: {Postgrex, :query!, ["SET TimeZone TO 'UTC';", []]}

As already mentioned, I have a custom schema:

defmodule MyApp.Schema do
  defmacro __using__(_) do
    quote do
      use Ecto.Schema
      @timestamps_opts [type: :utc_datetime_usec]
    end
  end
end

I don’t use the generators much, but still I have a private copy of the phx.gen.schema templates.
schema.ex, make it use my schema:

defmodule <%= inspect schema.module %> do
  use <%= hd(Module.split(schema.repo)) %>.Schema

migration.exs, make sure fields with utc_datetime and utc_datetime_usec are converted correctly to timestamptz, and at the same time make sure text is always used because I can’t stand the sight of varchar(255):

<%
internal_type_mapper = fn
  :string -> :text
  :utc_datetime_usec -> :timestamptz
  other -> other
end

type_mapper = fn
  :utc_datetime -> ":timestamptz, size: 0"
  {:array, :utc_datetime} -> "{:array, :timestamptz}, size: 0"
  {:array, subtype} -> inspect({:array, internal_type_mapper.(subtype)})
  other -> inspect(internal_type_mapper.(other))
end
%>defmodule <%= inspect schema.repo %>.Migrations.Create<%= Macro.camelize(schema.table) %> do
...
... <%= type_mapper.(Mix.Phoenix.Schema.type_for_migration(v)) %> ...
toraritte

toraritte

I was also wrong about this one, it is in plain sight in the docs:

# Define a module to be used as base
defmodule MyApp.Schema do
  defmacro __using__(_) do
    quote do
      use Ecto.Schema
      @primary_key {:id, :binary_id, autogenerate: true}
      @foreign_key_type :binary_id
    end
  end
end

# Now use MyApp.Schema to define new schemas
defmodule MyApp.Comment do
  use MyApp.Schema

  schema "comments" do
    belongs_to :post, MyApp.Post
  end
end

Where Next?

Popular in Questions Top

albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
RisingFromAshes
I’ve read in another post that it may be possible with a router helper - but I couldn’t find an appropriate one, and tbh, I’m still just ...
New
script
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
New
vegabook
I’m brand new to Phoenix and I have stripped one of the demo applications to the bone. I just want to get an svg up on the screen. Here i...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

Other popular topics Top

Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID&lt;0.412.0&gt; terminating ** (Postgrex.Error) FATAL...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
AngeloChecked
What learn first? Rust or Elixir Hi Elixir community! I’m here because i want learn a new language. I’m a junior developer and mainly i ...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 36352 110
New
rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list. ...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
svb
Hi! Currently I want to submit a form by pressing the Enter key. However, since my input field is of type “textarea” this is just adds a...
New

Latest on Elixir Forum

We're in Beta

About us Mission Statement