How to upgrade an app using timex_ecto to Phoenix 1.4 and Ecto 3.0

I have a application built on Phoenix 1.3. I use timex and timex_ecto in order to deal with datetimes with time zone.

  defp deps do
    [
      {:phoenix, "~> 1.3.3"},
      {:phoenix_pubsub, "~> 1.0"},
      {:phoenix_ecto, "~> 3.2"},
      {:postgrex, ">= 0.0.0"},
      {:phoenix_html, "~> 2.10"},
      {:phoenix_live_reload, "~> 1.0", only: :dev},
      {:gettext, "~> 0.11"},
      {:cowboy, "~> 1.0"},
      {:timex, "~> 3.1"},
      {:timex_ecto, "~> 3.2"}
    ]
  end

The schema module MyApp.Schedule.Item is defined as follows:

defmodule MyApp.Schedule.Item do
  use Ecto.Schema
  use Timex.Ecto.Timestamps

  schema "items" do
    field(:name, :string)
    field(:starts_at, Timex.Ecto.DateTime)
  end
end

In the priv/repo/seeds.exs, I inserts a record like this:

import MyApp.Repo
time0 = Timex.now("Asia/Tokyo") |> Timex.beginning_of_day()

insert!(%MyApp.Schedule.Item{
  name: "Example",
  starts_at: Timex.shift(time0, days: 1, hours: 10)
})

I need some adivice to upgrade my application to Phoenix 1.4 (rc.3) and Ecto 3.0.


Firstly, I removed timex_ecto from mix.exs because timex_ecto does not work with Ecto 3.0.

  defp deps do
    [
      {:phoenix, "~> 1.4.0-rc"},
      {:phoenix_pubsub, "~> 1.1"},
      {:phoenix_ecto, "~> 4.0"},
      {:ecto_sql, "~> 3.0-rc"},
      {:postgrex, ">= 0.0.0-rc"},
      {:phoenix_html, "~> 2.11"},
      {:phoenix_live_reload, "~> 1.2-rc", only: :dev},
      {:gettext, "~> 0.11"},
      {:jason, "~> 1.0"},
      {:plug_cowboy, "~> 2.0"},
      {:timex, "~> 3.1"}
    ]
  end

Secondry, I rewrote the MyApp.Schedule.Item module as follows:

defmodule MyApp.Schedule.Item do
  use Ecto.Schema

  schema "items" do
    field(:name, :string)
    field(:starts_at, :utc_datetime)
  end
end

Then, the priv/repo/seeds.exs came to cause this error:

** (ArgumentError) :utc_datetime expects the time zone to be "Etc/UTC", got `#DateTime<2018-11-03 11:00:00+09:00 JST Asia/Tokyo>`

Of course, to fix this error I can rewrite the script like this:

time0 = 
  Timex.now("Asia/Tokyo") 
  |> Timex.beginning_of_day()
  |> Timex.Timezone.convert("Etc/UTC")

insert!(%MyApp.Schedule.Item{
  name: "Example",
  starts_at: Timex.shift(time0, days: 1, hours: 10)
})

But, I think such a change is a little burdensome, because my app has quite a lot similar expressions.

So, taking advice in https://github.com/elixir-ecto/ecto/issues/2770#issuecomment-435140215, I created a custom Ecto type for my application:

defmodule MyApp.Ecto.DatetimeWithTimezone do
  @behaviour Ecto.Type
  def type, do: :datetime

  def cast(%DateTime{} = dt), do: {:ok, dt}
  def cast(_), do: :error

  def load(%NaiveDateTime{} = ndt), do: DateTime.from_naive(ndt, "Etc/UTC")
  def load(_), do: :error

  def dump(%DateTime{} = datetime) do
    case Timex.Timezone.convert(datetime, "Etc/UTC") do
      %DateTime{} = dt -> {:ok, dt}
      {:error, _} -> :error
    end
  end
  def dump(_), do: :error
end

Then, I made following modifications on my Model.User module:

defmodule MyApp.Schedule.Item do
  use Ecto.Schema
  # use Timex.Ecto.Timestamps           # <- Removed
  alias MyApp.Ecto.DatetimeWithTimezone # <- Added

  schema "users" do
    field(:name, :string, null: false)
    # field(:starts_at, Timex.Ecto.Datetime) # <- Removed
    field(:starts_at, DatetimeWithTimezone)  # <- Added
  ...

With these changes, my app works just like before.

Am I upgrading my app in a correct way? Or should I basically change its design?

6 Likes

Iā€™m hoping that we can continue to use timex_ecto to do this conversion, or something similar. Since Ecto 3.0 is brand new, perhaps we just need to wait for @bitwalker to update timex_ecto. If I understood it a bit better Iā€™d try to put together a PR to update it.

Iā€™m willing to hand maintainership of timex_ecto over to an interested party, but my own take is that with Ecto 3 timex_ecto should mostly be irrelevant, unless you were relying on some specific behavior (such as described in this post) - but even then, as demonstrated here, that can be worked around quite easily.

The only thing which timex_ecto does that is in any way unique is that it handles storing DateTimes with arbitrary timezones. The automatic conversion to UTC for cases where the underlying type requires it can be useful, but is a convenience that can be easily replicated with your own custom types. That said, I expect both of those (or at least the first) to also be deprecated when timezone support is added to Elixirā€™s calendar API, which I believe is an ongoing effort.

In the meantime though, I just donā€™t have the time to keep timex_ecto up to date, I have a few projects dominating virtually all of my time currently, so I would need some help from the community if we want to keep it up to date with Ecto 3. Reach out to me via issues in the repo, or DM me here, and Iā€™m happy to add someone as the new primary maintainer.

1 Like

Thanks for the quick reply @bitwalker, Iā€™m fine with using a custom type if itā€™s really that simple. Is @kurodaā€™s example above the correct way to do it? Do you have time to put that in a Gist thatā€™s linked from the timex_ecto README?

I donā€™t really get what makes Ecto 3.0 that different from previous versions as it relates to these datetime types, Iā€™m just interested in the simplest solution to keep using Ecto and Timex together :slight_smile:

@kurtome The approach above is fine, but probably shouldnā€™t be called DateTimeWithTimezone, since it doesnā€™t actually store the timezone, it just converts to and from UTC, but naming aside, yeah thatā€™s close enough. You can always refer to the current Timex.Ecto implementations for reference too.

Iā€™d be glad to merge a PR with a link! I just donā€™t have time to throw something together and test that it is correct at the moment :frowning: - normally I have a bit more free time, but Iā€™m more swamped than I have been in a long time, so my backlog has gotten a bit out of hand haha

I donā€™t really get what makes Ecto 3.0 that different from previous versions as it relates to these datetime types

Ecto 3.0 makes use of the new calendar types in Elixir, rather than its own Ecto.* calendar types. This is important because it means the last remaining piece needed to have Ecto handle everything natively (well, with a timezone library) is timezone support (which is under construction as far as Iā€™m aware). But if you donā€™t need to support arbitrary timezones in the database, Ecto 3 already supports NaiveDateTimes and DateTimes in UTC (as well as the other types of course). The bottom line is that Timex becomes a support library for the calendar types, but is no longer an active part of the Elixir/Ecto date/time interaction, since it is all natively supported :). This is ultimately one of the main goals of adding the Elixir calendar types, is to unify these things so that libraries are all mutually compatible with one another, without needing explicit adapters between them (which is effectively what Timex.Ecto was).

5 Likes

Just to clarify - Ecto uses the native elixir datetime types since Ecto 2.1 released almost 2 years ago. If the timex_ecto package is not needed now, it wasnā€™t needed back then as well. Itā€™s just that now the old custom ecto datetime types were removed after being deprecated for almost 2 years.

The things in this area that Ecto 3 changes are that the low-level ā€œrawā€ query interface also returns the elixir types instead of the erlang-style tuples and that we have the explicit separation of _usec and non-usec types.

6 Likes

One note on this subject - I migrated from using Timex to using naive datetimes when I upgraded after reading this thread. My services started erroring with data from my frontend that had been working prior to the upgrade. I use postgres for the database. timestamps were fine. the problem was in fields like ā€˜appointment datetimeā€™ - my frontend JS datetime pickers were not sending seconds ie ā€œ2018-12-12T16:00ā€ was sent - and the cast in my models was returning invalid date format errors. iso 8601 does not mandate seconds from what I read - but NaiveDateTime.from_iso8601 throws the exception, and the Timex usage/implementation did not error. So I rolled my ownNaiveDateTimeNoSeconds Ecto type to deal with this.

3 Likes