Upgrading to ecto 3 and ecto_sql: no ecto_sql OTP application running

Hello,

I’m updating a Hex package to work with Ecto 3 and the new ecto_sql package. I’ve added ecto_sql as an optional dependency and I’m working with Elixir 1.7.4 and OTP 21.1.

I’ve noticed that the ecto_sql and telemetry applications don’t start automatically, at least in dev and test. The result is that the queries are executed, but then an ArgumentError is raised when ecto tries to report the query, and the query SQL is in fact not logged:

02:08:07.348 [error] an exception was raised logging %DBConnection.LogEntry{call: :prepare_execute, connection_time: 1520000, decode_time: 1381000, params: ["foobar"], pool_time: 2910000, query: %Postgrex.Query{cache: :reference, columns: nil, name: "ecto_8002", param_formats: nil, param_oids: nil, param_types: nil, ref: nil, result_formats: nil, result_oids: nil, result_types: nil, statement: "SELECT f0.\"id\", f0.\"flag_name\", f0.\"gate_type\", f0.\"target\", f0.\"enabled\" FROM \"fun_with_flags_toggles\" AS f0 WHERE (f0.\"flag_name\" = $1)", types: nil}, result: {:ok, %Postgrex.Query{cache: :reference, columns: ["id", "flag_name", "gate_type", "target", "enabled"], name: "ecto_8002", param_formats: [:binary], param_oids: [25], param_types: [Postgrex.Extensions.Raw], ref: #Reference<0.3901140065.137101316.184360>, result_formats: [:binary, :binary, :binary, :binary, :binary], result_oids: [20, 1043, 1043, 1043, 16], result_types: [Postgrex.Extensions.Int8, Postgrex.Extensions.Raw, Postgrex.Extensions.Raw, Postgrex.Extensions.Raw, Postgrex.Extensions.Bool], statement: "SELECT f0.\"id\", f0.\"flag_name\", f0.\"gate_type\", f0.\"target\", f0.\"enabled\" FROM \"fun_with_flags_toggles\" AS f0 WHERE (f0.\"flag_name\" = $1)", types: {Postgrex.DefaultTypes, #Reference<0.3901140065.137232385.188074>}}, %Postgrex.Result{columns: ["id", "flag_name", "gate_type", "target", "enabled"], command: :select, connection_id: 15144, messages: [], num_rows: 0, rows: []}}}: ** (ArgumentError) argument error
    (stdlib) :ets.lookup(Telemetry.HandlerTable, [:fun_with_flags, :dev, :ecto_repo, :query])
    lib/telemetry/handler_table.ex:59: Telemetry.HandlerTable.list_for_event/1
    lib/telemetry.ex:76: Telemetry.execute/3
    lib/ecto/adapters/sql.ex:756: Ecto.Adapters.SQL.log/4
    (db_connection) lib/db_connection.ex:1303: DBConnection.log/5
    lib/ecto/adapters/sql.ex:531: Ecto.Adapters.SQL.execute!/4
    lib/ecto/adapters/sql.ex:523: Ecto.Adapters.SQL.execute/5
    (ecto) lib/ecto/repo/queryable.ex:147: Ecto.Repo.Queryable.execute/4

Of course, adding ecto_sql or telemetry to the list of extra_applications in the library mix file solves the problem (it’s configured to only add it in dev and test).
As expected, adding ecto_sql starts both, and adding telemetry only starts telemetry.

Still, I’m wondering why they’re not starting on their own. I suppose it’s because ecto_sql is an optional dependency, and I’m looking for more info on how this works.

Also, and I find this quite interesting, it looks that only telemetry is required to run for things to work. In fact I can start only the telemetry application and all queries will work even though the ecto_sql application is not running.

I imagine that this could be some Mix behaviour that I’m not aware of, and this will turn out to be an Elixir question rather than a library question. :slight_smile:

It should not be the case unless you are providing the applications list as well. If you do, the list will not be automatically assembled. I think (from top of my head, I looked at this Elixir code a while ago) that it figures out the list of applications either automatically (if not provided) or takes what you declared, then adds extra_applications. And the default list of applications would have all appropriate dependencies, so ecto should be there.

So the question is: are you providing applications list explicitly in your mix file?

As it’s explained here Confusing behavior of `optional` deps in `mix.exs` optional dependencies are not started automatically.

4 Likes

Thanks. Where can I find some documentation (or code) about this?

Update: it looks like this behaviour will change in Elixir 1.8: https://github.com/elixir-lang/elixir/pull/8263

That PR was actually reverted: https://github.com/elixir-lang/elixir/commit/9de51f88ca852e24af61420d1c3f12993c8680f0