Postgrex ArgumentError when updating to elixir 1.14 and otp25.1

I’m updating our services to elixir 1.14 and it has been going well. However, I’m running into an issue with one of them. Whenever I try to deploy I get the following error:

** (exit) exited in: :gen_statem.call(#PID<0.3194.0>, {:start, 15000}, :infinity)
    ** (EXIT) an exception was raised:
        ** (ArgumentError) errors were found at the given arguments:

  * 1st argument: not a binary

            :erlang.binary_to_list('..data/DB_SERVER_CA')
            (ssl 10.8.4) ssl_config.erl:366: :ssl_config.file_error/2
            (ssl 10.8.4) ssl_config.erl:48: :ssl_config.init/2
            (ssl 10.8.4) ssl_gen_statem.erl:166: :ssl_gen_statem.ssl_config/3
            (ssl 10.8.4) tls_connection.erl:151: :tls_connection.init/1
            (stdlib 4.1) proc_lib.erl:240: :proc_lib.init_p_do_apply/3

After several unsuccessful attempts of deploying due to this error, the deploy starts to rollback and I get these errors:

** (exit) {:EXIT,
     {%Protocol.UndefinedError{
        protocol: Jason.Encoder,
        value: {0, 0, 0, 0, 0, 65535, 32512, 6},
        description: "Jason.Encoder protocol must always be explicitly implemented"
      },
      [
        {Jason, :encode_to_iodata!, 2, [file: 'lib/jason.ex', line: 213, error_info: %{module: Exception}]},
        {LoggerJSON, :format_event, 5, [file: 'lib/logger_json.ex', line: 286]},
        {LoggerJSON, :buffer_event, 5, [file: 'lib/logger_json.ex', line: 227]},
        {LoggerJSON, :handle_event, 2, [file: 'lib/logger_json.ex', line: 126]},
        {:gen_event, :server_update, 4, [file: 'gen_event.erl', line: 802]},
        {:gen_event, :server_notify, 4, [file: 'gen_event.erl', line: 784]},
        {:gen_event, :server_notify, 4, [file: 'gen_event.erl', line: 786]},
        {:gen_event, :handle_msg, 6, [file: 'gen_event.erl', line: 526]}
      ]}}
Protocol.UndefinedError: protocol Jason.Encoder not implemented for {0, 0, 0, 0, 0, 65535, 32512, 6} of type Tuple, Jason.Encoder protocol must always be explicitly implemented. This protocol is implemented for the following type(s): Any, Atom, BitString, Date, DateTime, Decimal, Ecto.Association.NotLoaded, Ecto.Schema.Metadata, Float, Integer, Jason.Fragment, Jason.OrderedObject, List, Map, NaiveDateTime, Time
  File "lib/jason.ex", line 213, in Jason.encode_to_iodata!/2
  File "lib/logger_json.ex", line 286, in LoggerJSON.format_event/5
  File "lib/logger_json.ex", line 227, in LoggerJSON.buffer_event/5
  File "lib/logger_json.ex", line 126, in LoggerJSON.handle_event/2
  File "gen_event.erl", line 802, in :gen_event.server_update/4
  File "gen_event.erl", line 784, in :gen_event.server_notify/4
  File "gen_event.erl", line 786, in :gen_event.server_notify/4
  File "gen_event.erl", line 526, in :gen_event.handle_msg/6

our config looks like this:

config :my_app, MyApp.Repo,
    database: System.fetch_env!("DB_NAME"),
    username: C.fetch_secret!("DB_USERNAME"),
    password: C.fetch_secret!("DB_PASSWORD"),
    hostname: System.fetch_env!("DB_HOSTNAME"),
    port: System.get_env("DB_PORT", "5432") |> C.integer(),
    pool_size: System.get_env("POOL_SIZE", "20") |> C.integer(),
    queue_interval: System.get_env("DB_QUEUE_INTERVAL", "5000") |> C.integer(),
    queue_target: System.get_env("DB_QUEUE_TARGET", "1000") |> C.integer(),
    ssl: System.get_env("DB_SSL_BOOL", "true") |> C.boolean(),
    ssl_opts: [
      cacertfile: "/etc/secrets/DB_SERVER_CA",
      keyfile: "/etc/secrets/DB_CLIENT_KEY",
      certfile: "/etc/secrets/DB_CLIENT_CERT",
      versions: [:"tlsv1.2"]
    ]

Does anyone have any ideas of what might be going wrong?

That’s a Jason thing. Did you try changing your json library to Poison?

https://hexdocs.pm/postgrex/readme.html

config :postgrex, :json_library, Poison

The first error looking a LOT like config makes me wonder if there’s something incorrect loitering around in compiled form, like it was incorrectly configured in the past and hasn’t been recompiled. :thinking:

The second error matches reports on the LoggerJSON issue tracker, of an issue fixed in 4.2.0:

What version of LoggerJSON is currently in use?

It wasn’t a Jason error, it was a bug in how otp 25 handles symlinks. José pointed it out here: `ArgumentError` when trying to upgrade to elixir 1.14 and otp25 · Issue #621 · elixir-ecto/postgrex · GitHub

We updated to otp 25.1.1 and all errors were resolved!