Kafka Ex :timeout error on connector

Questions / Help
, ,
1

Hello everyone :slight_smile:
I have a strange problem with Kafka Ex (GitHub - kafkaex/kafka_ex: Kafka client library for Elixir). While running locally and connecting to a local Kafka instance, all is fine.
When deploying the application on a Kubernetes AWS cluster, I experience a weird error. The Kafka broker is an AWS managed one, and I’m sure it is reachable because other microservices (Java) can connect.

This is the error I’m experiencing:

07:05:21.853 [error] GenServer #PID<0.4672.0> terminating                                                                                                                                                │
│ ** (MatchError) no match of right hand side value: [async_commit: nil]                                                                                                                                   │
│     (kafka_ex 0.12.1) lib/kafka_ex/gen_consumer.ex:774: KafkaEx.GenConsumer.handle_message_set/2                                                                                                         │
│     (kafka_ex 0.12.1) lib/kafka_ex/gen_consumer.ex:668: KafkaEx.GenConsumer.handle_info/2                                                                                                                │
│     (stdlib 3.12.1) gen_server.erl:637: :gen_server.try_dispatch/4                                                                                                                                       │
│     (stdlib 3.12.1) gen_server.erl:711: :gen_server.handle_msg/6                                                                                                                                         │
│     (stdlib 3.12.1) proc_lib.erl:249: :proc_lib.init_p_do_apply/3                                                                                                                                        │
│ Last message: :timeout

and this is my broker and consumer group configuration:

# Kafka brokers configuration
config :kafka_ex,
  brokers: "csv list of the brokers"

config :my_app, :children, [
  %{
    id: "consumer",
    start:
      {KafkaEx.ConsumerGroup, :start_link,
       [
         MyApp.Kafka.Consumer,
         System.get_env("KAFKA_CONSUMER_GROUP"),
         [<topic1>,<topic2>],
         [
           # setting for the ConsumerGroup
           heartbeat_interval: 1_000,
           # this setting will be forwarded to the GenConsumer
           commit_interval: 1_000,
           auto_offset_reset: :latest
         ]
       ]}
  }
]

The application is running in a Docker image built from elixir.1.11-alpine with OTP 23.

I don’t understand what’s happening :frowning:

2

Nevermind, the problem may be some conflicting process… Rollbacking to a previous commit solved the issue :slight_smile: