francesco.pessina

francesco.pessina

Kafka Ex :timeout error on connector

Hello everyone :slight_smile:
I have a strange problem with Kafka Ex (GitHub - kafkaex/kafka_ex: Kafka client library for Elixir · GitHub). While running locally and connecting to a local Kafka instance, all is fine.
When deploying the application on a Kubernetes AWS cluster, I experience a weird error. The Kafka broker is an AWS managed one, and I’m sure it is reachable because other microservices (Java) can connect.

This is the error I’m experiencing:

07:05:21.853 [error] GenServer #PID<0.4672.0> terminating                                                                                                                                                │
│ ** (MatchError) no match of right hand side value: [async_commit: nil]                                                                                                                                   │
│     (kafka_ex 0.12.1) lib/kafka_ex/gen_consumer.ex:774: KafkaEx.GenConsumer.handle_message_set/2                                                                                                         │
│     (kafka_ex 0.12.1) lib/kafka_ex/gen_consumer.ex:668: KafkaEx.GenConsumer.handle_info/2                                                                                                                │
│     (stdlib 3.12.1) gen_server.erl:637: :gen_server.try_dispatch/4                                                                                                                                       │
│     (stdlib 3.12.1) gen_server.erl:711: :gen_server.handle_msg/6                                                                                                                                         │
│     (stdlib 3.12.1) proc_lib.erl:249: :proc_lib.init_p_do_apply/3                                                                                                                                        │
│ Last message: :timeout 

and this is my broker and consumer group configuration:

# Kafka brokers configuration
config :kafka_ex,
  brokers: "csv list of the brokers"

config :my_app, :children, [
  %{
    id: "consumer",
    start:
      {KafkaEx.ConsumerGroup, :start_link,
       [
         MyApp.Kafka.Consumer,
         System.get_env("KAFKA_CONSUMER_GROUP"),
         [<topic1>,<topic2>],
         [
           # setting for the ConsumerGroup
           heartbeat_interval: 1_000,
           # this setting will be forwarded to the GenConsumer
           commit_interval: 1_000,
           auto_offset_reset: :latest
         ]
       ]}
  }
]

The application is running in a Docker image built from elixir.1.11-alpine with OTP 23.

I don’t understand what’s happening :frowning:

First Post!

francesco.pessina

francesco.pessina

Nevermind, the problem may be some conflicting process… Rollbacking to a previous commit solved the issue :slight_smile:

Where Next?

Popular in Questions Top

New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
JulienCorb
I am trying to implement my new.html.eex file to create new posts on my website. new.html.eex: &lt;h1&gt;Create Post&lt;/h1&gt; &lt;%= ...
New
LegitStack
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New
script
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

Other popular topics Top

vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
274 41539 114
New
johnnyicon
Hi all, I’ve just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I’m trying to use Postgres...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? Ecto.Repo — Ecto v3.14.0 has exampl...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
dokuzbir
I want to highlight html closing tags when i click a html tag. That works in .html files but doesnt work for html.eex templates. How can...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New

We're in Beta

About us Mission Statement