Broadway kafka - failed to join group - reason: {:nxdomain


I am getting the following error while connecting to kafka running in docker and port forwarded to localhost. I can shell into the docker and get topics etc. I’ve tested localhost:9092 using kafkacat and am able to list the topics etc.

00:52:17.966 [info]  Group member (some_group,coor=#PID<0.261.0>,cb=#PID<0.256.0>,generation=0):
failed to join group
reason: {:nxdomain,
   {:kpro_connection, :connect, 4,
      file: '/home/bw/deps/kafka_protocol/src/kpro_connection.erl',
      line: 208
   {:kpro_connection, :init, 4,
      file: '/home/bw/deps/kafka_protocol/src/kpro_connection.erl',
      line: 170
   {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 240]}

00:52:17.967 [info]  Group member (some_group,coor=#PID<0.261.0>,cb=#PID<0.256.0>,generation=0):
Leaving group, reason: :shutdown

Config is

producer: [                               
  module: {BroadwayKafka.Producer, [      
    hosts: [localhost: 9092],             
    group_id: "some_group",               
    topics: ["some_topic"],                    
    partition: 0,                         
  concurrency: 1                          
processors: [                             
  default: [                              
    concurrency: 10                       

nxdomain is a DNS error meaning the host name you are trying to connect to can not be resolved.

In your case it is not localhost, but rather a host provided by Kafka to your client after establishing the initial connection.

When starting the Kafka docker container, did you specify the value for the environment variable KAFKA_ADVERTISED_LISTENERS? In case yes, what was it?

You can also check what host Kafka redirects you to by running kafkacat with the -L flag:
kafkacat -b localhost:9092 -L

More details here:

1 Like

Thanks for your inputs. This is what I get …

$> kafkacat -b localhost:9092 -t some_topic -L
Metadata for some_topic (from broker -1: localhost:9092/bootstrap):
 1 brokers:
  broker 1001 at some-kafka:9092
 1 topics:
  topic "some_topic" with 1 partitions:
    partition 0, leader 1001, replicas: 1001, isrs: 1001

and env var

Alright, so it looks like some-kafka is the host you can not reach.

Your actual set up might differ depending on what you need, but I can offer you one possible configuration - it’s two env variables:


In this way kafka should be accessible via port 29092 from the inside of the docker network, and via port 9092 from your computer.