vishal-h

vishal-h

Broadway kafka - failed to join group - reason: {:nxdomain

Hi,

I am getting the following error while connecting to kafka running in docker and port forwarded to localhost. I can shell into the docker and get topics etc. I’ve tested localhost:9092 using kafkacat and am able to list the topics etc.

00:52:17.966 [info]  Group member (some_group,coor=#PID<0.261.0>,cb=#PID<0.256.0>,generation=0):
failed to join group
reason: {:nxdomain,
 [
   {:kpro_connection, :connect, 4,
    [
      file: '/home/bw/deps/kafka_protocol/src/kpro_connection.erl',
      line: 208
    ]},
   {:kpro_connection, :init, 4,
    [
      file: '/home/bw/deps/kafka_protocol/src/kpro_connection.erl',
      line: 170
    ]},
   {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 240]}
 ]}

00:52:17.967 [info]  Group member (some_group,coor=#PID<0.261.0>,cb=#PID<0.256.0>,generation=0):
Leaving group, reason: :shutdown

Config is

producer: [                               
  module: {BroadwayKafka.Producer, [      
    hosts: [localhost: 9092],             
    group_id: "some_group",               
    topics: ["some_topic"],                    
    partition: 0,                         
  ]},                                     
  concurrency: 1                          
],                                        
processors: [                             
  default: [                              
    concurrency: 10                       
  ]                                       
]

Marked As Solved

mgibowski

mgibowski

Alright, so it looks like some-kafka is the host you can not reach.

Your actual set up might differ depending on what you need, but I can offer you one possible configuration - it’s two env variables:

        KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
        KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092

In this way kafka should be accessible via port 29092 from the inside of the docker network, and via port 9092 from your computer.

Also Liked

mgibowski

mgibowski

nxdomain is a DNS error meaning the host name you are trying to connect to can not be resolved.

In your case it is not localhost, but rather a host provided by Kafka to your client after establishing the initial connection.

When starting the Kafka docker container, did you specify the value for the environment variable KAFKA_ADVERTISED_LISTENERS? In case yes, what was it?

You can also check what host Kafka redirects you to by running kafkacat with the -L flag:
kafkacat -b localhost:9092 -L

More details here:

Where Next?

Popular in Questions Top

joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New
Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I forese...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New

Other popular topics Top

Qqwy
Update: How to use the Blogs &amp; Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3271 130286 1222
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New

We're in Beta

About us Mission Statement