| Author |
Message
|
| mqjeff |
 Posted: Mon Jun 26, 2017 7:03 am Post subject: |
 |
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| KafkaSolution wrote: |
| Quote: |
| You need three zookeepers. |
may i know what is the need of 3 zookeepers to connect with kafka node...? |
If you have not configured your one zookeeper to run in single server mode, then you need an odd number of zookeepers, and enough zookeepers so that a majority/quorum can be achieved. The minimum value for this is three.
Otherwise it sounds like your zookeeper is having trouble processing requests - likely due to resource constraints (memory, disk, cpu, etc.)
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
 |
| KafkaSolution |
| Posted: Tue Jun 27, 2017 3:27 am Post subject: |
|
|
Novice
Joined: 21 Jun 2017
Posts: 16
|
| we tried it with 3 zookeeper servers but we are unable to access the data from the Kafka node. We are getting the same exception. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Jun 27, 2017 4:22 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Ok, so it sounds like your zookeeper(s)) don't know where your kafka server is - that's the "broker".
Check the kafka logs, check the kafka configuration to make sure it knows where the zookeeper url is, try a kafka-topics --list command against one of the zookeepers from kafka machine.
It could possibly be the other way, that your kafka can't find your zookeeper(s). So check the zookeeper logs in more detail.
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| KafkaSolution |
| Posted: Fri Jul 07, 2017 2:08 am Post subject: |
|
|
Novice
Joined: 21 Jun 2017
Posts: 16
|
Can anyone Provide me the Documentation of Kafka node with Zookeeper and Kafka server in IIB ?  |
|
| Back to top |
|
|
| zpat |
| Posted: Fri Jul 07, 2017 2:29 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
|
| Back to top |
|
|
| KafkaSolution |
| Posted: Fri Jul 07, 2017 2:46 am Post subject: |
|
|
Novice
Joined: 21 Jun 2017
Posts: 16
|
Thank You zpat for the docs.
I have seen those documents but they have not provided how to work with IIB node using Zookeeper and Kafka server in our local system |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Jul 07, 2017 3:53 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| KafkaSolution wrote: |
Thank You zpat for the docs.
I have seen those documents but they have not provided how to work with IIB node using Zookeeper and Kafka server in our local system |
Generally when using Kafka, you don't directly connect to Zookeeper.
If you are having difficulty getting credentials configured, then as my esteemed colleague says, IBM documents their products
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| KafkaSolution |
| Posted: Tue Jul 11, 2017 6:55 am Post subject: |
|
|
Novice
Joined: 21 Jun 2017
Posts: 16
|
i am getting this exception while connecting with kafka server and zookeeper server when connecting to my kafka producer and kafka consumer node in IIB10
| Quote: |
ExceptionList
RecoverableException
File:CHARACTER:F:\build\S1000_slot1\S1000_P\src\DataFlowEngine\MessageServices\ImbDataFlowNode.cpp
Line:INTEGER:1251
Function:CHARACTER:ImbDataFlowNode::createExceptionList
Type:CHARACTER:ComIbmMQInputNode
Name:CHARACTER:KafkaProFlow#FCMComposite_1_1
Label:CHARACTER:KafkaProFlow.MQ Input
Catalog:CHARACTER:BIPmsgs
Severity:INTEGER:3
Number:INTEGER:2230
Text:CHARACTER:Node throwing exception
Insert
Type:INTEGER:14
Text:CHARACTER:KafkaProFlow.MQ Input
RecoverableException
File:CHARACTER:F:\build\S1000_slot1\S1000_P\src\DataFlowEngine\TemplateNodes\ImbOutputTemplateNode.cpp
Line:INTEGER:303
Function:CHARACTER:ImbOutputTemplateNode::processMessageAssemblyToFailure
Type:CHARACTER:ComIbmOutputNode
Name:CHARACTER:KafkaProFlow#FCMComposite_1_3
Label:CHARACTER:KafkaProFlow.KafkaProducer
Catalog:CHARACTER:BIPmsgs
Severity:INTEGER:3
Number:INTEGER:2230
Text:CHARACTER:Caught exception and rethrowing
Insert
Type:INTEGER:14
Text:CHARACTER:KafkaProFlow.KafkaProducer
RecoverableException
File:CHARACTER:ContainerServices.java
Line:INTEGER:138
Function:CHARACTER:throwException
Type:CHARACTER:
Name:CHARACTER:
Label:CHARACTER:
Catalog:CHARACTER:BIPmsgs
Severity:INTEGER:3
Number:INTEGER:3893
Text:CHARACTER:
Insert
Type:INTEGER:5
Text:CHARACTER:Firstprogram
Insert
Type:INTEGER:5
Text:CHARACTER:localhost:9092
Insert
Type:INTEGER:5
Text:CHARACTER:org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.
|
please help me with any documentation ? |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Jul 11, 2017 7:04 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| KafkaSolution wrote: |
| please help me with any documentation ? |
| Code: |
Insert
Type:INTEGER:5
Text:CHARACTER:org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.
|
Your connection timed out. There are probably previous BIP messages in the system log that mention other failures, or give additional info.
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| KafkaSolution |
| Posted: Tue Jul 11, 2017 7:06 am Post subject: |
|
|
Novice
Joined: 21 Jun 2017
Posts: 16
|
| could you please explain me in detail? |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Jul 11, 2017 7:15 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| KafkaSolution wrote: |
| could you please explain me in detail? |
The IIB kafka client didn't get any information from the kafka server, at least within 60000 ms
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| KafkaSolution |
| Posted: Tue Jul 11, 2017 7:23 am Post subject: |
|
|
Novice
Joined: 21 Jun 2017
Posts: 16
|
Thank you mqjeff
how can we resolve the issue? |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Jul 11, 2017 7:31 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| KafkaSolution wrote: |
Thank you mqjeff
how can we resolve the issue? |
By troubleshooting.
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Jul 11, 2017 7:50 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mqjeff wrote: |
| KafkaSolution wrote: |
Thank you mqjeff
how can we resolve the issue? |
By troubleshooting. |
There's no one on this forum who can tell you why that update didn't happen. There are a myriad of possible reasons, and you have to work out which one is the actual reason because you're the only one with access to diagnostic tools at your site.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| KafkaSolution |
| Posted: Tue Jul 11, 2017 7:54 am Post subject: |
|
|
Novice
Joined: 21 Jun 2017
Posts: 16
|
| yah I will look after it. Thank you guys for your help. |
|
| Back to top |
|
|
|
|
