ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexWebSphere Message Broker Supportkafka node credentials issues

Post new topicReply to topic Goto page Previous  1, 2
kafka node credentials issues View previous topic :: View next topic
Author Message
mqjeff
PostPosted: Mon Jun 26, 2017 7:03 am Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17384

KafkaSolution wrote:
Quote:
You need three zookeepers.


may i know what is the need of 3 zookeepers to connect with kafka node...?


If you have not configured your one zookeeper to run in single server mode, then you need an odd number of zookeepers, and enough zookeepers so that a majority/quorum can be achieved. The minimum value for this is three.

Otherwise it sounds like your zookeeper is having trouble processing requests - likely due to resource constraints (memory, disk, cpu, etc.)
_________________
Read, Think, Try, Repeat
Back to top
View user's profile Send private message
KafkaSolution
PostPosted: Tue Jun 27, 2017 3:27 am Post subject: Reply with quote

Novice

Joined: 21 Jun 2017
Posts: 16

we tried it with 3 zookeeper servers but we are unable to access the data from the Kafka node. We are getting the same exception.
Back to top
View user's profile Send private message
mqjeff
PostPosted: Tue Jun 27, 2017 4:22 am Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17384

Ok, so it sounds like your zookeeper(s)) don't know where your kafka server is - that's the "broker".

Check the kafka logs, check the kafka configuration to make sure it knows where the zookeeper url is, try a kafka-topics --list command against one of the zookeepers from kafka machine.

It could possibly be the other way, that your kafka can't find your zookeeper(s). So check the zookeeper logs in more detail.
_________________
Read, Think, Try, Repeat
Back to top
View user's profile Send private message
KafkaSolution
PostPosted: Fri Jul 07, 2017 2:08 am Post subject: Reply with quote

Novice

Joined: 21 Jun 2017
Posts: 16

Can anyone Provide me the Documentation of Kafka node with Zookeeper and Kafka server in IIB ?
Back to top
View user's profile Send private message
zpat
PostPosted: Fri Jul 07, 2017 2:29 am Post subject: Reply with quote

Jedi Council

Joined: 19 May 2001
Posts: 5471
Location: UK

I am reliably informed that IBM document their products and that Google can find it ...

https://developer.ibm.com/integration/blog/2016/11/25/using-the-new-kafka-nodes-in-ibm-integration-bus-10-0-0-7/

https://www.ibm.com/support/knowledgecenter/en/SSMKHH_10.0.0/com.ibm.etools.mft.doc/bz91040_.htm
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error.
Back to top
View user's profile Send private message
KafkaSolution
PostPosted: Fri Jul 07, 2017 2:46 am Post subject: Reply with quote

Novice

Joined: 21 Jun 2017
Posts: 16

Thank You zpat for the docs.

I have seen those documents but they have not provided how to work with IIB node using Zookeeper and Kafka server in our local system
Back to top
View user's profile Send private message
mqjeff
PostPosted: Fri Jul 07, 2017 3:53 am Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17384

KafkaSolution wrote:
Thank You zpat for the docs.

I have seen those documents but they have not provided how to work with IIB node using Zookeeper and Kafka server in our local system


Generally when using Kafka, you don't directly connect to Zookeeper.

If you are having difficulty getting credentials configured, then as my esteemed colleague says, IBM documents their products
_________________
Read, Think, Try, Repeat
Back to top
View user's profile Send private message
KafkaSolution
PostPosted: Tue Jul 11, 2017 6:55 am Post subject: Reply with quote

Novice

Joined: 21 Jun 2017
Posts: 16

i am getting this exception while connecting with kafka server and zookeeper server when connecting to my kafka producer and kafka consumer node in IIB10

Quote:


ExceptionList
RecoverableException
File:CHARACTER:F:\build\S1000_slot1\S1000_P\src\DataFlowEngine\MessageServices\ImbDataFlowNode.cpp
Line:INTEGER:1251
Function:CHARACTER:ImbDataFlowNode::createExceptionList
Type:CHARACTER:ComIbmMQInputNode
Name:CHARACTER:KafkaProFlow#FCMComposite_1_1
Label:CHARACTER:KafkaProFlow.MQ Input
Catalog:CHARACTER:BIPmsgs
Severity:INTEGER:3
Number:INTEGER:2230
Text:CHARACTER:Node throwing exception
Insert
Type:INTEGER:14
Text:CHARACTER:KafkaProFlow.MQ Input
RecoverableException
File:CHARACTER:F:\build\S1000_slot1\S1000_P\src\DataFlowEngine\TemplateNodes\ImbOutputTemplateNode.cpp
Line:INTEGER:303
Function:CHARACTER:ImbOutputTemplateNode::processMessageAssemblyToFailure
Type:CHARACTER:ComIbmOutputNode
Name:CHARACTER:KafkaProFlow#FCMComposite_1_3
Label:CHARACTER:KafkaProFlow.KafkaProducer
Catalog:CHARACTER:BIPmsgs
Severity:INTEGER:3
Number:INTEGER:2230
Text:CHARACTER:Caught exception and rethrowing
Insert
Type:INTEGER:14
Text:CHARACTER:KafkaProFlow.KafkaProducer
RecoverableException
File:CHARACTER:ContainerServices.java
Line:INTEGER:138
Function:CHARACTER:throwException
Type:CHARACTER:
Name:CHARACTER:
Label:CHARACTER:
Catalog:CHARACTER:BIPmsgs
Severity:INTEGER:3
Number:INTEGER:3893
Text:CHARACTER:
Insert
Type:INTEGER:5
Text:CHARACTER:Firstprogram
Insert
Type:INTEGER:5
Text:CHARACTER:localhost:9092
Insert
Type:INTEGER:5
Text:CHARACTER:org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.


please help me with any documentation ?
Back to top
View user's profile Send private message
mqjeff
PostPosted: Tue Jul 11, 2017 7:04 am Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17384

KafkaSolution wrote:
please help me with any documentation ?

Code:
Insert
Type:INTEGER:5
Text:CHARACTER:org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.


Your connection timed out. There are probably previous BIP messages in the system log that mention other failures, or give additional info.
_________________
Read, Think, Try, Repeat
Back to top
View user's profile Send private message
KafkaSolution
PostPosted: Tue Jul 11, 2017 7:06 am Post subject: Reply with quote

Novice

Joined: 21 Jun 2017
Posts: 16

could you please explain me in detail?
Back to top
View user's profile Send private message
mqjeff
PostPosted: Tue Jul 11, 2017 7:15 am Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17384

KafkaSolution wrote:
could you please explain me in detail?


The IIB kafka client didn't get any information from the kafka server, at least within 60000 ms
_________________
Read, Think, Try, Repeat
Back to top
View user's profile Send private message
KafkaSolution
PostPosted: Tue Jul 11, 2017 7:23 am Post subject: Reply with quote

Novice

Joined: 21 Jun 2017
Posts: 16

Thank you mqjeff

how can we resolve the issue?
Back to top
View user's profile Send private message
mqjeff
PostPosted: Tue Jul 11, 2017 7:31 am Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17384

KafkaSolution wrote:
Thank you mqjeff

how can we resolve the issue?


By troubleshooting.
_________________
Read, Think, Try, Repeat
Back to top
View user's profile Send private message
Vitor
PostPosted: Tue Jul 11, 2017 7:50 am Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 24450
Location: Ohio, USA

mqjeff wrote:
KafkaSolution wrote:
Thank you mqjeff

how can we resolve the issue?


By troubleshooting.





There's no one on this forum who can tell you why that update didn't happen. There are a myriad of possible reasons, and you have to work out which one is the actual reason because you're the only one with access to diagnostic tools at your site.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
KafkaSolution
PostPosted: Tue Jul 11, 2017 7:54 am Post subject: Reply with quote

Novice

Joined: 21 Jun 2017
Posts: 16

yah I will look after it. Thank you guys for your help.
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Goto page Previous  1, 2 Page 2 of 2

MQSeries.net Forum IndexWebSphere Message Broker Supportkafka node credentials issues
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.