ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » IBM MQ Telemetry / Low Latency Messaging / Everyplace » MQ Telemetry Clients not connecting or disconnecting

Post new topic  Reply to topic
 MQ Telemetry Clients not connecting or disconnecting « View previous topic :: View next topic » 
Author Message
Slasher
PostPosted: Wed Jan 22, 2014 9:07 pm    Post subject: MQ Telemetry Clients not connecting or disconnecting Reply with quote

Newbie

Joined: 07 May 2012
Posts: 5

This is a peculiar issue I have come across , which I thought I should document for anyone who might need it. The solution was arrived at by some extrapolations , so if someone could give a reasoning behind it , that would be awesome.
windows server environment , MQ 7.1.0.2 , Single QMGR/ Broker 7 environment. MQ Client on remote server on one end , multiple MQTT Clients at the other end.
1. MQ Telemetry clients using IBM's standard MQTT Client ( which is no longer suipported by IBM ) was connecting as usual.
2. After a prolonged 2G network outage ( more than 4 hrs ) 10 to 12 out of 300 clients would reconnect , and after a few mins only two would remain connected.
3. Queue Manager was restarted , MQTT channel was purged and it would still exibit the same behaviour ( step 2 ) . The two clients would be some random client , no affinity here.
4. the task manager showed a java32.exe process that was cosuming nearly all of CPU resources. This was unusual and I could not determine what this process was.
5. QMGR errors or MQ Errors did not show any particular error. MQXR logs showed nothing other than "the client disconnected abnormally" a usual behaviour - the clients go in and out of network coverage and disconnect abnormally
6. The system was rebooted but no change was observed
7. MQTT client utility on MQ Explorer did not connect as well.

8. I created another telemetry channel on another port. The MQTT Client utility could connect to this one. Infact multipleinstances of MQTT Client utility could.
9. A netstat -a showed that all clients are going into state fin_wait_2 . All i could make out of it was that it is waiting to connect and it cannot establish connection on 1883.
10. So I decided to make resources used by channel running on 1883 , as it would be for a fresh new channel.
11. I emptied the system.mqtt.transmit.queue and the MQTT clients started connecting.
12. netstat -a showed the connection as established.

I honestly have no idea why it did. The java process that consumed all resources earlier , started to exhibit normal resource consumption.

After effects :
The MQTT Clients as soon as they re connnected , published a large no of messages that they had stored during the outage. These messages took some time to clear , they are used by broker to process , enrich and place them in a output queue. Also the MQ client apps who would connect to the QM to pick up the processed messages , had to be restarted as they had stopped retrying connection attempts to QMGR.

This was a unusual issue. I hope it helps someone with similar issue. If one can figure out a reasoning , please post it for discussion. especially , why did emptying of SYSTEM.MQTT.TRANSMIT.QUEUE solve the problem.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » IBM MQ Telemetry / Low Latency Messaging / Everyplace » MQ Telemetry Clients not connecting or disconnecting
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.