ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » IBM MQ Installation/Configuration Support » Message is missing

Post new topic  Reply to topic Goto page 1, 2  Next
 Message is missing « View previous topic :: View next topic » 
Author Message
kalam475
PostPosted: Fri Jan 20, 2017 4:37 am    Post subject: Message is missing Reply with quote

Acolyte

Joined: 16 Jan 2015
Posts: 63

I have set up a Queue manager Cluster with cluster queue on two queu managers and connecting to the queue manager through JMS application through connection name list or CCDT file from Websphere application server connection factory.

When testing I sent a 20000 messages through JMeter. I purpose fully stoped the one of the queue manager the application goes on connects to another QMGR in cluster and puts the messages in available cluster queue.

But when i check number of message in both the clustered queues I found some of the messages is missing. and when I see the logs of WAS i was getting the error "WebSphere MQ call failed with compcode '2' ('MQCC_FAILED') reason '2009' ('MQRC_CONNECTION_BROKEN')." which is perfectly fine.

I made sure i was setting the message as persistant message and queue default is also persistant. still i am loosing the messages.

I have gone through some of the older posts where they discussed about the assured delivery for client applications and it being one of the thing we have no control on. Is there some solution for this.
Right now i have not implemented the reconnection logic if I do so do i get any reprieve or is it the same.
Thanks
Back to top
View user's profile Send private message
mqjeff
PostPosted: Fri Jan 20, 2017 4:41 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

MQ cluisters do not duplicate messages across the queues in a cluster.

MQ clusters distribute messages across the queues in a cluster.

So if you have queue A on two qmgrs, and both queue A's are shared in the cluster, when you put two messages to queue A, then one will go to each queue, not two to both.
_________________
chmod -R ugo-wx /
Back to top
View user's profile Send private message
bruce2359
PostPosted: Fri Jan 20, 2017 4:47 am    Post subject: Re: Message is missing Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

kalam475 wrote:
When testing I sent a 20000 messages ...

But when i check number of message in both the clustered queues I found some of the messages is missing ...

Please be precise. Exactly how many of the 20000 messages were not delivered? Any messages in the dead letter queues?

Did you run this test more than one time? If so, did it fail every time? Same error? Exactly how many of the 20000 messages failed to be delivered on each attempt?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
kalam475
PostPosted: Fri Jan 20, 2017 5:26 am    Post subject: Reply with quote

Acolyte

Joined: 16 Jan 2015
Posts: 63

Hi have tested more than 20 times not each time i am missing the messages. sometime i am getting all the 20000 messages.

But i am missing about 3 to 10 messages on testing some times that to when i am getting the 2009 error in WAS logs. what i am guessing is before it is getting comited the connection is lost then only messge is lost. but it is have a clean break i am getting all the messages in the queue.

Yes i am adding both the messages in two cluster queues after that i am missing 5 messages on an average and sometimes not even one. Please help.Thanks for your instant reply
Back to top
View user's profile Send private message
bruce2359
PostPosted: Fri Jan 20, 2017 5:46 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

Display both queues. What is the UNCOM value of both?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
Vitor
PostPosted: Fri Jan 20, 2017 5:49 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

kalam475 wrote:
Yes i am adding both the messages in two cluster queues


No you're not. As my most worthy associate points out, a message put to a queue shared in a cluster goes to one instance of that queue and one only.

bruce2359 wrote:
Display both queues. What is the UNCOM value of both?




Demonstrate that these messages are truly lost and not sitting on the queue uncommitted.

Also demonstrate that a bug in the application is not causing these messages to be discarded when the 2009 is received.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
kalam475
PostPosted: Fri Jan 20, 2017 6:14 am    Post subject: Reply with quote

Acolyte

Joined: 16 Jan 2015
Posts: 63

[mqadmin@mq1 bin]$ ./runmqsc CLUSQM1
5724-H72 (C) Copyright IBM Corp. 1994, 2015.
Starting MQSC for queue manager CLUSQM1.


display queue('mdbq')
1 : display queue('mdbq')
AMQ8409: Display Queue details.
QUEUE(mdbq) TYPE(QLOCAL)
ACCTQ(QMGR) ALTDATE(2017-01-19)
ALTTIME(10.42.16) BOQNAME(mdbq.backout)
BOTHRESH(2) CLUSNL( )
CLUSTER(CLUS123) CLCHNAME( )
CLWLPRTY(0) CLWLRANK(0)
CLWLUSEQ(ANY) CRDATE(2017-01-15)
CRTIME(22.33.55) CURDEPTH(12570)
CUSTOM( ) DEFBIND(NOTFIXED)
DEFPRTY(0) DEFPSIST(YES)
DEFPRESP(SYNC) DEFREADA(NO)
DEFSOPT(SHARED) DEFTYPE(PREDEFINED)
DESCR( ) DISTL(NO)
GET(ENABLED) HARDENBO
INITQ( ) IPPROCS(0)
MAXDEPTH(50000) MAXMSGL(4194304)
MONQ(QMGR) MSGDLVSQ(PRIORITY)
NOTRIGGER NPMCLASS(NORMAL)
OPPROCS(0) PROCESS( )
PUT(ENABLED) PROPCTL(COMPAT)
QDEPTHHI(80) QDEPTHLO(20)
QDPHIEV(DISABLED) QDPLOEV(DISABLED)
QDPMAXEV(ENABLED) QSVCIEV(NONE)
QSVCINT(999999999) RETINTVL(999999999)
SCOPE(QMGR) SHARE
STATQ(QMGR) TRIGDATA( )
TRIGDPTH(1) TRIGMPRI(0)
TRIGTYPE(FIRST) USAGE(NORMAL)

[mqadmin@mq1 bin]$ ./runmqsc CLUSQM2
5724-H72 (C) Copyright IBM Corp. 1994, 2015.
Starting MQSC for queue manager CLUSQM2.


display queue('mdbq')
1 : display queue('mdbq')
AMQ8409: Display Queue details.
QUEUE(mdbq) TYPE(QLOCAL)
ACCTQ(QMGR) ALTDATE(2017-01-19)
ALTTIME(08.11.02) BOQNAME( )
BOTHRESH(0) CLUSNL( )
CLUSTER(CLUS123) CLCHNAME( )
CLWLPRTY(0) CLWLRANK(0)
CLWLUSEQ(ANY) CRDATE(2017-01-1
CRTIME(05.32.44) CURDEPTH(7427)
CUSTOM( ) DEFBIND(NOTFIXED)
DEFPRTY(0) DEFPSIST(YES)
DEFPRESP(SYNC) DEFREADA(NO)
DEFSOPT(SHARED) DEFTYPE(PREDEFINED)
DESCR( ) DISTL(NO)
GET(ENABLED) HARDENBO
INITQ( ) IPPROCS(0)
MAXDEPTH(50000) MAXMSGL(4194304)
MONQ(QMGR) MSGDLVSQ(PRIORITY)
NOTRIGGER NPMCLASS(NORMAL)
OPPROCS(0) PROCESS( )
PUT(ENABLED) PROPCTL(COMPAT)
QDEPTHHI(80) QDEPTHLO(20)
QDPHIEV(DISABLED) QDPLOEV(DISABLED)
QDPMAXEV(ENABLED) QSVCIEV(NONE)
QSVCINT(999999999) RETINTVL(999999999)
SCOPE(QMGR) SHARE
STATQ(QMGR) TRIGDATA( )
TRIGDPTH(1) TRIGMPRI(0)
TRIGTYPE(FIRST) USAGE(NORMAL)


these are the display of both the cluster queues. As you can see there are 3 messages missing.

below are the display qstatus output at present no client application is connected.

for CLUSQM2 qstatus is ----

DISPLAY QSTATUS('mdbq') TYPE(HANDLE) ALL
1 : DISPLAY QSTATUS('mdbq') TYPE(HANDLE) ALL
AMQ8565: Queue status not found.
display qstatus('mdbq')
2 : display qstatus('mdbq')
AMQ8450: Display queue status details.
QUEUE(mdbq) TYPE(QUEUE)
CURDEPTH(7427) IPPROCS(0)
LGETDATE( ) LGETTIME( )
LPUTDATE( ) LPUTTIME( )
MEDIALOG(S0000046.LOG) MONQ(OFF)
MSGAGE( ) OPPROCS(0)
QTIME( , ) UNCOM(NO)


display qstatus for CLUSQM1 with clusterq is


DISPLAY QSTATUS('mdbq') TYPE(HANDLE) ALL
1 : DISPLAY QSTATUS('mdbq') TYPE(HANDLE) ALL
AMQ8565: Queue status not found.
display qstatus('mdbq')
2 : display qstatus('mdbq')
AMQ8450: Display queue status details.
QUEUE(mdbq) TYPE(QUEUE)
CURDEPTH(12570) IPPROCS(0)
LGETDATE( ) LGETTIME( )
LPUTDATE( ) LPUTTIME( )
MEDIALOG( ) MONQ(OFF)
MSGAGE( ) OPPROCS(0)
QTIME( , ) UNCOM(NO)


Hope this helps. Again thanks for your time.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Fri Jan 20, 2017 6:22 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

Display the system cluster transmission queues on both qmgrs.

Display the dead letter queues on both qmgrs.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Fri Jan 20, 2017 6:27 am    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

Don't have the op display queue setup. In this case it's an exercise in futility.
His first post says it all
Quote:
But when i check number of message in both the clustered queues I found some of the messages is missing. and when I see the logs of WAS i was getting the error "WebSphere MQ call failed with compcode '2' ('MQCC_FAILED') reason '2009' ('MQRC_CONNECTION_BROKEN')." which is perfectly fine

So which part of the reason code MQRC_CONNECTION_BROKEN did the OP not understand??
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
kalam475
PostPosted: Fri Jan 20, 2017 6:30 am    Post subject: Reply with quote

Acolyte

Joined: 16 Jan 2015
Posts: 63

Here are the details

Dead letter queue for CLUSQM1

[mqadmin@mq1 bin]$ ./runmqsc CLUSQM1
5724-H72 (C) Copyright IBM Corp. 1994, 2015.
Starting MQSC for queue manager CLUSQM1.


display queue('DeadLetterQ')
1 : display queue('DeadLetterQ')
AMQ8409: Display Queue details.
QUEUE(DeadLetterQ) TYPE(QLOCAL)
ACCTQ(QMGR) ALTDATE(2017-01-19)
ALTTIME(11.27.34) BOQNAME( )
BOTHRESH(0) CLUSNL( )
CLUSTER( ) CLCHNAME( )
CLWLPRTY(0) CLWLRANK(0)
CLWLUSEQ(QMGR) CRDATE(2017-01-19)
CRTIME(11.27.34) CURDEPTH(0)
CUSTOM( ) DEFBIND(OPEN)
DEFPRTY(0) DEFPSIST(YES)
DEFPRESP(SYNC) DEFREADA(NO)
DEFSOPT(SHARED) DEFTYPE(PREDEFINED)
DESCR(WebSphere MQ Default Dead Letter Queue)
DISTL(NO) GET(ENABLED)
HARDENBO INITQ( )
IPPROCS(0) MAXDEPTH(999999999)
MAXMSGL(4194304) MONQ(QMGR)
MSGDLVSQ(PRIORITY) NOTRIGGER
NPMCLASS(NORMAL) OPPROCS(0)
PROCESS( ) PUT(ENABLED)
PROPCTL(COMPAT) QDEPTHHI(80)
QDEPTHLO(20) QDPHIEV(DISABLED)
QDPLOEV(DISABLED) QDPMAXEV(ENABLED)
QSVCIEV(NONE) QSVCINT(999999999)
RETINTVL(999999999) SCOPE(QMGR)
SHARE STATQ(QMGR)
TRIGDATA( ) TRIGDPTH(1)
TRIGMPRI(0) TRIGTYPE(NONE)
USAGE(NORMAL)


dead letter queue for CLUSQM2

[mqadmin@mq1 bin]$ ./runmqsc CLUSQM2
5724-H72 (C) Copyright IBM Corp. 1994, 2015.
Starting MQSC for queue manager CLUSQM2.


display queue(DEADLETTERQ)
1 : display queue(DEADLETTERQ)
AMQ8409: Display Queue details.
QUEUE(DEADLETTERQ) TYPE(QLOCAL)
ACCTQ(QMGR) ALTDATE(2017-01-19)
ALTTIME(11.28.24) BOQNAME( )
BOTHRESH(0) CLUSNL( )
CLUSTER( ) CLCHNAME( )
CLWLPRTY(0) CLWLRANK(0)
CLWLUSEQ(QMGR) CRDATE(2017-01-19)
CRTIME(11.28.24) CURDEPTH(0)
CUSTOM( ) DEFBIND(OPEN)
DEFPRTY(0) DEFPSIST(YES)
DEFPRESP(SYNC) DEFREADA(NO)
DEFSOPT(SHARED) DEFTYPE(PREDEFINED)
DESCR(WebSphere MQ Default Dead Letter Queue)
DISTL(NO) GET(ENABLED)
HARDENBO INITQ( )
IPPROCS(0) MAXDEPTH(999999999)
MAXMSGL(4194304) MONQ(QMGR)
MSGDLVSQ(PRIORITY) NOTRIGGER
NPMCLASS(NORMAL) OPPROCS(0)
PROCESS( ) PUT(ENABLED)
PROPCTL(COMPAT) QDEPTHHI(80)
QDEPTHLO(20) QDPHIEV(DISABLED)
QDPLOEV(DISABLED) QDPMAXEV(ENABLED)
QSVCIEV(NONE) QSVCINT(999999999)
RETINTVL(999999999) SCOPE(QMGR)
SHARE STATQ(QMGR)
TRIGDATA( ) TRIGDPTH(1)
TRIGMPRI(0) TRIGTYPE(NONE)
USAGE(NORMAL)


system cluster transmission queue for CLUSQM2

display queue(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
2 : display queue(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
AMQ8409: Display Queue details.
QUEUE(SYSTEM.CLUSTER.TRANSMIT.QUEUE) TYPE(QLOCAL)
ACCTQ(QMGR) ALTDATE(2016-12-30)
ALTTIME(01.45.2 BOQNAME( )
BOTHRESH(0) CLUSNL( )
CLUSTER( ) CLCHNAME( )
CLWLPRTY(0) CLWLRANK(0)
CLWLUSEQ(QMGR) CRDATE(2016-12-30)
CRTIME(01.39.15) CURDEPTH(0)
CUSTOM( ) DEFBIND(OPEN)
DEFPRTY(0) DEFPSIST(NO)
DEFPRESP(SYNC) DEFREADA(NO)
DEFSOPT(SHARED) DEFTYPE(PREDEFINED)
DESCR(WebSphere MQ Cluster Transmission Queue)
DISTL(NO) GET(ENABLED)
HARDENBO INITQ( )
IPPROCS(0) MAXDEPTH(999999999)
MAXMSGL(4194304) MONQ(QMGR)
MSGDLVSQ(PRIORITY) TRIGGER
NPMCLASS(NORMAL) OPPROCS(0)
PROCESS( ) PUT(ENABLED)
PROPCTL(COMPAT) QDEPTHHI(80)
QDEPTHLO(20) QDPHIEV(DISABLED)
QDPLOEV(DISABLED) QDPMAXEV(ENABLED)
QSVCIEV(NONE) QSVCINT(999999999)
RETINTVL(999999999) SCOPE(QMGR)
SHARE STATQ(QMGR)
TRIGDATA( ) TRIGDPTH(1)
TRIGMPRI(0) TRIGTYPE(FIRST)
USAGE(XMITQ)


system cluster transmission queue for CLUSQM1


[mqadmin@mq1 bin]$ ./runmqsc CLUSQM1
5724-H72 (C) Copyright IBM Corp. 1994, 2015.
Starting MQSC for queue manager CLUSQM1.


display queue(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
1 : display queue(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
AMQ8409: Display Queue details.
QUEUE(SYSTEM.CLUSTER.TRANSMIT.QUEUE) TYPE(QLOCAL)
ACCTQ(QMGR) ALTDATE(2016-12-30)
ALTTIME(01.45.2 BOQNAME( )
BOTHRESH(0) CLUSNL( )
CLUSTER( ) CLCHNAME( )
CLWLPRTY(0) CLWLRANK(0)
CLWLUSEQ(QMGR) CRDATE(2016-12-30)
CRTIME(01.33.01) CURDEPTH(0)
CUSTOM( ) DEFBIND(OPEN)
DEFPRTY(0) DEFPSIST(NO)
DEFPRESP(SYNC) DEFREADA(NO)
DEFSOPT(SHARED) DEFTYPE(PREDEFINED)
DESCR(WebSphere MQ Cluster Transmission Queue)
DISTL(NO) GET(ENABLED)
HARDENBO INITQ( )
IPPROCS(0) MAXDEPTH(999999999)
MAXMSGL(4194304) MONQ(QMGR)
MSGDLVSQ(PRIORITY) TRIGGER
NPMCLASS(NORMAL) OPPROCS(0)
PROCESS( ) PUT(ENABLED)
PROPCTL(COMPAT) QDEPTHHI(80)
QDEPTHLO(20) QDPHIEV(DISABLED)
QDPLOEV(DISABLED) QDPMAXEV(ENABLED)
QSVCIEV(NONE) QSVCINT(999999999)
RETINTVL(999999999) SCOPE(QMGR)
SHARE STATQ(QMGR)
TRIGDATA( ) TRIGDPTH(1)
TRIGMPRI(0) TRIGTYPE(FIRST)
USAGE(XMITQ)


above are the details you asked for. Thanks
Back to top
View user's profile Send private message
mqjeff
PostPosted: Fri Jan 20, 2017 6:54 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

If you are missing some messages AND you are not expecting all messages to be on both then...

The interruption of the connection is very likely to cause the missing messages to be rolled back by WAS - and likely dropped entirely.
_________________
chmod -R ugo-wx /
Back to top
View user's profile Send private message
kalam475
PostPosted: Fri Jan 20, 2017 7:34 am    Post subject: Reply with quote

Acolyte

Joined: 16 Jan 2015
Posts: 63

I am not expecting messages to be present in both the queues the total number of messages in the queues should be equal to 20000

Is there any kind of work around for this kind of problem thanks everyone for quick replies.
Back to top
View user's profile Send private message
Vitor
PostPosted: Fri Jan 20, 2017 8:02 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

kalam475 wrote:
Is there any kind of work around for this kind of problem


Correctly handle the 2009 as I indicated above.

You've proved that the messages are not uncommitted on the queue. So your application is trying to put the message, the put / JMS send is failing with a 2009 and the application is continuing as if the message had been successfully put just because the message is persistent. Even a persistent message will be lost if it's not successfully put into the queue manager's care.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Fri Jan 20, 2017 8:48 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

So, in summary, your app attempted to put 20000 messages, but failed to successfully put (and commit) some of them - in this instance due to a connection failure.

In such a failure, it is the responsibility of the app to reconnect and retry the put, or take other appropriate action.

A well-written app would count and echo (display) the total attempts to put, and the total number of fails.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
kalam475
PostPosted: Sun Jan 22, 2017 9:14 pm    Post subject: Reply with quote

Acolyte

Joined: 16 Jan 2015
Posts: 63

Can anyone provide me a link or suggest a example to incorparate mentioned features in my application i.e how to handle 2009 RC and client reconnect. Thanks
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum Index » IBM MQ Installation/Configuration Support » Message is missing
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.