| Author |
Message
|
| kalam475 |
 Posted: Fri Jan 20, 2017 4:37 am Post subject: Message is missing |
 |
|
Acolyte
Joined: 16 Jan 2015
Posts: 63
|
I have set up a Queue manager Cluster with cluster queue on two queu managers and connecting to the queue manager through JMS application through connection name list or CCDT file from Websphere application server connection factory.
When testing I sent a 20000 messages through JMeter. I purpose fully stoped the one of the queue manager the application goes on connects to another QMGR in cluster and puts the messages in available cluster queue.
But when i check number of message in both the clustered queues I found some of the messages is missing. and when I see the logs of WAS i was getting the error "WebSphere MQ call failed with compcode '2' ('MQCC_FAILED') reason '2009' ('MQRC_CONNECTION_BROKEN')." which is perfectly fine.
I made sure i was setting the message as persistant message and queue default is also persistant. still i am loosing the messages.
I have gone through some of the older posts where they discussed about the assured delivery for client applications and it being one of the thing we have no control on. Is there some solution for this.
Right now i have not implemented the reconnection logic if I do so do i get any reprieve or is it the same.
Thanks |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Fri Jan 20, 2017 4:41 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
MQ cluisters do not duplicate messages across the queues in a cluster.
MQ clusters distribute messages across the queues in a cluster.
So if you have queue A on two qmgrs, and both queue A's are shared in the cluster, when you put two messages to queue A, then one will go to each queue, not two to both.
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Jan 20, 2017 4:47 am Post subject: Re: Message is missing |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| kalam475 wrote: |
When testing I sent a 20000 messages ...
But when i check number of message in both the clustered queues I found some of the messages is missing ... |
Please be precise. Exactly how many of the 20000 messages were not delivered? Any messages in the dead letter queues?
Did you run this test more than one time? If so, did it fail every time? Same error? Exactly how many of the 20000 messages failed to be delivered on each attempt?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| kalam475 |
| Posted: Fri Jan 20, 2017 5:26 am Post subject: |
|
|
Acolyte
Joined: 16 Jan 2015
Posts: 63
|
Hi have tested more than 20 times not each time i am missing the messages. sometime i am getting all the 20000 messages.
But i am missing about 3 to 10 messages on testing some times that to when i am getting the 2009 error in WAS logs. what i am guessing is before it is getting comited the connection is lost then only messge is lost. but it is have a clean break i am getting all the messages in the queue.
Yes i am adding both the messages in two cluster queues after that i am missing 5 messages on an average and sometimes not even one. Please help.Thanks for your instant reply |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Jan 20, 2017 5:46 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Display both queues. What is the UNCOM value of both?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jan 20, 2017 5:49 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| kalam475 wrote: |
| Yes i am adding both the messages in two cluster queues |
No you're not. As my most worthy associate points out, a message put to a queue shared in a cluster goes to one instance of that queue and one only.
| bruce2359 wrote: |
| Display both queues. What is the UNCOM value of both? |
Demonstrate that these messages are truly lost and not sitting on the queue uncommitted.
Also demonstrate that a bug in the application is not causing these messages to be discarded when the 2009 is received.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| kalam475 |
| Posted: Fri Jan 20, 2017 6:14 am Post subject: |
|
|
Acolyte
Joined: 16 Jan 2015
Posts: 63
|
[mqadmin@mq1 bin]$ ./runmqsc CLUSQM1
5724-H72 (C) Copyright IBM Corp. 1994, 2015.
Starting MQSC for queue manager CLUSQM1.
display queue('mdbq')
1 : display queue('mdbq')
AMQ8409: Display Queue details.
QUEUE(mdbq) TYPE(QLOCAL)
ACCTQ(QMGR) ALTDATE(2017-01-19)
ALTTIME(10.42.16) BOQNAME(mdbq.backout)
BOTHRESH(2) CLUSNL( )
CLUSTER(CLUS123) CLCHNAME( )
CLWLPRTY(0) CLWLRANK(0)
CLWLUSEQ(ANY) CRDATE(2017-01-15)
CRTIME(22.33.55) CURDEPTH(12570)
CUSTOM( ) DEFBIND(NOTFIXED)
DEFPRTY(0) DEFPSIST(YES)
DEFPRESP(SYNC) DEFREADA(NO)
DEFSOPT(SHARED) DEFTYPE(PREDEFINED)
DESCR( ) DISTL(NO)
GET(ENABLED) HARDENBO
INITQ( ) IPPROCS(0)
MAXDEPTH(50000) MAXMSGL(4194304)
MONQ(QMGR) MSGDLVSQ(PRIORITY)
NOTRIGGER NPMCLASS(NORMAL)
OPPROCS(0) PROCESS( )
PUT(ENABLED) PROPCTL(COMPAT)
QDEPTHHI(80) QDEPTHLO(20)
QDPHIEV(DISABLED) QDPLOEV(DISABLED)
QDPMAXEV(ENABLED) QSVCIEV(NONE)
QSVCINT(999999999) RETINTVL(999999999)
SCOPE(QMGR) SHARE
STATQ(QMGR) TRIGDATA( )
TRIGDPTH(1) TRIGMPRI(0)
TRIGTYPE(FIRST) USAGE(NORMAL)
[mqadmin@mq1 bin]$ ./runmqsc CLUSQM2
5724-H72 (C) Copyright IBM Corp. 1994, 2015.
Starting MQSC for queue manager CLUSQM2.
display queue('mdbq')
1 : display queue('mdbq')
AMQ8409: Display Queue details.
QUEUE(mdbq) TYPE(QLOCAL)
ACCTQ(QMGR) ALTDATE(2017-01-19)
ALTTIME(08.11.02) BOQNAME( )
BOTHRESH(0) CLUSNL( )
CLUSTER(CLUS123) CLCHNAME( )
CLWLPRTY(0) CLWLRANK(0)
CLWLUSEQ(ANY) CRDATE(2017-01-1
CRTIME(05.32.44) CURDEPTH(7427)
CUSTOM( ) DEFBIND(NOTFIXED)
DEFPRTY(0) DEFPSIST(YES)
DEFPRESP(SYNC) DEFREADA(NO)
DEFSOPT(SHARED) DEFTYPE(PREDEFINED)
DESCR( ) DISTL(NO)
GET(ENABLED) HARDENBO
INITQ( ) IPPROCS(0)
MAXDEPTH(50000) MAXMSGL(4194304)
MONQ(QMGR) MSGDLVSQ(PRIORITY)
NOTRIGGER NPMCLASS(NORMAL)
OPPROCS(0) PROCESS( )
PUT(ENABLED) PROPCTL(COMPAT)
QDEPTHHI(80) QDEPTHLO(20)
QDPHIEV(DISABLED) QDPLOEV(DISABLED)
QDPMAXEV(ENABLED) QSVCIEV(NONE)
QSVCINT(999999999) RETINTVL(999999999)
SCOPE(QMGR) SHARE
STATQ(QMGR) TRIGDATA( )
TRIGDPTH(1) TRIGMPRI(0)
TRIGTYPE(FIRST) USAGE(NORMAL)
these are the display of both the cluster queues. As you can see there are 3 messages missing.
below are the display qstatus output at present no client application is connected.
for CLUSQM2 qstatus is ----
DISPLAY QSTATUS('mdbq') TYPE(HANDLE) ALL
1 : DISPLAY QSTATUS('mdbq') TYPE(HANDLE) ALL
AMQ8565: Queue status not found.
display qstatus('mdbq')
2 : display qstatus('mdbq')
AMQ8450: Display queue status details.
QUEUE(mdbq) TYPE(QUEUE)
CURDEPTH(7427) IPPROCS(0)
LGETDATE( ) LGETTIME( )
LPUTDATE( ) LPUTTIME( )
MEDIALOG(S0000046.LOG) MONQ(OFF)
MSGAGE( ) OPPROCS(0)
QTIME( , ) UNCOM(NO)
display qstatus for CLUSQM1 with clusterq is
DISPLAY QSTATUS('mdbq') TYPE(HANDLE) ALL
1 : DISPLAY QSTATUS('mdbq') TYPE(HANDLE) ALL
AMQ8565: Queue status not found.
display qstatus('mdbq')
2 : display qstatus('mdbq')
AMQ8450: Display queue status details.
QUEUE(mdbq) TYPE(QUEUE)
CURDEPTH(12570) IPPROCS(0)
LGETDATE( ) LGETTIME( )
LPUTDATE( ) LPUTTIME( )
MEDIALOG( ) MONQ(OFF)
MSGAGE( ) OPPROCS(0)
QTIME( , ) UNCOM(NO)
Hope this helps. Again thanks for your time. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Jan 20, 2017 6:22 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Display the system cluster transmission queues on both qmgrs.
Display the dead letter queues on both qmgrs.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Jan 20, 2017 6:27 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Don't have the op display queue setup. In this case it's an exercise in futility.
His first post says it all
| Quote: |
| But when i check number of message in both the clustered queues I found some of the messages is missing. and when I see the logs of WAS i was getting the error "WebSphere MQ call failed with compcode '2' ('MQCC_FAILED') reason '2009' ('MQRC_CONNECTION_BROKEN')." which is perfectly fine |
So which part of the reason code MQRC_CONNECTION_BROKEN did the OP not understand?? 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| kalam475 |
| Posted: Fri Jan 20, 2017 6:30 am Post subject: |
|
|
Acolyte
Joined: 16 Jan 2015
Posts: 63
|
Here are the details
Dead letter queue for CLUSQM1
[mqadmin@mq1 bin]$ ./runmqsc CLUSQM1
5724-H72 (C) Copyright IBM Corp. 1994, 2015.
Starting MQSC for queue manager CLUSQM1.
display queue('DeadLetterQ')
1 : display queue('DeadLetterQ')
AMQ8409: Display Queue details.
QUEUE(DeadLetterQ) TYPE(QLOCAL)
ACCTQ(QMGR) ALTDATE(2017-01-19)
ALTTIME(11.27.34) BOQNAME( )
BOTHRESH(0) CLUSNL( )
CLUSTER( ) CLCHNAME( )
CLWLPRTY(0) CLWLRANK(0)
CLWLUSEQ(QMGR) CRDATE(2017-01-19)
CRTIME(11.27.34) CURDEPTH(0)
CUSTOM( ) DEFBIND(OPEN)
DEFPRTY(0) DEFPSIST(YES)
DEFPRESP(SYNC) DEFREADA(NO)
DEFSOPT(SHARED) DEFTYPE(PREDEFINED)
DESCR(WebSphere MQ Default Dead Letter Queue)
DISTL(NO) GET(ENABLED)
HARDENBO INITQ( )
IPPROCS(0) MAXDEPTH(999999999)
MAXMSGL(4194304) MONQ(QMGR)
MSGDLVSQ(PRIORITY) NOTRIGGER
NPMCLASS(NORMAL) OPPROCS(0)
PROCESS( ) PUT(ENABLED)
PROPCTL(COMPAT) QDEPTHHI(80)
QDEPTHLO(20) QDPHIEV(DISABLED)
QDPLOEV(DISABLED) QDPMAXEV(ENABLED)
QSVCIEV(NONE) QSVCINT(999999999)
RETINTVL(999999999) SCOPE(QMGR)
SHARE STATQ(QMGR)
TRIGDATA( ) TRIGDPTH(1)
TRIGMPRI(0) TRIGTYPE(NONE)
USAGE(NORMAL)
dead letter queue for CLUSQM2
[mqadmin@mq1 bin]$ ./runmqsc CLUSQM2
5724-H72 (C) Copyright IBM Corp. 1994, 2015.
Starting MQSC for queue manager CLUSQM2.
display queue(DEADLETTERQ)
1 : display queue(DEADLETTERQ)
AMQ8409: Display Queue details.
QUEUE(DEADLETTERQ) TYPE(QLOCAL)
ACCTQ(QMGR) ALTDATE(2017-01-19)
ALTTIME(11.28.24) BOQNAME( )
BOTHRESH(0) CLUSNL( )
CLUSTER( ) CLCHNAME( )
CLWLPRTY(0) CLWLRANK(0)
CLWLUSEQ(QMGR) CRDATE(2017-01-19)
CRTIME(11.28.24) CURDEPTH(0)
CUSTOM( ) DEFBIND(OPEN)
DEFPRTY(0) DEFPSIST(YES)
DEFPRESP(SYNC) DEFREADA(NO)
DEFSOPT(SHARED) DEFTYPE(PREDEFINED)
DESCR(WebSphere MQ Default Dead Letter Queue)
DISTL(NO) GET(ENABLED)
HARDENBO INITQ( )
IPPROCS(0) MAXDEPTH(999999999)
MAXMSGL(4194304) MONQ(QMGR)
MSGDLVSQ(PRIORITY) NOTRIGGER
NPMCLASS(NORMAL) OPPROCS(0)
PROCESS( ) PUT(ENABLED)
PROPCTL(COMPAT) QDEPTHHI(80)
QDEPTHLO(20) QDPHIEV(DISABLED)
QDPLOEV(DISABLED) QDPMAXEV(ENABLED)
QSVCIEV(NONE) QSVCINT(999999999)
RETINTVL(999999999) SCOPE(QMGR)
SHARE STATQ(QMGR)
TRIGDATA( ) TRIGDPTH(1)
TRIGMPRI(0) TRIGTYPE(NONE)
USAGE(NORMAL)
system cluster transmission queue for CLUSQM2
display queue(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
2 : display queue(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
AMQ8409: Display Queue details.
QUEUE(SYSTEM.CLUSTER.TRANSMIT.QUEUE) TYPE(QLOCAL)
ACCTQ(QMGR) ALTDATE(2016-12-30)
ALTTIME(01.45.2 BOQNAME( )
BOTHRESH(0) CLUSNL( )
CLUSTER( ) CLCHNAME( )
CLWLPRTY(0) CLWLRANK(0)
CLWLUSEQ(QMGR) CRDATE(2016-12-30)
CRTIME(01.39.15) CURDEPTH(0)
CUSTOM( ) DEFBIND(OPEN)
DEFPRTY(0) DEFPSIST(NO)
DEFPRESP(SYNC) DEFREADA(NO)
DEFSOPT(SHARED) DEFTYPE(PREDEFINED)
DESCR(WebSphere MQ Cluster Transmission Queue)
DISTL(NO) GET(ENABLED)
HARDENBO INITQ( )
IPPROCS(0) MAXDEPTH(999999999)
MAXMSGL(4194304) MONQ(QMGR)
MSGDLVSQ(PRIORITY) TRIGGER
NPMCLASS(NORMAL) OPPROCS(0)
PROCESS( ) PUT(ENABLED)
PROPCTL(COMPAT) QDEPTHHI(80)
QDEPTHLO(20) QDPHIEV(DISABLED)
QDPLOEV(DISABLED) QDPMAXEV(ENABLED)
QSVCIEV(NONE) QSVCINT(999999999)
RETINTVL(999999999) SCOPE(QMGR)
SHARE STATQ(QMGR)
TRIGDATA( ) TRIGDPTH(1)
TRIGMPRI(0) TRIGTYPE(FIRST)
USAGE(XMITQ)
system cluster transmission queue for CLUSQM1
[mqadmin@mq1 bin]$ ./runmqsc CLUSQM1
5724-H72 (C) Copyright IBM Corp. 1994, 2015.
Starting MQSC for queue manager CLUSQM1.
display queue(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
1 : display queue(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
AMQ8409: Display Queue details.
QUEUE(SYSTEM.CLUSTER.TRANSMIT.QUEUE) TYPE(QLOCAL)
ACCTQ(QMGR) ALTDATE(2016-12-30)
ALTTIME(01.45.2 BOQNAME( )
BOTHRESH(0) CLUSNL( )
CLUSTER( ) CLCHNAME( )
CLWLPRTY(0) CLWLRANK(0)
CLWLUSEQ(QMGR) CRDATE(2016-12-30)
CRTIME(01.33.01) CURDEPTH(0)
CUSTOM( ) DEFBIND(OPEN)
DEFPRTY(0) DEFPSIST(NO)
DEFPRESP(SYNC) DEFREADA(NO)
DEFSOPT(SHARED) DEFTYPE(PREDEFINED)
DESCR(WebSphere MQ Cluster Transmission Queue)
DISTL(NO) GET(ENABLED)
HARDENBO INITQ( )
IPPROCS(0) MAXDEPTH(999999999)
MAXMSGL(4194304) MONQ(QMGR)
MSGDLVSQ(PRIORITY) TRIGGER
NPMCLASS(NORMAL) OPPROCS(0)
PROCESS( ) PUT(ENABLED)
PROPCTL(COMPAT) QDEPTHHI(80)
QDEPTHLO(20) QDPHIEV(DISABLED)
QDPLOEV(DISABLED) QDPMAXEV(ENABLED)
QSVCIEV(NONE) QSVCINT(999999999)
RETINTVL(999999999) SCOPE(QMGR)
SHARE STATQ(QMGR)
TRIGDATA( ) TRIGDPTH(1)
TRIGMPRI(0) TRIGTYPE(FIRST)
USAGE(XMITQ)
above are the details you asked for. Thanks |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Jan 20, 2017 6:54 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
If you are missing some messages AND you are not expecting all messages to be on both then...
The interruption of the connection is very likely to cause the missing messages to be rolled back by WAS - and likely dropped entirely.
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| kalam475 |
| Posted: Fri Jan 20, 2017 7:34 am Post subject: |
|
|
Acolyte
Joined: 16 Jan 2015
Posts: 63
|
I am not expecting messages to be present in both the queues the total number of messages in the queues should be equal to 20000
Is there any kind of work around for this kind of problem thanks everyone for quick replies. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jan 20, 2017 8:02 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| kalam475 wrote: |
| Is there any kind of work around for this kind of problem |
Correctly handle the 2009 as I indicated above.
You've proved that the messages are not uncommitted on the queue. So your application is trying to put the message, the put / JMS send is failing with a 2009 and the application is continuing as if the message had been successfully put just because the message is persistent. Even a persistent message will be lost if it's not successfully put into the queue manager's care.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Jan 20, 2017 8:48 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
So, in summary, your app attempted to put 20000 messages, but failed to successfully put (and commit) some of them - in this instance due to a connection failure.
In such a failure, it is the responsibility of the app to reconnect and retry the put, or take other appropriate action.
A well-written app would count and echo (display) the total attempts to put, and the total number of fails.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| kalam475 |
| Posted: Sun Jan 22, 2017 9:14 pm Post subject: |
|
|
Acolyte
Joined: 16 Jan 2015
Posts: 63
|
| Can anyone provide me a link or suggest a example to incorparate mentioned features in my application i.e how to handle 2009 RC and client reconnect. Thanks |
|
| Back to top |
|
|
|
|
