| Author |
Message
|
| manishpnp |
 Posted: Tue Jun 28, 2011 4:01 am Post subject: Cluster transmission queue - slow process |
 |
|
Apprentice
Joined: 21 Feb 2008
Posts: 46
|
Hello All,
WebSphere MQ 7.0.1.2 in AIX 5.3
WebSphere Message Broker 7.0 in AIX 5.3
Broker1 - RABBROKER1 (Broker1 Queue Manager - RABQMGR1)
Broker2 - RAMBROKER2 (Broker2 Queue Manager - RABQMGR2)
RABQMGR1 and RABQMGR2 queue managers are members of MQ cluster (RABCLUSTER).
There are multiple queues setup as a cluster queue for RABCLUSTER.
For example :-
runmqsc RABQMGR1
dis qcluster(RABSAP.ESB)
1 : dis qcluster(RABSAP.ESB)
AMQ8409: Display Queue details.
QUEUE(RABSAP.ESB) TYPE(QCLUSTER)
ALTDATE(2011-03-01) ALTTIME(10.24.26)
CLUSDATE(2011-04-16) CLUSTER(RABCLUSTER)
CLUSQMGR(RABQMGR2) CLUSQT(QLOCAL)
CLUSTIME(13.38.39) CLWLPRTY(0)
CLWLRANK(0) DEFBIND(NOTFIXED)
DEFPRTY(0) DEFPSIST(NO)
DEFPRESP(SYNC) DESCR(Response Queue for SAP)
PUT(ENABLED) QMID(RABQMGR2_2011-02-19_14.03.36)
AMQ8409: Display Queue details.
QUEUE(RABSAP.ESB) TYPE(QCLUSTER)
ALTDATE(2011-02-2 ALTTIME(18.02.33)
CLUSDATE(2011-02-2 CLUSTER(RABCLUSTER)
CLUSQMGR(RABQMGR1) CLUSQT(QLOCAL)
CLUSTIME(18.02.33) CLWLPRTY(0)
CLWLRANK(0) DEFBIND(NOTFIXED)
DEFPRTY(0) DEFPSIST(NO)
DEFPRESP(SYNC) DESCR(Response Queue for SAP)
PUT(ENABLED) QMID(RABQMGR1_2011-02-28_17.58.53)
Cluster is working fine and all messages are processess fine with Queue Manager and Broker.
Problem :- For the maintenance, when we stopped any queue manager or Broker (for example we stopped - RABQMGR1/RABBROKER1), we found that all messages are stuck in RABQMGR1 - cluster transmission queue and RABQMGR2/RAMBROKER2 is taking more than 20 to 30 minutes to process the pending queues from cluser tranmission queues.
We have the SLA defined for 3 to 4 minutes for every messages, but due to slow processing from cluster transmission queue, our SLA has breach many times.
Can anyone please share your experience why queues stuck in cluster transmission queue for such long time.
Please share your experience / inputs to resolve this problem.
Thanks,
manishP |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Tue Jun 28, 2011 4:04 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| The most likely cause is that your applications are using BIND_ON_OPEN. |
|
| Back to top |
|
|
| manishpnp |
| Posted: Tue Jun 28, 2011 5:23 am Post subject: |
|
|
Apprentice
Joined: 21 Feb 2008
Posts: 46
|
Application are using 'Bind not Fixed' option.
As mentioned, When both queue manager up and running everything works fine.
we are facing issue only when we stopped the queue manager or broker (for example - when we stopped - RABQMGR1/RABBROKER1), we found that all messages are stuck in RABQMGR1 - cluster transmission queue and RABQMGR2/RAMBROKER2 is taking more than 20 to 30 minutes to process the pending queues from cluser tranmission queues.
Thanks,
ManishP
. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Jun 28, 2011 5:29 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| manishpnp wrote: |
| Application are using 'Bind not Fixed' option. |
Are you sure?
Are the messages sitting on the cluster transmit queue all destined to the stopped qmgr? or to all qmgrs in the cluster? |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Jun 28, 2011 5:47 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| mqjeff wrote: |
| manishpnp wrote: |
| Application are using 'Bind not Fixed' option. |
Are you sure?
|
The DEFBIND(NOTFIXED) attribute of the queue does not determine whether the app opens the queue fixed/not fixed. Rather, it is the application specifying FIXED or NOTFIXED or AS_Q_DEF.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Jun 28, 2011 9:32 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
And even if the app is using Bind Not Fixed explicitly, it doesn't matter if they chose to address the message to a specific queue / Queue Manager combination on the MQOPEN or MQPUT1 call.
BUT, if this was the case, or Bind On Open was the culprit, it would not be "slow". They would never be moved to an alternate QM.
What is the CLUSRCVR channel's short retry interval? I think that is what drives how quickly the cluster algorithm will look at stranded messages on the S.C.T.Q. to see if they are eligible to be routed to another QM that is up. Not sure though.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Jun 28, 2011 10:15 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| manishpnp wrote: |
Application are using 'Bind not Fixed' option.
As mentioned, When both queue manager up and running everything works fine.
we are facing issue only when we stopped the queue manager or broker (for example - when we stopped - RABQMGR1/RABBROKER1), we found that all messages are stuck in RABQMGR1 - cluster transmission queue and RABQMGR2/RAMBROKER2 is taking more than 20 to 30 minutes to process the pending queues from cluser tranmission queues.
Thanks,
ManishP
. |
Did you verify the status of the channel to the qmgr that is up?
Are all the messages persistent?
Is the channel in retrying status or in doubt status?
I have found that non persistent msgs take way longer to travel a channel in doubt than a channel with normal status.
Also another way to make sure that the channel is awfully slow is to have a full queue on the destination qmgr. The time the channel takes to redeliver and the number of attempts (all configurable) before it puts the msg to the DLQ will significantly slow down msg delivery and channel performance...
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
