MQSeries.net :: View topic - Messages stuck in SYSTEM.CLUSTER.TRANSMIT.QUEUE and refuse

MQSeries.net

Tech Exchange

Education

Certifications

Library

Info Center

SupportPacs

FAQÂ Â

Usergroups

RSS Feed - WebSphere MQ Support

RSS Feed - Message Broker Support

MQSeries.net Forum Index » Clustering » Messages stuck in SYSTEM.CLUSTER.TRANSMIT.QUEUE and refuse

Goto page Previous 1, 2, 3 Next

Messages stuck in SYSTEM.CLUSTER.TRANSMIT.QUEUE and refuse

« View previous topic :: View next topic »

Author

Message

mikeHT

Posted: Sun Mar 19, 2006 4:20 pm Post subject:

Voyager

Joined: 01 Jul 2005
Posts: 82

I got uncomitted messages in SCTQ again (4) and I did a amqsbcg and I can see the contents of the 4 msgs. But there is no destination qmgr to be seen in any of the 4 message descriptor. How to get the MQXQH? Is there a tool other than amqsbcg? Thank you

PeterPotkay

Posted: Sun Mar 19, 2006 4:53 pm Post subject:

Poobah

Joined: 15 May 2001
Posts: 7717

mikeHT wrote:

I got uncomitted messages in SCTQ again (4) and I did a amqsbcg and I can see the contents of the 4 msgs.

They are commited, otherwise amqsbcg wouldn't be able to browse them to display them.

mikeHT wrote:

But there is no destination qmgr to be seen in any of the 4 message descriptor.

Once the cluster workload algorithim decides where a message is to go in the cluster, it puts it on the S.C.T.Q. with a new MQMD (the original MQMD is pushed into the message buffer to be restored on the receiving end). That new MQMD's CorrelID will have the name of the channel that should take that message. Hopefully all your cluster channels are named something like TO.QueueManageNameHere. By looking at this Correl ID, you can tell what the destination QM is for that message.

mikeHT wrote:

How to get the MQXQH? Is there a tool other than amqsbcg?

Any tool that can browse a message and display its contents will do just fine.
_________________
Peter Potkay
Keep Calm and MQ On

mikeHT

Posted: Sun Mar 19, 2006 5:44 pm Post subject:

Voyager

Joined: 01 Jul 2005
Posts: 82

dis qs(SYSTEM.CLUSTER.TRANSMIT.QUEUE) type(queue) all
AMQ8450: Display queue status details.
QUEUE(SYSTEM.CLUSTER.TRANSMIT.QUEUE) IPPROCS(2)
OPPROCS(25) CURDEPTH(4)
UNCOM(YES)

But why is UNCOM is Yes? UNCOM stands for uncommitted right?
As of now, the 4 msgs still stuck and any subsequent msg to the remote QMGR-Full-Repos is stuck. How can I force the UNCOM to be No?
Thanks.

fjb_saper

Posted: Mon Mar 20, 2006 4:32 am Post subject:

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

Could there be that there is a "poison message" in the System.cluster.xmitq?
A message to big for the channel? Said message would be in the channels LUW and not necessarily visible in the xmitq until the channel is stopped...

So yes the rest of the messages behind it that are not in the same LUW should be visible and they will look like they are stuck.

Enjoy

_________________
MQ & Broker admin

Nigelg

Posted: Mon Mar 20, 2006 4:59 am Post subject:

Grand Master

Joined: 02 Aug 2004
Posts: 1046

One of the cluster channels must be INDOUBT. Find that channel from the DIS CHS list, and start the channel.
_________________
MQSeries.net helps those who help themselves..

mikeHT

Posted: Mon Mar 20, 2006 6:25 am Post subject:

Voyager

Joined: 01 Jul 2005
Posts: 82

fjb_saper wrote:

The maxmsgl of the clussdr/clusrcvr is set to 100000000.

jefflowrey

Posted: Mon Mar 20, 2006 8:36 am Post subject:

Grand Poobah

Joined: 16 Oct 2002
Posts: 19981

mikeHT wrote:

The maxmsgl of the clussdr/clusrcvr is set to 100000000.

104857600 is the actual max.

What about Max msgln on SCTQ, and the queue managers, and the destination queue?

The MQXQH is put onto the message body immediately following the MQMD. So it will be the first several hundred bytes.

So, again, post the output from AMQSBCG that has been clipped to the first character of your message data - this will be after the MQXQH and the original MQMD.
_________________
I am *not* the model of the modern major general.

Nigelg

Posted: Mon Mar 20, 2006 8:54 am Post subject:

Grand Master

Joined: 02 Aug 2004
Posts: 1046

I think you guys are barking up the wrong tree with this stuff about msg size.

Quote:

A message to big for the channel? Said message would be in the channels LUW and not necessarily visible in the xmitq

No it wouldn't. The msg would not get as far as the LUW.
If there were a msg on the xmitq that was too big for the channel it would not cause the channel to go indoubt, or be uncommitted; the channel status would be RETRYING if there was no local DLQ (or the msg was too big for the DLQ), and RUNNING if there were a local DLQ with MAXMSGL large enought to take the msg.
This assumes a persistent msg or NPMSPEED(NORMAL); a non-persistent msg with NPMSPEED(FAST) would just be discarded.
amqsbcg output will not help; that will just show the msgs which are still in the queue, not the indoubt msgs. If there are any msgs in the queue, they must be for STOPPED or RETRYING channels.
Post the output from DIS CHS(*) INDOUBT.
_________________
MQSeries.net helps those who help themselves..

mikeHT

Posted: Mon Mar 20, 2006 10:50 am Post subject:

Voyager

Joined: 01 Jul 2005
Posts: 82

dis chs(*) indoubt
80 : dis chs(*) indoubt
AMQ8417: Display Channel Status details.
CHANNEL(TO.QMGR-Full-Repost) XMITQ(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
CONNAME(192.168.231.16(1414)) CURRENT
CHLTYPE(CLUSSDR) INDOUBT(NO)
STATUS(RUNNING) RQMNAME(QMGR-Full-Repost)
AMQ8417: Display Channel Status details.
CHANNEL(TO.LocalQMGR) XMITQ( )
CONNAME(192.168.231.16) CURRENT
CHLTYPE(CLUSRCVR) INDOUBT(NO)
STATUS(RUNNING) RQMNAME(QMGR-Full-Repost)
dis qs(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
81 : dis qs(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
AMQ8450: Display queue status details.
QUEUE(SYSTEM.CLUSTER.TRANSMIT.QUEUE) IPPROCS(2)
OPPROCS(31) CURDEPTH(0)
UNCOM(NO)

The maxmsgl on SCTQ, lq is all set to 100000000. I'll go run the applications which puts to the qa which targets to localqueue on QMGR-Full-Repos

PeterPotkay

Posted: Mon Mar 20, 2006 10:52 am Post subject:

Poobah

Joined: 15 May 2001
Posts: 7717

Nigelg wrote:

Post the output from DIS CHS(*) INDOUBT.

Yes, I am curious about the results of this.

Mike,
In the meantime.....
When I have messages stuck on a S.C.T.Q., I browse them and look at the Correl ID. That Correl ID tells me what channel is supposed to pick up that message. Once I have that channel name, I go look at it, and fix the problem.

90%+ of "cluster" problems are nothing more than simple channel problems.

I created a clustered q on only 1 QM in the cluster. I sen tmessages to it from another QM in the cluster to prove it works. I then STOPPED the CLUSRCVR on that destination QM, to simulate a channel problem. When I went to put another message to that q on another QM, it got stuck in that S.C.T.Q. Browsing that queue gave me the below. Converting the MQMD's Correl ID gives me TO.HIGGWL1.CLUSFL1, which gave me A. the name of the destination QM (HIGGWL1 because of our naming standards) and B. the channel that should be looked at in this case (TO.HIGGWL1.CLUSFL1). which of course was simply retrying.

When looking for the destination QM in these scenarios, the MQXQH doesn't really provide any info. Its all in the MQMD.

Code:

X:\>amqsbcgc SYSTEM.CLUSTER.TRANSMIT.QUEUE

AMQSBCG0 - starts here
**********************

MQOPEN - 'SYSTEM.CLUSTER.TRANSMIT.QUEUE'

MQGET of message number 1
****Message descriptor****

StrucId : 'MD ' Version : 2
Report : 0 MsgType : 8
Expiry : -1 Feedback : 0
Encoding : 546 CodedCharSetId : 437
Format : 'MQXMIT '
Priority : 0 Persistence : 0
MsgId : X'414D51204849474D51494C312020202092021E4420000112'
CorrelId : X'544F2E48494747574C312E434C5553464C31202020202020'
BackoutCount : 0
ReplyToQ : ' '
ReplyToQMgr : 'HIGMQIL1 '
** Identity Context
UserIdentifier : 'mqappsu1 '
AccountingToken :
X'0100000000000000000000000000000000000000000000000000000000000000'
ApplIdentityData : ' '
** Origin Context
PutApplType : '7'
PutApplName : 'HIGMQIL1 '
PutDate : '20060320' PutTime : '18240502'
ApplOriginData : ' '

GroupId : X'000000000000000000000000000000000000000000000000'
MsgSeqNumber : '1'
Offset : '0'
MsgFlags : '0'
OriginalLength : '-1'

**** Message ****

length - 439 bytes

00000000: 5851 4820 0100 0000 4849 472E 4849 4747 'XQH ....HIG.HIGG'
00000010: 574C 312E 4C4F 4341 4C2E 5155 4555 452E 'WL1.LOCAL.QUEUE.'
00000020: 434C 5553 464C 3120 2020 2020 2020 2020 'CLUSFL1 '
00000030: 2020 2020 2020 2020 2020 2020 2020 2020 ' '
00000040: 2020 2020 2020 2020 2020 2020 2020 2020 ' '
00000050: 2020 2020 2020 2020 2020 2020 2020 2020 ' '
00000060: 2020 2020 2020 2020 4D44 2020 0100 0000 ' MD ....'
00000070: 0000 0000 0800 0000 FFFF FFFF 0000 0000 '........Â Â Â Â ....'
00000080: 2202 0000 B501 0000 4D51 5354 5220 2020 '"...╡...MQSTR '
00000090: 0000 0000 0000 0000 414D 5120 4849 474D '........AMQ HIGM'
000000A0: 5149 4C31 2020 2020 9202 1E44 2000 3201 'QIL1 Ã†..D .2.'
000000B0: 0000 0000 0000 0000 0000 0000 0000 0000 '................'
000000C0: 0000 0000 0000 0000 0000 0000 2020 2020 '............ '
000000D0: 2020 2020 2020 2020 2020 2020 2020 2020 ' '
000000E0: 2020 2020 2020 2020 2020 2020 2020 2020 ' '
000000F0: 2020 2020 2020 2020 2020 2020 4849 474D ' HIGM'
00000100: 5149 4C31 2020 2020 2020 2020 2020 2020 'QIL1 '
00000110: 2020 2020 2020 2020 2020 2020 2020 2020 ' '
00000120: 2020 2020 2020 2020 2020 2020 6D71 6170 ' mqap'
00000130: 7073 7531 2020 2020 1601 0515 0000 009E 'psu1 .......₧'
00000140: 407E 1442 C774 5223 5F63 6B22 3A02 0000 '@~.B╟tR#_ck":...'
00000150: 0000 0000 0000 000B 2020 2020 2020 2020 '........ '
00000160: 2020 2020 2020 2020 2020 2020 2020 2020 ' '
00000170: 2020 2020 2020 2020 0B00 0000 6562 5370 ' ....ebSp'
00000180: 6865 7265 204D 515C 6269 6E5C 616D 7173 'here MQ\bin\amqs'
00000190: 7075 7463 2E65 7865 3230 3036 3033 3230 'putc.exe20060320'
000001A0: 3138 3233 3236 3035 2020 2020 4845 4C4C '18232605 HELL'
000001B0: 4F20 574F 524C 44 'O WORLD '

No more messages
MQCLOSE
MQDISC
X:\>

_________________
Peter Potkay
Keep Calm and MQ On

mikeHT

Posted: Mon Mar 20, 2006 10:58 am Post subject:

Voyager

Joined: 01 Jul 2005
Posts: 82

After application is submitted, UNCOM - becomes YES, INDOUBT - No
dis qs(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
84 : dis qs(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
AMQ8450: Display queue status details.
QUEUE(SYSTEM.CLUSTER.TRANSMIT.QUEUE) IPPROCS(2)
OPPROCS(35) CURDEPTH(1)
UNCOM(YES)
dis chs(*) indoubt
85 : dis chs(*) indoubt
AMQ8417: Display Channel Status details.
CHANNEL(TO.QMGR-Full-Repos) XMITQ(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
CONNAME(192.168.231.16(1414)) CURRENT
CHLTYPE(CLUSSDR) INDOUBT(NO)
STATUS(RUNNING) RQMNAME(QMGR-Full-Repos)
AMQ8417: Display Channel Status details.
CHANNEL(LocalQMGR) XMITQ( )
CONNAME(192.168.231.16) CURRENT
CHLTYPE(CLUSRCVR) INDOUBT(NO)
STATUS(RUNNING) RQMNAME(QMGR-Full-Repos)
dis qs(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
86 : dis qs(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
AMQ8450: Display queue status details.
QUEUE(SYSTEM.CLUSTER.TRANSMIT.QUEUE) IPPROCS(2)
OPPROCS(35) CURDEPTH(1)
UNCOM(YES)
dis qs(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
87 : dis qs(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
AMQ8450: Display queue status details.
QUEUE(SYSTEM.CLUSTER.TRANSMIT.QUEUE) IPPROCS(2)
OPPROCS(35) CURDEPTH(126)
UNCOM(YES)
dis chs(*) indoubt
88 : dis chs(*) indoubt
AMQ8417: Display Channel Status details.
CHANNEL(TO.QMGR-Full-Repos) XMITQ(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
CONNAME(192.168.231.16(1414)) CURRENT
CHLTYPE(CLUSSDR) INDOUBT(NO)
STATUS(RUNNING) RQMNAME(QMGR-Full-Repos)
AMQ8417: Display Channel Status details.
CHANNEL(LocalQMGR) XMITQ( )
CONNAME(192.168.231.16) CURRENT
CHLTYPE(CLUSRCVR) INDOUBT(NO)
STATUS(RUNNING) RQMNAME(QMGR-Full-Repos)

mikeHT

Posted: Mon Mar 20, 2006 11:10 am Post subject:

Voyager

Joined: 01 Jul 2005
Posts: 82

PeterPotkay wrote:

Nigelg wrote:

Post the output from DIS CHS(*) INDOUBT.

Code:

Converting the MQMD's Correl ID gives me TO.HIGGWL1.CLUSFL1, which gave me A. the name of the destination QM (HIGGWL1 because of our naming standards) and B. the channel that should be looked at in this case (TO.HIGGWL1.CLUSFL1). which of course was simply retrying.

How to convert Correl ID to the channel name? Thanks

PeterPotkay

Posted: Mon Mar 20, 2006 12:10 pm Post subject:

Poobah

Joined: 15 May 2001
Posts: 7717

mikeHT wrote:

How to convert Correl ID to the channel name? Thanks

I use QPASA with its built in Hex to ASCII convertor for IDs.

Anything that converts Hex to ASCII or EBCIDIC will do it for you.
_________________
Peter Potkay
Keep Calm and MQ On

mikeHT

Posted: Mon Mar 20, 2006 3:41 pm Post subject:

Voyager

Joined: 01 Jul 2005
Posts: 82

Converting Correl ID to ascii, gives me the TO.QMGR-Full-Repos names.
But from the dis chs(*) gives, which displays RUNNING and UNCOM still at YES and CURDEPTH at 9.
Now if I use qload
CHANNEL(TO.QMGR-Full-Repos) XMITQ(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
CONNAME(192.168.231.16(1414)) CURRENT
CHLTYPE(CLUSSDR) INDOUBT(NO)
STATUS(RUNNING) RQMNAME(QMGR-Full-Repos)
dis qs(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
114 : dis qs(SYSTEM.CLUSTER.TRANSMIT.QUEUE)
AMQ8450: Display queue status details.
QUEUE(SYSTEM.CLUSTER.TRANSMIT.QUEUE) IPPROCS(2)
OPPROCS(36) CURDEPTH(9)
UNCOM(YES)

C:\>qload -iSYSTEM.CLUSTER.TRANSMIT.QUEUE -fmonday-6pm.bk
QLOAD Program by Paul Clarke [ V1.0 Build:Jul 13 2005 ]
Messages Read : 9 Bytes Read : 147800
Messages Written: 9 Bytes Written: 147800
Why is it that UNCOM is YES and I can still use qload to backup the files in SCTQ?

jefflowrey

Posted: Mon Mar 20, 2006 3:47 pm Post subject:

Grand Poobah

Joined: 16 Oct 2002
Posts: 19981

Because the uncommitted message is not the one you're seeing..
_________________
I am *not* the model of the modern major general.

Display posts from previous:

Goto page Previous 1, 2, 3 Next

Page 2 of 3

MQSeries.net Forum Index » Clustering » Messages stuck in SYSTEM.CLUSTER.TRANSMIT.QUEUE and refuse

Jump to:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

Protected by Anti-Spam ACP