ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » General IBM MQ Support » Probe id XC371020

Post new topic  Reply to topic Goto page 1, 2  Next
 Probe id XC371020 « View previous topic :: View next topic » 
Author Message
kevinf2349
PostPosted: Thu Oct 12, 2006 6:11 am    Post subject: Probe id XC371020 Reply with quote

Grand Master

Joined: 28 Feb 2003
Posts: 1311
Location: USA

OK.

We are already talking to IBM about this but I thought I would ask for ideas here too.

Yesterday afternoon one of our development Windows queue managers threw a FDC file and died.

The queue manager is showing 'Unavailable' on MQ windows explorer and all the services under the queue manager are showing 'Stopped'

The header for the FDC follows:

Code:
+-----------------------------------------------------------------------------+
|                                                                             |
| WebSphere MQ First Failure Symptom Report                                   |
| =========================================                                   |
|                                                                             |
| Date/Time         :- Wed October 11 15:51:51 Central Daylight Time 2006     |
| Host Name         :- XXXXXXK1 (Windows Ver 5.2 Build 3790: Service Pack 1)  |
| PIDS              :- 5724B4100                                              |
| LVLS              :- 530.11  CSD11                                          |
| Product Long Name :- WebSphere MQ for Windows                               |
| Vendor            :- IBM                                                    |
| Probe Id          :- XC371020                                               |
| Application Name  :- MQM                                                    |
| Component         :- xstServerRequest                                       |
| Build Date        :- Aug  2 2005                                            |
| CMVC level        :- p530-11-L050802                                        |
| Build Type        :- IKAP - (Production)                                    |
| UserID            :- MUSR_MQADMIN                                           |
| Process Name      :- C:\Program Files\IBM\WebSphere MQ\bin\AMQPCSEA.EXE     |
| Process           :- 00002172                                               |
| Thread            :- 00000001                                               |
| QueueManager      :- WINFPLDV                                               |
| Major Errorcode   :- xecF_E_UNEXPECTED_SYSTEM_RC                            |
| Minor Errorcode   :- OK                                                     |
| Probe Type        :- MSGAMQ6119                                             |
| Probe Severity    :- 2                                                      |
| Probe Description :- AMQ6119: An internal WebSphere MQ error has occurred   |
|   (Subpool Server 1 pid[2092] has terminated while application connected.)  |
| FDCSequenceNumber :- 0                                                      |
| Arith2            :- 2092 82c                                               |
| Comment1          :- Subpool Server 1 pid[2092] has terminated while        |
|   application con                                                           |
|                                                                             |
+-----------------------------------------------------------------------------+


This implies that the command server 'bought the farm'. The only hit I get on the probe id is muttering about over 300,000 persistent messages. Firstly I would be mildly surprised if this were the case, but secondly we can't get the queue manager up to find out!

Short of a reboot, has anyone got any ideas of how to get out of this?

We could reboot that that almost requires an act of congress and would mean dropping 2 working queue managers.

As I say, we are still working with IBM on this but I thought I would throw it out there.

(We are still investigating what (if anything) actually happened on the box at the time of failure) ....not really anything in the error log for the queue manager other than the command server getting a 2009 at the time of failure.

Update

We now have the queue manager started. We had to kill the process AMQRMPPA. Once this was done the queue manager started. As expected there are no where near 300,000 persistent messages on any queue nor indeed on the queue manager itself in total.

Investigations continue at a more leisurely pace
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Thu Oct 12, 2006 3:37 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20700
Location: LI,NY

Don't know how to handle shared memory under windows. Under Unix I would use amqiclen to clean up all shared memory associated with this qmgr.

Enjoy
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
Nick Lethbridge
PostPosted: Wed Oct 18, 2006 2:57 am    Post subject: Reply with quote

Voyager

Joined: 13 Aug 2001
Posts: 88
Location: Santander, UK

Hi Kevin,

I'll see your "XC371020", and raise you...

....we have started getting this as well.

+-----------------------------------------------------------------------------+
| |
| WebSphere MQ First Failure Symptom Report |
| ========================================= |
| |
| Date/Time :- Tue October 17 16:45:49 GMT Daylight Time 2006 |
| Host Name :- UKPWG174WIN (Windows Ver 5.2 Build 3790: Service Pack |
| 1) |
| PIDS :- 5724B4100 |
| LVLS :- 530.11 CSD11 |
| Product Long Name :- WebSphere MQ for Windows |
| Vendor :- IBM |
| Probe Id :- XC371020 |
| Application Name :- MQM |
| Component :- xstServerRequest |
| Build Date :- Aug 2 2005 |
| CMVC level :- p530-11-L050802 |
| Build Type :- IKAP - (Production) |
| UserID :- mqsiuat |
| Process Name :- D:\Program Files\IBM\WebSphere MQ\bin\RUNMQCHI.exe |
| Process :- 00000752 |
| Thread :- 00000001 |
| QueueManager :- WMBCMA1 |
| Major Errorcode :- xecF_E_UNEXPECTED_SYSTEM_RC |
| Minor Errorcode :- OK |
| Probe Type :- MSGAMQ6119 |
| Probe Severity :- 2 |
| Probe Description :- AMQ6119: An internal WebSphere MQ error has occurred |
| (Subpool Server 1 pid[6040] has terminated while application connected.) |
| FDCSequenceNumber :- 0 |
| Arith2 :- 6040 1798 |
| Comment1 :- Subpool Server 1 pid[6040] has terminated while |
| application con |
| |
+-----------------------------------------------------------------------------+

I tried rebooting the server, but it didn't make any difference.

Did IBM have any idea of what might be causing it ?

Perhaps it's just an "issue" with Fix Pack 11 ?

Regards,
Nick.
Back to top
View user's profile Send private message Send e-mail
Ross
PostPosted: Tue Nov 21, 2006 6:14 am    Post subject: Reply with quote

Centurion

Joined: 15 Jun 2005
Posts: 127
Location: Ireland

I am getting this too.
Did anyone figure out a reason for it?

Thanks,
Ross.
Back to top
View user's profile Send private message
jefflowrey
PostPosted: Tue Nov 21, 2006 6:19 am    Post subject: Reply with quote

Grand Poobah

Joined: 16 Oct 2002
Posts: 19981

Are you also at MQ v5.3, FixPack/CSD 11?
_________________
I am *not* the model of the modern major general.
Back to top
View user's profile Send private message
Ross
PostPosted: Tue Nov 21, 2006 6:38 am    Post subject: Reply with quote

Centurion

Joined: 15 Jun 2005
Posts: 127
Location: Ireland

It has happened at CSD10, and I upgraded to CSD12 and it still happens.
Back to top
View user's profile Send private message
Gaya3
PostPosted: Tue Nov 21, 2006 6:43 am    Post subject: Reply with quote

Jedi

Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US

Hi

This is due to large number of persistent messages in the application queue.

Please refer the below site for more information

http://www-1.ibm.com/support/docview.wss?uid=swg21178058

Thanks and Regards
Gayathri
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die
Back to top
View user's profile Send private message
jefflowrey
PostPosted: Tue Nov 21, 2006 6:45 am    Post subject: Reply with quote

Grand Poobah

Joined: 16 Oct 2002
Posts: 19981

Gaya3 wrote:
This is due to large number of persistent messages in the application queue.


It can be caused by that, yes.

It seems clear from the other posters that it doesn't have to be caused by that.

Ross - can you post the header block of your FDC - also the stack trace.
_________________
I am *not* the model of the modern major general.
Back to top
View user's profile Send private message
Ross
PostPosted: Tue Nov 21, 2006 7:03 am    Post subject: Reply with quote

Centurion

Joined: 15 Jun 2005
Posts: 127
Location: Ireland

Here is the FDC below.
I have no messages in any queue, so that isn't the problem.
How do I get the Stack Trace?
I suspect it may be to do with shared memory server or security access, but not 110% sure how to verify this.

Thanks.

+-----------------------------------------------------------------------------+
| |
| WebSphere MQ First Failure Symptom Report |
| ========================================= |
| |
| Date/Time :- Tue November 21 08:00:11 Eastern Standard Time 2006 |
| Host Name :- CBPPAP13 (Windows 2000 Build 2195: Service Pack 4) |
| PIDS :- 5724B4100 |
| LVLS :- 530.12 CSD12 |
| Product Long Name :- WebSphere MQ for Windows |
| Vendor :- IBM |
| Probe Id :- XC371020 |
| Application Name :- MQM |
| Component :- xstServerRequest |
| Build Date :- Dec 8 2005 |
| CMVC level :- p530-12-L051208 |
| Build Type :- IKAP - (Production) |
| UserID :- MUSR_MQADMIN |
| Process Name :- C:\Program Files\IBM\WebSphere MQ\bin\AMQPCSEA.EXE |
| Process :- 00001928 |
| Thread :- 00000001 |
| QueueManager :- CBPPAP13 |
| Major Errorcode :- xecF_E_UNEXPECTED_SYSTEM_RC |
| Minor Errorcode :- OK |
| Probe Type :- MSGAMQ6119 |
| Probe Severity :- 2 |
| Probe Description :- AMQ6119: An internal WebSphere MQ error has occurred |
| (Subpool Server 1 pid[1584] has terminated while application connected.) |
| FDCSequenceNumber :- 0 |
| Arith2 :- 1584 630 |
| Comment1 :- Subpool Server 1 pid[1584] has terminated while |
| application con |
| |
+-----------------------------------------------------------------------------+
Back to top
View user's profile Send private message
jefflowrey
PostPosted: Tue Nov 21, 2006 7:06 am    Post subject: Reply with quote

Grand Poobah

Joined: 16 Oct 2002
Posts: 19981

The stack trace would be the next section of the FDC.
_________________
I am *not* the model of the modern major general.
Back to top
View user's profile Send private message
Ross
PostPosted: Tue Nov 21, 2006 7:10 am    Post subject: Reply with quote

Centurion

Joined: 15 Jun 2005
Posts: 127
Location: Ireland

MQM Function Stack
pcmMain
MQGET
zstMQGET
ziiBreakConnection
zcpDetachPipe
xcsDisconnectSharedMemSet
xstDisconnectExtent
xstStorageRequest
xstServerRequest
xcsFFST
Back to top
View user's profile Send private message
jpeela
PostPosted: Tue Dec 05, 2006 1:17 am    Post subject: Reply with quote

Centurion

Joined: 23 Jan 2006
Posts: 139

Hi Kevin,

we too are under the similar situation.Did you solve yours.
If so could you share with us.

awaiting your reply.

Thanks in advance.
_________________
Jogi
Back to top
View user's profile Send private message AIM Address Yahoo Messenger
kevinf2349
PostPosted: Tue Dec 05, 2006 5:31 am    Post subject: Reply with quote

Grand Master

Joined: 28 Feb 2003
Posts: 1311
Location: USA

jpeela wrote:
Hi Kevin,

we too are under the similar situation.Did you solve yours.
If so could you share with us.

awaiting your reply.

Thanks in advance.


Quote:
Update

We now have the queue manager started. We had to kill the process AMQRMPPA. Once this was done the queue manager started. As expected there are no where near 300,000 persistent messages on any queue nor indeed on the queue manager itself in total.

Back to top
View user's profile Send private message
Ross
PostPosted: Tue Dec 05, 2006 5:57 am    Post subject: Reply with quote

Centurion

Joined: 15 Jun 2005
Posts: 127
Location: Ireland

I was able to get this error by starting the qmgr using strmqm command. It works fine if started with MQ Explorer.
How are you starting it?
Are you using BMC Patrol?
What MQ Version/CSD?
Back to top
View user's profile Send private message
jpeela
PostPosted: Tue Dec 05, 2006 9:47 pm    Post subject: Reply with quote

Centurion

Joined: 23 Jan 2006
Posts: 139

Hi all,

We too were abble to start the quemanager successfully and the qmgr is running fine presently, but the quemanager is ending unexpectedly frequently for the last few days.We dont know the exact reason , why this is happening.And we doubt that this would happen again.Any remedy against this.

We are using MQ5.3 on windows with CSD7.BUt CSD 12 also does not solve this problem As for the previous posts.Any solution or rsuggesions

Thanx in advance
_________________
Jogi
Back to top
View user's profile Send private message AIM Address Yahoo Messenger
Display posts from previous:   
Post new topic  Reply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum Index » General IBM MQ Support » Probe id XC371020
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.