ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » General IBM MQ Support » Solved - Queue Manager start problem - FDC Genereted

Post new topic  Reply to topic
 Solved - Queue Manager start problem - FDC Genereted « View previous topic :: View next topic » 
Author Message
mhashkhan
PostPosted: Sun Mar 06, 2011 11:08 pm    Post subject: Solved - Queue Manager start problem - FDC Genereted Reply with quote

Apprentice

Joined: 01 Dec 2008
Posts: 28
Hi All,

I am hitting error on starting the Queue manager, find below the MQ error

Find below the FDC report

+-----------------------------------------------------------------------------+
| |
| WebSphere MQ First Failure Symptom Report |
| ========================================= |
| |
| Date/Time :- Monday March 07 06:07:57 UTC 2011 |
| Host Name :- ebbsTapp01 (Linux 2.6.9-67.ELsmp) |
| PIDS :- 5724H7204 |
| LVLS :- 6.0.2.6 |
| Product Long Name :- WebSphere MQ for Linux (x86 platform) |
| Vendor :- IBM |
| Probe Id :- XC381001 |
| Application Name :- MQM |
| Component :- xcsDisplayMessageForSubpool |
| SCCS Info :- lib/cs/amqxeida.c, 1.52.1.24 |
| Line Number :- 3466 |
| Build Date :- Feb 17 2009 |
| CMVC level :- p600-206-090217 |
| Build Type :- IKAP - (Production) |
| UserID :- 00000507 (isismqms) |
| Program Name :- strmqm |
| Addressing mode :- 32-bit |
| Process :- 22521 |
| Thread-Process :- 22521 |
| Thread :- 1 |
| ThreadingModel :- PosixThreads |
| QueueManager :- HKEBG1S3 |
| Last HQC :- 0.0.0-0 |
| Last HSHMEMB :- 0.0.0-0 |
| Major Errorcode :- MQRC_UNEXPECTED_ERROR |
| Minor Errorcode :- OK |
| Probe Type :- MSGAMQ0893 |
| Probe Severity :- 4 |
| Probe Description :- AMQ6090: WebSphere MQ was unable to display an error |
| message 893. |
| FDCSequenceNumber :- 0 |
| |
+-----------------------------------------------------------------------------+

MQM Function Stack
xcsDisplayMessageForSubpool
xcsFFST

MQM Trace History
---} xusGetLineType rc=OK
---{ xusAddStanzaLineList
----{ xcsGetMem
----} xcsGetMem rc=OK
---} xusAddStanzaLineList rc=OK
---{ xusGetLineType
---} xusGetLineType rc=OK
---{ xusAddStanzaLineList
----{ xcsGetMem
----} xcsGetMem rc=OK
---} xusAddStanzaLineList rc=OK
---{ xusGetLineType
---} xusGetLineType rc=OK
---{ xusAddStanzaLineList
----{ xcsGetMem
----} xcsGetMem rc=OK
---} xusAddStanzaLineList rc=OK
---{ xusGetLineType
---} xusGetLineType rc=OK
---{ xusAddStanzaLineList
----{ xcsGetMem
----} xcsGetMem rc=OK
---} xusAddStanzaLineList rc=OK
--} xusReadSingleStanza rc=OK
--{ xusDelStanzaLineList
---{ xcsFreeMem
---} xcsFreeMem rc=OK
--} xusDelStanzaLineList rc=OK
--{ xusDelStanzaLineList
---{ xcsFreeMem
---} xcsFreeMem rc=OK
--} xusDelStanzaLineList rc=OK
--{ xusDelStanzaLineList
---{ xcsFreeMem
---} xcsFreeMem rc=OK
--} xusDelStanzaLineList rc=OK
--{ xusDelStanzaLineList
---{ xcsFreeMem
---} xcsFreeMem rc=OK
--} xusDelStanzaLineList rc=OK
--{ xusDelStanzaLineList
---{ xcsFreeMem
---} xcsFreeMem rc=OK
--} xusDelStanzaLineList rc=OK
--{ xufCloseIniRead
--} xufCloseIniRead rc=OK
-} xgrGetQMIniRestrictedMode rc=OK
} xgrValidateUser rc=OK
{ zutEmptyDirectory
-{ xcsReaddir
-} xcsReaddir rc=OK
-{ xcsReaddir
-} xcsReaddir rc=OK
-{ xcsReaddir
-} xcsReaddir rc=OK
} zutEmptyDirectory rc=OK
{ zutEmptyDirectory
-{ xcsReaddir
-} xcsReaddir rc=OK
-{ xcsReaddir
-} xcsReaddir rc=OK
-{ xcsReaddir
-} xcsReaddir rc=OK
} zutEmptyDirectory rc=OK
{ zslStartEC
-{ xcsQueryValueForSubpool
-} xcsQueryValueForSubpool rc=OK
-{ zutWriteECRCfile
-} zutWriteECRCfile rc=OK
-{ xcsExecProgram
--{ xcsRegisterAsySigHandler
---{ xcsRequestThreadMutexSem
---} xcsRequestThreadMutexSem rc=OK
---{ xcsGetMem
---} xcsGetMem rc=OK
---{ xcsReleaseThreadMutexSem
---} xcsReleaseThreadMutexSem rc=OK
--} xcsRegisterAsySigHandler rc=OK
--{ xcsUnregisterAsySigHandler
---{ xcsRequestThreadMutexSem
---} xcsRequestThreadMutexSem rc=OK
---{ xcsFreeMem
---} xcsFreeMem rc=OK
---{ xcsReleaseThreadMutexSem
---} xcsReleaseThreadMutexSem rc=OK
--} xcsUnregisterAsySigHandler rc=OK
Data: 0x000057fd
-} xcsExecProgram rc=OK
} zslStartEC rc=OK
{ zslWaitEC
-{ zslDisplayMessage
--{ xcsGetMessage
---{ xcsQueryCCSIDType
---} xcsQueryCCSIDType rc=OK
---{ xcsConvertString
---} xcsConvertString rc=OK
--} xcsGetMessage rc=OK
-} zslDisplayMessage rc=OK
-{ xcsSleep
-} xcsSleep rc=OK
-{ zslCheckIfRunning
--{ xcsInitialize
---{ xcsRequestThreadMutexSem
---} xcsRequestThreadMutexSem rc=OK
---{ xcsStartPersistentTimerThread
----{ xtmStartTimerThread
-----{ xcsCreateThread
-----} xcsCreateThread rc=OK
----} xtmStartTimerThread rc=OK
---} xcsStartPersistentTimerThread rc=OK
---{ xcsConnectSharedSubpool
----{ xcsAttachSharedSubpool
-----{ xcsGetMem
-----} xcsGetMem rc=OK
-----{ xcsCreateThreadMutexSem
-----} xcsCreateThreadMutexSem rc=OK
-----{ xcsCreateThreadMutexSem
-----} xcsCreateThreadMutexSem rc=OK
-----{ xcsGetMem
-----} xcsGetMem rc=OK
-----{ xcsConnectSharedMemSet
------{ xcsRequestThreadMutexSem
------} xcsRequestThreadMutexSem rc=OK
------{ xcsRequestThreadMutexSem
------} xcsRequestThreadMutexSem rc=OK
------{ xcsReleaseThreadMutexSem
------} xcsReleaseThreadMutexSem rc=OK
------{ xstAttachSharedMemSet
-------{ xstConnectExtent
--------{ xcsRequestThreadMutexSem
--------} xcsRequestThreadMutexSem rc=OK
--------{ xstVerifySysVIdInFile
--------} xstVerifySysVIdInFile rc=xecX_E_NO_SUCH_EXTENT
--------{ xcsReleaseThreadMutexSem
--------} xcsReleaseThreadMutexSem rc=OK
-------} xstConnectExtent rc=xecX_E_NO_SUCH_EXTENT
------} xstAttachSharedMemSet rc=xecX_E_NO_SUCH_EXTENT
------{ xcsReleaseThreadMutexSem
------} xcsReleaseThreadMutexSem rc=OK
-----} xcsConnectSharedMemSet rc=xecX_E_NO_SUCH_EXTENT
-----{ xcsFreeMem
-----} xcsFreeMem rc=OK
-----{ xcsDestroyThreadMutexSem
-----} xcsDestroyThreadMutexSem rc=OK
-----{ xcsDestroyThreadMutexSem
-----} xcsDestroyThreadMutexSem rc=OK
-----{ xcsFreeMem
-----} xcsFreeMem rc=OK
----} xcsAttachSharedSubpool rc=xecX_E_NO_SUCH_EXTENT
---} xcsConnectSharedSubpool rc=xecI_E_NO_SUCH_SUBPOOL
---{ xtmStopTimerThread
----{ xcsWaitThread
----} xcsWaitThread rc=OK
---} xtmStopTimerThread rc=OK
---{ xcsReleaseThreadMutexSem
---} xcsReleaseThreadMutexSem rc=OK
--} xcsInitialize rc=xecI_S_FAIL
-} zslCheckIfRunning rc=lrcE_Q_MGR_NOT_AVAILABLE
-{ xcsCheckProcess
Data: 0x000057fd
-} xcsCheckProcess rc=xecP_E_INVALID_PID
-{ lpiSPIQueryQMStatus
--{ xcsInitialize
---{ xcsRequestThreadMutexSem
---} xcsRequestThreadMutexSem rc=OK
---{ xcsStartPersistentTimerThread
----{ xtmStartTimerThread
-----{ xcsCreateThread
-----} xcsCreateThread rc=OK
----} xtmStartTimerThread rc=OK
---} xcsStartPersistentTimerThread rc=OK
---{ xcsConnectSharedSubpool
----{ xcsAttachSharedSubpool
-----{ xcsGetMem
-----} xcsGetMem rc=OK
-----{ xcsCreateThreadMutexSem
-----} xcsCreateThreadMutexSem rc=OK
-----{ xcsCreateThreadMutexSem
-----} xcsCreateThreadMutexSem rc=OK
-----{ xcsGetMem
-----} xcsGetMem rc=OK
-----{ xcsConnectSharedMemSet
------{ xcsRequestThreadMutexSem
------} xcsRequestThreadMutexSem rc=OK
------{ xcsRequestThreadMutexSem
------} xcsRequestThreadMutexSem rc=OK
------{ xcsReleaseThreadMutexSem
------} xcsReleaseThreadMutexSem rc=OK
------{ xstAttachSharedMemSet
-------{ xstConnectExtent
--------{ xcsRequestThreadMutexSem
--------} xcsRequestThreadMutexSem rc=OK
--------{ xstVerifySysVIdInFile
--------} xstVerifySysVIdInFile rc=xecX_E_NO_SUCH_EXTENT
--------{ xcsReleaseThreadMutexSem
--------} xcsReleaseThreadMutexSem rc=OK
-------} xstConnectExtent rc=xecX_E_NO_SUCH_EXTENT
------} xstAttachSharedMemSet rc=xecX_E_NO_SUCH_EXTENT
------{ xcsReleaseThreadMutexSem
------} xcsReleaseThreadMutexSem rc=OK
-----} xcsConnectSharedMemSet rc=xecX_E_NO_SUCH_EXTENT
-----{ xcsFreeMem
-----} xcsFreeMem rc=OK
-----{ xcsDestroyThreadMutexSem
-----} xcsDestroyThreadMutexSem rc=OK
-----{ xcsDestroyThreadMutexSem
-----} xcsDestroyThreadMutexSem rc=OK
-----{ xcsFreeMem
-----} xcsFreeMem rc=OK
----} xcsAttachSharedSubpool rc=xecX_E_NO_SUCH_EXTENT
---} xcsConnectSharedSubpool rc=xecI_E_NO_SUCH_SUBPOOL
---{ xtmStopTimerThread
----{ xcsWaitThread
----} xcsWaitThread rc=OK
---} xtmStopTimerThread rc=OK
---{ xcsReleaseThreadMutexSem
---} xcsReleaseThreadMutexSem rc=OK
--} xcsInitialize rc=xecI_S_FAIL
-} lpiSPIQueryQMStatus rc=krcE_UNEXPECTED_ERROR
-{ zutReadECRCfile
--{ xcsGetMem
--} xcsGetMem rc=OK
-} zutReadECRCfile rc=OK
} zslWaitEC rc=krcE_UNEXPECTED_ERROR
{ xcsDisplayMessageForSubpool
-{ xcsQueryProcessDetails
-} xcsQueryProcessDetails rc=OK
-{ xcsGetEnvironmentString
-} xcsGetEnvironmentString rc=xecE_E_ENV_VAR_NOT_FOUND
-{ xcsGetEnvironmentInteger
--{ xcsGetEnvironmentString
--} xcsGetEnvironmentString rc=xecE_E_ENV_VAR_NOT_FOUND
-} xcsGetEnvironmentInteger rc=xecE_E_ENV_VAR_NOT_FOUND
-{ xcsQueryValueForSubpool
-} xcsQueryValueForSubpool rc=OK
-{ xcsDisplayMessage
--{ xcsGetMem
--} xcsGetMem rc=OK
--{ xcsGetMessage
---{ xcsQueryCCSIDType
---} xcsQueryCCSIDType rc=OK
---{ xcsConvertString
---} xcsConvertString rc=OK
--} xcsGetMessage rc=xecE_W_DEFAULT_TEXT_SUBSTITUTED
--{ xcsFreeMem
--} xcsFreeMem rc=OK
-} xcsDisplayMessage rc=xecE_E_NO_DISP_MSG
-{ xcsFFST



Please help me to understand what is wrong
OS = Linux RHEL4-U6-i386-AS-20080522

Last edited by mhashkhan on Mon Mar 07, 2011 11:49 pm; edited 2 times in total
Back to top
View user's profile Send private message
exerk
PostPosted: Mon Mar 07, 2011 12:09 am    Post subject: Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 6339
Please do not post the full FDC, just the header is sufficient. Editing your post to remove the additional stuff would be appreciated.

What messages (if any) were printed in the queue manager error log? What is the history of this queue manager, i.e. has it been working fine in the past but now is not? Were changes made to the environment, e.g. maintenance applied to the server etc.? And 6.0.2.6 is way back-level by the way.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.
Back to top
View user's profile Send private message
mhashkhan
PostPosted: Mon Mar 07, 2011 12:15 am    Post subject: Reply with quote

Apprentice

Joined: 01 Dec 2008
Posts: 28
internal error '893' i see in the log.
the box has multiple queue managers, previously running fine.
Now all queue managers are down, and none of them we are able to start.

Environment team claims no changes has been done in the box.
I have done following to resolve the issue,
1- Space check: There is no space problem in the box
2- Semaphores: We have cleared all the semaphores
3- Box has been restarted.

I know there is some thing wrong with the box, it just that i dont have any idea what to look for;

Looking for some tips/tricks
Back to top
View user's profile Send private message
exerk
PostPosted: Mon Mar 07, 2011 12:25 am    Post subject: Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 6339
Was the server restarted before the environment team did 'no changes'? Find a working box and compare file ownerships/permissions etc.

And have another go at editing your original post
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Mon Mar 07, 2011 4:15 am    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
The probe ID gave me one hit hinting at an incorrect DCOM configuration... but it looks like you're on Linux and not winx... Open a PMR with IBM....
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
bruce2359
PostPosted: Mon Mar 07, 2011 6:42 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
Actually, this is one of those situations where more than just the FDC header can be of some use:

-} zslCheckIfRunning rc=lrcE_Q_MGR_NOT_AVAILABLE

Do you get a good response from dspmq command?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
mhashkhan
PostPosted: Mon Mar 07, 2011 6:18 pm    Post subject: Reply with quote

Apprentice

Joined: 01 Dec 2008
Posts: 28
found one issue where by the permission of directory /opt/mqm/bin was being changed and -r-sr-sr-x was changed to -r-xr-xr-x . I hope after fixing this will resolve the problem
Back to top
View user's profile Send private message
exerk
PostPosted: Mon Mar 07, 2011 11:36 pm    Post subject: Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 6339
mhashkhan wrote:
...found one issue where by the permission of directory /opt/mqm/bin was being changed and -r-sr-sr-x was changed to -r-xr-xr-x . I hope after fixing this will resolve the problem...


exerk wrote:
...Find a working box and compare file ownerships/permissions etc...


This one's come up before You may find it quicker and easier to reinstall...
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.
Back to top
View user's profile Send private message
mhashkhan
PostPosted: Mon Mar 07, 2011 11:40 pm    Post subject: Reply with quote

Apprentice

Joined: 01 Dec 2008
Posts: 28
reinstall means all object lost ... well .. the permission was the actual problem. then we removed all the semaphores, restarted the box and walllaaaa ... all queue managers are now able to start...

close case...
Back to top
View user's profile Send private message
exerk
PostPosted: Mon Mar 07, 2011 11:44 pm    Post subject: Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 6339
Please amend your post title to read SOLVED - Queue Manager start problem - FDC Genereted, it makes it easier for people when they search and to see that it may contain a fix for their problem.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » General IBM MQ Support » Solved - Queue Manager start problem - FDC Genereted
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
  
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.