ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » IBM MQ Installation/Configuration Support » SIGSEGV FDC External Application

Post new topic  Reply to topic
 SIGSEGV FDC External Application « View previous topic :: View next topic » 
Author Message
Shalini
PostPosted: Tue Apr 19, 2005 9:15 am    Post subject: SIGSEGV FDC External Application Reply with quote

Master

Joined: 30 Apr 2002
Posts: 224
Location: India

Hi All,

AIX MQ5.3 CSD throwing following FDC

Looking @ the FDC I belive the FDC is generated due to UID ("LINK" under mqm group) and due to the exe "kinc.exe".

But really do not know why the exe is failing (due to this we are loosing the messages Its a trigger process).

Is there issue with kinc.exe ???

Can any body suggest looking @ FDC whats the issue with kinc or with MQ (I think its not due to MQ).

+-----------------------------------------------------------------------------+
| |
| WebSphere MQ First Failure Symptom Report |
| ========================================= |
| |
| Date/Time :- Tuesday April 05 19:15:44 SAUST 2005 |
| Host Name :- linkprd (AIX 5.1) |
| PIDS :- 5724B4101 |
| LVLS :- 530.9 CSD09 |
| Product Long Name :- WebSphere MQ for AIX |
| Vendor :- IBM |
| Probe Id :- XC130003 |
| Application Name :- MQM |
| Component :- xehExceptionHandler |
| Build Date :- Dec 13 2004 |
| CMVC level :- p530-09-L041213 |
| Build Type :- IKAP - (Production) |
| UserID :- 00000102 (link) |
| Program Name :- kinc.exe |
| Process :- 00053542 |
| QueueManager :- LINK |
| Major Errorcode :- STOP |
| Minor Errorcode :- OK |
| Probe Type :- HALT6109 |
| Probe Severity :- 1 |
| Probe Description :- AMQ6109: An internal WebSphere MQ error has occurred. |
| FDCSequenceNumber :- 0 |
| Arith1 :- 11 b |
| Comment1 :- SIGSEGV |
| |
| |
+-----------------------------------------------------------------------------+

MQM Function Stack
xcsGetpwnam
xcsFFST

MQM Trace History
} zstVerifyPCD rc=OK
{ zstCheckODAddressability
{ xcsCheckPointer
} xcsCheckPointer rc=OK
} zstCheckODAddressability rc=OK
{ xcsCheckPointer
} xcsCheckPointer rc=OK
{ ziiMQOPEN
{ ziiCreateIPCCMessage
{ zcpCreateMessage
} zcpCreateMessage rc=OK
} ziiCreateIPCCMessage rc=OK
{ ziiSendReceiveAgent
{ zcpSendOnPipe
{ xcsResetEventSem
} xcsResetEventSem rc=OK
{ xcsPostEventSem
-{ xlsLockEvent
-} xlsLockEvent rc=OK
-{ xlsUnlockEvent
-} xlsUnlockEvent rc=OK
} xcsPostEventSem rc=OK
} zcpSendOnPipe rc=OK
{ zcpReceiveOnPipe
{ xcsWaitEventSem
-{ xlsLockEvent
-} xlsLockEvent rc=OK
-{ xlsUnlockEvent
-} xlsUnlockEvent rc=OK
} xcsWaitEventSem rc=OK
} zcpReceiveOnPipe rc=OK
} ziiSendReceiveAgent rc=OK
{ zcpDeleteMessage
} zcpDeleteMessage rc=OK
} ziiMQOPEN rc=OK
} zstMQOPEN rc=OK
} MQOPEN rc=OK
{ MQGET
{ zstMQGET
{ zstVerifyPCD
} zstVerifyPCD rc=OK
{ xcsCheckPointer
} xcsCheckPointer rc=OK
{ xcsCheckPointer
} xcsCheckPointer rc=OK
{ xcsCheckPointer
} xcsCheckPointer rc=OK
{ xcsCheckPointer
} xcsCheckPointer rc=OK
{ ziiMQGET
{ ziiCreateIPCCMessage
{ zcpCreateMessage
} zcpCreateMessage rc=OK
} ziiCreateIPCCMessage rc=OK
{ ziiSendReceiveAgent
{ zcpSendOnPipe
{ xcsResetEventSem
} xcsResetEventSem rc=OK
{ xcsPostEventSem
-{ xlsLockEvent
-} xlsLockEvent rc=OK
-{ xlsUnlockEvent
-} xlsUnlockEvent rc=OK
} xcsPostEventSem rc=OK
} zcpSendOnPipe rc=OK
{ zcpReceiveOnPipe
{ xcsWaitEventSem
-{ xlsLockEvent
-} xlsLockEvent rc=OK
-{ xlsUnlockEvent
-} xlsUnlockEvent rc=OK
} xcsWaitEventSem rc=OK
} zcpReceiveOnPipe rc=OK
} ziiSendReceiveAgent rc=arcE_NO_MSG_AVAILABLE
{ zcpDeleteMessage
} zcpDeleteMessage rc=OK
} ziiMQGET rc=OK
} zstMQGET rc=arcE_NO_MSG_AVAILABLE
} MQGET rc=arcE_NO_MSG_AVAILABLE
{ MQCLOSE
{ zstMQCLOSE
{ zstVerifyPCD
} zstVerifyPCD rc=OK
{ xcsCheckPointer
} xcsCheckPointer rc=OK
{ ziiMQCLOSE
{ ziiCreateIPCCMessage
{ zcpCreateMessage
} zcpCreateMessage rc=OK
} ziiCreateIPCCMessage rc=OK
{ ziiSendReceiveAgent
{ zcpSendOnPipe
{ xcsResetEventSem
} xcsResetEventSem rc=OK
{ xcsPostEventSem
-{ xlsLockEvent
-} xlsLockEvent rc=OK
-{ xlsUnlockEvent
-} xlsUnlockEvent rc=OK
} xcsPostEventSem rc=OK
} zcpSendOnPipe rc=OK
{ zcpReceiveOnPipe
{ xcsWaitEventSem
-{ xlsLockEvent
-} xlsLockEvent rc=OK
-{ xlsUnlockEvent
-} xlsUnlockEvent rc=OK
} xcsWaitEventSem rc=OK
} zcpReceiveOnPipe rc=OK
} ziiSendReceiveAgent rc=OK
{ zcpDeleteMessage
} zcpDeleteMessage rc=OK
} ziiMQCLOSE rc=OK
} zstMQCLOSE rc=OK
} MQCLOSE rc=OK
{ MQDISC
{ zstMQDISC
{ xcsCheckPointer
} xcsCheckPointer rc=OK
{ zstVerifyPCD
} zstVerifyPCD rc=OK
{ ziiMQDISC
{ ziiCreateIPCCMessage
{ zcpCreateMessage
} zcpCreateMessage rc=OK
} ziiCreateIPCCMessage rc=OK
{ ziiSendReceiveAgent
{ zcpSendOnPipe
{ xcsResetEventSem
} xcsResetEventSem rc=OK
{ xcsPostEventSem
-{ xlsLockEvent
-} xlsLockEvent rc=OK
-{ xlsUnlockEvent
-} xlsUnlockEvent rc=OK
} xcsPostEventSem rc=OK
} zcpSendOnPipe rc=OK
{ zcpReceiveOnPipe
{ xcsWaitEventSem
-{ xlsLockEvent
-} xlsLockEvent rc=OK
-{ xlsUnlockEvent
-} xlsUnlockEvent rc=OK
} xcsWaitEventSem rc=OK
} zcpReceiveOnPipe rc=OK
} ziiSendReceiveAgent rc=OK
{ zcpDeleteMessage
} zcpDeleteMessage rc=OK
} ziiMQDISC rc=OK
{ ziiClearUpAgent
{ ziiStopAllPlugServices
{ xcsDisconnectSharedMemSet
{ xcsFreeMem
} xcsFreeMem rc=OK
{ xcsFreeMem
} xcsFreeMem rc=OK
{ xstDisconnectExtent
} xstDisconnectExtent rc=OK
} xcsDisconnectSharedMemSet rc=OK
} ziiStopAllPlugServices rc=OK
{ xlsSetSuspendEvent
} xlsSetSuspendEvent rc=OK
{ zcpDetachPipe
{ xcsUnregisterDestructor
} xcsUnregisterDestructor rc=OK
} zcpDetachPipe rc=OK
{ ziiFinishedWithAgent
} ziiFinishedWithAgent rc=OK
{ xcsUnregisterDestructor
} xcsUnregisterDestructor rc=OK
{ xcsTerminate
{ xcsDisconnectSharedSubpool
{ xlsThreadTermination
-{ xcsFreeQuickCell
--{ xllSpinLockRequest
--} xllSpinLockRequest rc=OK
--{ xllSpinLockRelease
--} xllSpinLockRelease rc=OK
-} xcsFreeQuickCell rc=OK
} xlsThreadTermination rc=OK
{ xcsDettachSharedSubpool
-{ xcsGetSetConnectCount
--{ xstGetExtentConnectCount
--} xstGetExtentConnectCount rc=OK
-} xcsGetSetConnectCount rc=OK
-{ xcsFreeQuickCell
--{ xllSpinLockRequest
--} xllSpinLockRequest rc=OK
--{ xllSpinLockRelease
--} xllSpinLockRelease rc=OK
-} xcsFreeQuickCell rc=OK
-{ xcsDisconnectSharedMemSet
--{ xcsFreeMem
--} xcsFreeMem rc=OK
--{ xcsFreeMem
--} xcsFreeMem rc=OK
-} xcsDisconnectSharedMemSet rc=OK
-{ xcsDisconnectSharedMemSet
--{ xcsFreeMem
--} xcsFreeMem rc=OK
--{ xcsFreeMem
--} xcsFreeMem rc=OK
-} xcsDisconnectSharedMemSet rc=OK
-{ xstDisconnectExtent
-} xstDisconnectExtent rc=OK
-{ xstDisconnectExtent
-} xstDisconnectExtent rc=OK
-{ xcsFreeMem
-} xcsFreeMem rc=OK
-{ xcsFreeMem
-} xcsFreeMem rc=OK
} xcsDettachSharedSubpool rc=OK
} xcsDisconnectSharedSubpool rc=OK
{ TermPrivateServices
{ xxxTerminate
-{ xcsFreeMem
-} xcsFreeMem rc=OK
} xxxTerminate rc=OK
} TermPrivateServices rc=OK
} xcsTerminate rc=OK
} ziiClearUpAgent rc=OK
{ zstDeletePCD
{ xcsFreeMem
} xcsFreeMem rc=OK
} zstDeletePCD rc=OK
} zstMQDISC rc=OK
} MQDISC rc=OK
{ MQCONN
{ zstMQCONN
{ xcsInitialize
} xcsInitialize rc=OK
{ zstMQConnect
{ zutBlankPad
} zutBlankPad rc=OK
{ zutIsItBlank
} zutIsItBlank rc=OK
{ zutCheckQMName
{ zutCheckValidName
} zutCheckValidName rc=OK
} zutCheckQMName rc=OK
{ zutCvtMQName2Str
} zutCvtMQName2Str rc=OK
{ zstGetPCDbyTID
} zstGetPCDbyTID rc=OK
{ zstInitCS
{ xcsInitialize
{ InitPrivateServices
} InitPrivateServices rc=OK
{ xcsGetpwnam
-{ xcsFFST

Back to top
View user's profile Send private message Send e-mail
Shalini
PostPosted: Thu Apr 21, 2005 1:22 am    Post subject: Reply with quote

Master

Joined: 30 Apr 2002
Posts: 224
Location: India

Hi,

Can any one please comment/suggest ...


Back to top
View user's profile Send private message Send e-mail
clindsey
PostPosted: Thu Apr 21, 2005 5:28 am    Post subject: Reply with quote

Knight

Joined: 12 Jul 2002
Posts: 586
Location: Dallas, Tx

Shalini,

This might help you with where to look. Take a look at the code for kinc.
It appears that is disconnect from the queue manager and then connects again. The FDC is generated during the MQCONN call on the reconnect. See if you can spot any problems in this area. First of all, I would question why you are doing a MQDISC and MQCONN in a triggered program.

Hope this helps,
Charlie
Back to top
View user's profile Send private message
clindsey
PostPosted: Thu Apr 21, 2005 5:50 am    Post subject: Reply with quote

Knight

Joined: 12 Jul 2002
Posts: 586
Location: Dallas, Tx

http://www-1.ibm.com/support/docview.wss?uid=swg1IY52676

A similar problem has been reported on Linux. You should open a pmr and report that you are having this problem on AIX. It looks like the problem is related to gathering information for user Link as you suspected.
It is possible that resources for this user, like file descriptors, is limited.

Charlie
Back to top
View user's profile Send private message
Shalini
PostPosted: Fri Apr 22, 2005 3:12 am    Post subject: Reply with quote

Master

Joined: 30 Apr 2002
Posts: 224
Location: India

Hi Charlie,

1)
Quote:
First of all, I would question why you are doing a MQDISC and MQCONN in a triggered program.
.

We are doing disconnect and connect again in a trigger type First Which takes the depth of the queue in one shot.

If I donot disconnect The client will develop the open orphan connection which our QMgr will fail with "Max channel Connectivity problem".

TCPKeepAlive will not help us as it will release the connection after 2 hrs (approx) but by that time QMGr may fail due to orphan connection


2)
Quote:
It is possible that resources for this user, like file descriptors, is limited.


As U suggested abt the file descriptors

"A per-process unique, non-negative integer used to identify an open file for the purpose of file access. The value of a file descriptor is from zero to {OPEN_MAX}. A process can have no more than {OPEN_MAX} file descriptors open simultaneously. File descriptors may also be used to implement message catalog descriptors and directory streams; "

How can I find the (or any command to know) the file descriptor limit

or

Is the file descriptor limit to be increased in the system ?????

Back to top
View user's profile Send private message Send e-mail
clindsey
PostPosted: Fri Apr 22, 2005 5:06 am    Post subject: Reply with quote

Knight

Joined: 12 Jul 2002
Posts: 586
Location: Dallas, Tx

Definitely, you should do the MQDISC before your application exits. In the stack trace of the FDC, there is a MQDISC followed by an MQCONN. Since your app is a client app, this could be for connection recovery logic, I suppose. At any rate, the second connect should not cause an FDC.

To get the file descriptors, log in as user link and run 'ulimit -a' or 'ulimit -n' to see just file descriptors. Change it with 'ulimit -n unlimited'. You would have to set this in the .profile to make it permanent.

Are you using runmqtrmc as the trigger monitor, or one of your own? Can you run the code standalone (not triggered) without problems?

File desc is only a possible problem that could happen in getting user info. Try it to see if it helps. I think you may need to get help from IBM support on this one. I appended the reference to the APAR because it was reported on linux. The apar fix may be only for linux and one is needed for aix. IBM support could answer that for you.


Last edited by clindsey on Fri Apr 22, 2005 12:04 pm; edited 1 time in total
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Fri Apr 22, 2005 11:11 am    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7722

Shalini wrote:
We are doing disconnect and connect again in a trigger type First Which takes the depth of the queue in one shot.

Why?

You get triggered, you connect, read the queue until it is empty, maybe do one more MQGET with wait to see if any new messages arrive in the next few seconds and disconnect.

You should need to inquire on the depth of the queue.


Regardless, even if you do this extra work for nothing, you should not be gettings FDCs like Charlie mentioned.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
JT
PostPosted: Fri Apr 22, 2005 11:16 am    Post subject: Reply with quote

Padawan

Joined: 27 Mar 2003
Posts: 1564
Location: Hartford, CT.

PeterPotkay thought he wrote:
You should NOT need to inquire on the depth of the queue.

I believe this was what Peter intended.
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Fri Apr 22, 2005 11:49 am    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7722

Yes we have no contact admin today.

Thanks JT!

You should NOT have to inquire.....
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
jefflowrey
PostPosted: Fri Apr 22, 2005 12:55 pm    Post subject: Reply with quote

Grand Poobah

Joined: 16 Oct 2002
Posts: 19981

PeterPotkay wrote:
You should NOT inquire.....


Fixed...

_________________
I am *not* the model of the modern major general.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » IBM MQ Installation/Configuration Support » SIGSEGV FDC External Application
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.