| Author |
Message
|
| m00300 |
 Posted: Mon Jul 26, 2004 2:35 am Post subject: BIP2228E: Severe error in WBIMB CSD03 under AIX |
 |
|
Apprentice
Joined: 01 May 2002
Posts: 31
|
We've recently upgraded our AIX server (UAT only at present..) to WBIMB CSD03 and in general all seems fine, migrated code works as expected.
However occasionally a message is backed out to a backout queue and no message is written to our error table (on an application database - usually all errors write something to this table).
Re-running the messages through the flows always shows up no problems and they process fine so it looks like an environment issue.
The syslog contains entries like this :
Jul 22 09:13:55 MQSIUAT1 MQSIv500[54730]: (MQSBK01A.93d459b3-fd00-0000-0080-cab9a07f464c)[5141]BIP2228E: Severe error: /build/S500_P/src/CommonServices/Unix/ImbAbend.cpp 458 signal received Abend file: /var/mqsi/errors/MQSBK01A.93d459b3-fd00-0000-0080-cab9a07f464c.54730.5141.abend action: abort
Jul 22 09:16:21 MQSIUAT1 MQSIv500[48200]: (MQSBK01A)[4370]BIP2060W: The broker has detected that the Execution Group EX04, process ID 54730, has shutdown. : MQSBK01A.agent: /build/S500_P/src/AdminAgent/ImbAdminAgent.cpp: 3869: ImbAdminAgent::startAndMonitorADataFlowEngine: :
Jul 22 09:16:24 MQSIUAT1 MQSIv500[43396]: (MQSBK01A.EX04)[1]BIP2201I: Execution Group started: process '43396'; thread '1'; additional information: brokerName 'MQSBK01A'; executionGroupUUID '93d459b3-fd00-0000-0080-cab9a07f464c'; executionGroupLabel 'EX04'; defaultExecutionGroup 'false'; queueManagerName 'MQSBK01A'; trusted 'false'; dataSourceName 'MQSBK01A'; userId 'mqsiuat'; migrationNeeded 'false'; brokerUUID '4ce946a3-fd00-0000-0080-f7af2099c34b'; filePath '/usr/opt/mqsi'; workPath '/var/mqsi'. : MQSBK01A.93d459b3-fd00-0000-0080-cab9a07f464c: /build/S500_P/src/DataFlowEngine/ImbMain.cpp: 235: main: :
Jul 22 09:16:46 MQSIUAT1 MQSIv500[43396]: (MQSBK01A.EX04)[5141]BIP2648E: Message backed out to a queue; node 'DM_APP_CategoryA_Thread3.MQInput1'. : MQSBK01A.93d459b3-fd00-0000-0080-cab9a07f464c: /build/S500_P/src/DataFlowEngine/ImbMqInputNode.cpp: 3096: ImbMqInputNode::eligibleForBackout: ComIbmMQInputNode: DM_APP_CategoryA_Thread3#FCMComposite_1_2
Jul 22 09:16:54 MQSIUAT1 MQSIv500[43396]: (MQSBK01A.EX04)[5141]BIP2232E: Error detected whilst handling a previous error in node 'DM_APP_CategoryA_Thread3.DM_APP_CategoryA_Pt2.DM_XXX_HandleException1.DM_XXX_HandleException_Sub1.Throw'. : MQSBK01A.93d459b3-fd00-0000-0080-cab9a07f464c: /build/S500_P/src/DataFlowEngine/ImbDataFlowNode.cpp: 387: ImbDataFlowNode::logExceptionList: ComIbmThrowNode: DM_APP_CategoryA_Thread3#FCMComposite_1_1.DM_APP_CategoryA_Pt2#FCMComposite_1_18.DM_XXX_HandleException#FCMComposite_1_1.DM_XXX_HandleException_Sub#FCMComposite_1_3
Jul 22 09:16:54 MQSIUAT1 MQSIv500[43396]: (MQSBK01A.EX04)[5141]BIP3019I: Exception thrown by throw node 'DM_APP_CategoryA_Thread3.DM_APP_CategoryA_Pt2.DM_TRA_TranslateToUdf1.DM_TRA_URN_subflow1.DM_XXX_HandleException1.DM_XXX_HandleException_Sub1.Throw_3019'; text is ''. : MQSBK01A.93d459b3-fd00-0000-0080-cab9a07f464c: /build/S500_P/src/DataFlowEngine/BasicNodes/ImbThrowNode.cpp: 259: ImbThrowNode::evaluate: ComIbmThrowNode: DM_APP_CategoryA_Thread3#FCMComposite_1_1.DM_APP_CategoryA_Pt2#FCMComposite_1_13.DM_TRA_TranslateToUdf#FCMComposite_1_2.DM_TRA_URN_subflow#FCMComposite_1_1.DM_XXX_HandleException#FCMComposite_1_1.DM_XXX_HandleException_Sub#FCMComposite_1_4
Jul 22 09:16:54 MQSIUAT1 MQSIv500[43396]: (MQSBK01A.EX04)[5141]BIP2628E: Exception condition detected on input node 'DM_APP_CategoryA_Thread3.MQInput1'. : MQSBK01A.93d459b3-fd00-0000-0080-cab9a07f464c: /build/S500_P/src/DataFlowEngine/ImbMqInputNode.cpp: 1781: ImbMqInputNode::readQueue: ComIbmMQInputNode: DM_APP_CategoryA_Thread3#FCMComposite_1_2
Jul 22 09:16:54 MQSIUAT1 MQSIv500[43396]: (MQSBK01A.EX04)[5141]BIP3001I: Exception thrown by throw node 'DM_APP_CategoryA_Thread3.DM_APP_CategoryA_Pt2.DM_XXX_HandleException1.DM_XXX_HandleException_Sub1.Throw'; text is 'To back out queue'. : MQSBK01A.93d459b3-fd00-0000-0080-cab9a07f464c: /build/S500_P/src/DataFlowEngine/BasicNodes/ImbThrowNode.cpp: 259: ImbThrowNode::evaluate: ComIbmThrowNode: DM_APP_CategoryA_Thread3#FCMComposite_1_1.DM_APP_CategoryA_Pt2#FCMComposite_1_18.DM_XXX_HandleException#FCMComposite_1_1.DM_XXX_HandleException_Sub#FCMComposite_1_3
Jul 22 09:16:55 MQSIUAT1 MQSIv500[43396]: (MQSBK01A.EX04)[5141]BIP2648E: Message backed out to a queue; node 'DM_APP_CategoryA_Thread3.MQInput1'. : MQSBK01A.93d459b3-fd00-0000-0080-cab9a07f464c: /build/S500_P/src/DataFlowEngine/ImbMqInputNode.cpp: 3096: ImbMqInputNode::eligibleForBackout: ComIbmMQInputNode: DM_APP_CategoryA_Thread3#FCMComposite_1_2
This problem hasnt occured in previous test AIX environments - nor in any of our Windows based ones.
Anyone any idea of what could be causing this ?
Thanks.
Kev. |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Mon Jul 26, 2004 6:06 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| Quote: |
| Exception thrown by throw node 'DM_APP_CategoryA_Thread3.DM_APP_CategoryA_Pt2.DM_XXX_HandleException1.DM_XXX_HandleException_Sub1.Throw' |
That means that your message flow is throwing an exception, by using the throw node named.
As to why that's happening, you'd have to find the throw node in your flow to determine that.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| m00300 |
| Posted: Mon Jul 26, 2004 7:01 am Post subject: |
|
|
Apprentice
Joined: 01 May 2002
Posts: 31
|
The throw node is used to push messages to the backout queue after the details of any error - system or user have been captured on the database.
The throw node throws with message number 3001 but this doesnt usually cause an entry in the syslog, or the execution group to die.
Its part of our standard exception handling procedure used by all applications. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Jul 26, 2004 8:33 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
I'd say that the two events are unrelated, or at least that the throw is not causative of the abend. They're 20 seconds apart - at best the throw is being caused by some something like a database handle having gone out of scope when the execution group abended.
What's happening in your log BEFORE the abend?
What does it say in the abend file "/var/mqsi/errors/MQSBK01A.93d459b3-fd00-0000-0080-cab9a07f464c.54730.5141.abend"?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| m00300 |
| Posted: Wed Jul 28, 2004 1:51 am Post subject: |
|
|
Apprentice
Joined: 01 May 2002
Posts: 31
|
The abend file contains :
abend record for pid 54730 tid 5141 time in seconds since 01/01/1970: 1090484035
File: /build/S500_P/src/CommonServices/Unix/ImbAbend.cpp
Line: 458
Function: signal received
---- Inserts ----
11
@(#) 1.33.2.2 CommonServices/Unix/ImbAbend.cpp, CommonServices, S500, S500-L31218 03/11/25 15:51:20 [12/18/03 17:10:51]
1044085984
-----------------
----------------------------- Stack dump for current thread ( 5141)
(0xd8ccacbc+0x0000077c) bindParameters__17SqlExternalDbStmtCFP16ImbOdbcStatementRCQ2_3std6vectorXT8ImbValueTQ2_3std9allocatorXT8ImbValue__ [/var/mqsi/lib/libImbRdl.a(libImbRdl.a.so)]
(0xd8cbc900+0x00000f58) executeStmt__17SqlExternalDbStmtCFR14SqlEvalEnvironRC13ImbStringBaseXTwTQ2_3std11char_traitsXTw_TcSP37_Q2_3std6_PtritXTP17SqlExpressionNodeTlTPCP17SqlExpressionNodeTRCP17SqlExpressionNodeTPP17SqlExpressionNodeTRP17SqlExpressionNode_T3RC18SqlGeneralLocationi [/var/mqsi/lib/libImbRdl.a(libImbRdl.a.so)]
(0xd8f9ea50+0x00000520) execute__20SqlDatabaseStatementCFR18SqlStatementResult [/var/mqsi/lib/libImbRdl.a(libImbRdl.a.so)]
(0xd8bbeee0+0x000000d8) execute__17SqlStatementGroupCFR18SqlStatementResult [/var/mqsi/lib/libImbRdl.a(libImbRdl.a.so)]
(0xd8efe5b4+0x000001a8) execute__15SqlCompoundStmtCFR18SqlStatementResult [/var/mqsi/lib/libImbRdl.a(libImbRdl.a.so)]
(0xd8e203d0+0x00000138) execute__19SqlCallableFunctionCFR18SqlStatementResult [/var/mqsi/lib/libImbRdl.a(libImbRdl.a.so)]
(0xd9046378+0x000000ec) execute__9SqlModuleCFR18SqlStatementResult [/var/mqsi/lib/libImbRdl.a(libImbRdl.a.so)]
(0xd8f1855c+0x0000002c) execute__9SqlSchemaCFR18SqlStatementResult [/var/mqsi/lib/libImbRdl.a(libImbRdl.a.so)]
(0xd9073320+0x00000468) evaluate__20SqlDatabaseInterfaceFRC18ImbMessageAssembly [/var/mqsi/lib/libImbRdl.a(libImbRdl.a.so)]
(0xd9aefc54+0x00000100) evaluate__15ImbDatabaseNodeFRC18ImbMessageAssemblyPC19ImbDataFlowTerminal [/usr/opt/mqsi/lil/imbdfsql.lil]
(0xd7cb922c+0x000001d8) evaluate__19ImbDataFlowTerminalFRC18ImbMessageAssembly [/var/mqsi/lib/libDataFlowDLL.a(libDataFlowDLL.a.so)]
(0xd7e7892c+0x0000034c) propagate__19ImbDataFlowTerminalFRC18ImbMessageAssembly [/var/mqsi/lib/libDataFlowDLL.a(libDataFlowDLL.a.so)]
(0xd6ac05fc+0x00000108) evaluate__15ImbTryCatchNodeFRC18ImbMessageAssemblyPC19ImbDataFlowTerminal [/usr/opt/mqsi/lil/imbdfbas.lil]
(0xd7cb922c+0x000001d8) evaluate__19ImbDataFlowTerminalFRC18ImbMessageAssembly [/var/mqsi/lib/libDataFlowDLL.a(libDataFlowDLL.a.so)]
(0xd7e7892c+0x0000034c) propagate__19ImbDataFlowTerminalFRC18ImbMessageAssembly [/var/mqsi/lib/libDataFlowDLL.a(libDataFlowDLL.a.so)]
(0xd6ac65d8+0x00001318) evaluate__23ImbResetContentDescNodeFRC18ImbMessageAssemblyPC19ImbDataFlowTerminal [/usr/opt/mqsi/lil/imbdfbas.lil]
(0xd7cb922c+0x000001d8) evaluate__19ImbDataFlowTerminalFRC18ImbMessageAssembly [/var/mqsi/lib/libDataFlowDLL.a(libDataFlowDLL.a.so)]
(0xd7e7892c+0x0000034c) propagate__19ImbDataFlowTerminalFRC18ImbMessageAssembly [/var/mqsi/lib/libDataFlowDLL.a(libDataFlowDLL.a.so)]
(0xd6b70620+0x00003aec) readQueue__14ImbMqInputNodeFP11ImbOsThread [/var/mqsi/lib/libMQLibrary.a(libMQLibrary.a.so)]
(0xd6b7a314+0x00000048) run__Q2_14ImbMqInputNode10ParametersFP11ImbOsThread [/var/mqsi/lib/libMQLibrary.a(libMQLibrary.a.so)]
(0xd3527410+0x00000070) run__27ImbThreadPoolThreadFunctionFP11ImbOsThread [/var/mqsi/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd351a9b0+0x00000054) threadRun__11ImbOsThreadFv [/var/mqsi/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd351a554+0x00000064) threadBootStrap__11ImbOsThreadFPv [/var/mqsi/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd004b490+0x000000ec) _pthread_body [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0x00000000) <invalid code address>
----------------------------------------------------------------------
------------------------------------------ Stack dumps for all threads
=== Thread 1
(0xd0055d8c+0x000000a8) _event_sleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd00563e0+0x00000060) _event_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0061c54+0x000002e0) _cond_wait_local [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062478+0x00000054) _cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062c08+0x000002dc) pthread_cond_timedwait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd351999c+0x0000028c) wait__12ImbSemaphoreFl [/var/mqsi/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd36477cc+0x00000018) waitForChildEvent__15ImbOsThreadBaseFv [/var/mqsi/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd2d58128+0x00002e10) start__7ImbMainFR18ImbStartParameters [/var/mqsi/lib/libbipmain.a]
(0x10001f78+0x00000b4c) main [DataFlowEngine]
(0x10000128+0x0000008c) __start [DataFlowEngine]
=== Thread 258
(0xd0055d8c+0x000000a8) _event_sleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd00563e0+0x00000060) _event_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0061c54+0x00000414) _cond_wait_local [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062478+0x00000054) _cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062f94+0x000001dc) pthread_cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd365c138+0x000000c8) timerThreadFunction__20ImbGlobalMutexHelperFPv [/var/mqsi/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd365c05c+0x00000048) startTimerThread [/var/mqsi/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd004b490+0x000000ec) _pthread_body [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0x00000000) <invalid code address>
=== Thread 515
(0xd005ad34+0x00000010) _p_nsleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd01e41e0+0x000000a8) nsleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd01e3e78+0x0000005c) usleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd352af78+0x00000014) imbSleep__Fi [/var/mqsi/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd7ec3250+0x00000040) runState__30ImbStatsCollectorState_WaitingFP17ImbStatsCollector [/var/mqsi/lib/libDataFlowDLL.a(libDataFlowDLL.a.so)]
(0xd7ea2064+0x000001cc) runCollectorCycle__17ImbStatsCollectorFP11ImbOsThread [/var/mqsi/lib/libDataFlowDLL.a(libDataFlowDLL.a.so)]
(0xd7ea1f68+0x00000048) run__Q2_17ImbStatsCollector10ParametersFP11ImbOsThread [/var/mqsi/lib/libDataFlowDLL.a(libDataFlowDLL.a.so)]
(0xd3527410+0x00000070) run__27ImbThreadPoolThreadFunctionFP11ImbOsThread [/var/mqsi/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd351a9b0+0x00000054) threadRun__11ImbOsThreadFv [/var/mqsi/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd351a554+0x00000064) threadBootStrap__11ImbOsThreadFPv [/var/mqsi/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd004b490+0x000000ec) _pthread_body [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0x00000000) <invalid code address>
=== Thread 772
(0xd005ad34+0x00000010) _p_nsleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd01e41e0+0x000000a8) nsleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd01eecac+0x00000034) sleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd2fa8fb8+0x0000006c) sysSignalWait [/usr/java131/jre/bin/libhpi.a]
(0xd3b494b4+0x0000006c) signalDispatcherThread [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a52c48+0x00000148) xmExecuteThread [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a38d58+0x00000034) threadStart [/usr/java131/jre/bin/classic/libjvm.a]
(0xd2f9ed58+0x00000058) _start [/usr/java131/jre/bin/libhpi.a]
(0xd004b490+0x000000ec) _pthread_body [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0x00000000) <invalid code address>
=== Thread 1029
(0xd0055d8c+0x000000a8) _event_sleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd00563e0+0x00000060) _event_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0061c54+0x00000414) _cond_wait_local [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062478+0x00000054) _cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062f94+0x000001dc) pthread_cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd2fa495c+0x00000050) condvarWait [/usr/java131/jre/bin/libhpi.a]
(0xd2fa35a8+0x000000c4) sysMonitorWait [/usr/java131/jre/bin/libhpi.a]
(0xd3a9515c+0x000001b4) lkMonitorWait [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a07c0c+0x00000140) JVM_MonitorWait [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a34c98+0x0000019c) sysInvokeNative [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a0f088+0x000002a0) mmisInvokeJniMethodHelper [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a35360+0xfffffec4) changeCodes (name probably invalid) [/usr/java131/jre/bin/classic/libjvm.a]
=== Thread 1286
(0xd0055d8c+0x000000a8) _event_sleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd00563e0+0x00000060) _event_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0061c54+0x00000414) _cond_wait_local [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062478+0x00000054) _cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062f94+0x000001dc) pthread_cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd2fa495c+0x00000050) condvarWait [/usr/java131/jre/bin/libhpi.a]
(0xd2fa35a8+0x000000c4) sysMonitorWait [/usr/java131/jre/bin/libhpi.a]
(0xd3a9515c+0x000001b4) lkMonitorWait [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a07c0c+0x00000140) JVM_MonitorWait [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a34c98+0x0000019c) sysInvokeNative [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a0f088+0x000002a0) mmisInvokeJniMethodHelper [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a35360+0xfffffec4) changeCodes (name probably invalid) [/usr/java131/jre/bin/classic/libjvm.a]
=== Thread 1543
(0xd0055d8c+0x000000a8) _event_sleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd00563e0+0x00000060) _event_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0061c54+0x00000414) _cond_wait_local [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062478+0x00000054) _cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062f94+0x000001dc) pthread_cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd2fa495c+0x00000050) condvarWait [/usr/java131/jre/bin/libhpi.a]
(0xd2fa35a8+0x000000c4) sysMonitorWait [/usr/java131/jre/bin/libhpi.a]
(0xd3b1a098+0x0000018c) gcHelper [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a52c48+0x00000148) xmExecuteThread [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a38d58+0x00000034) threadStart [/usr/java131/jre/bin/classic/libjvm.a]
(0xd2f9ed58+0x00000058) _start [/usr/java131/jre/bin/libhpi.a]
(0xd004b490+0x000000ec) _pthread_body [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0x00000000) <invalid code address>
=== Thread 1800
(0xd005ad34+0x00000010) _p_nsleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd01e41e0+0x000000a8) nsleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd01eecac+0x00000034) sleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd167956c+0x000000c4) PipeServer [/usr/lib/librabnd.so]
(0xd004b490+0x000000ec) _pthread_body [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0x00000000) <invalid code address>
=== Thread 2057
(0xd0055d8c+0x000000a8) _event_sleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd00563e0+0x00000060) _event_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0061c54+0x000002e0) _cond_wait_local [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062478+0x00000054) _cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062c08+0x000002dc) pthread_cond_timedwait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd6eda874+0x00000000) <no name available> [/usr/mqm/lib/libmqmcs_r.a(shr.o)]
(0xd70de0e4+0x00000000) <no name available> [/usr/mqm/lib/libmqz_r.a(libmqz_r.o)]
(0xd6e73cb0+0x00000000) <no name available> [/usr/mqm/lib/libmqmcs_r.a(shr.o)]
(0xd004b490+0x000000ec) _pthread_body [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0x00000000) <invalid code address>
=== Thread 2314
(0xd0055d8c+0x000000a8) _event_sleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd00563e0+0x00000060) _event_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0061c54+0x000002e0) _cond_wait_local [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062478+0x00000054) _cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062c08+0x000002dc) pthread_cond_timedwait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd351999c+0x0000028c) wait__12ImbSemaphoreFl [/var/mqsi/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd9585e34+0x0000155c) run__Q2_16ImbPubSubThreads14DeletionThreadFP11ImbOsThread [/usr/opt/mqsi/lil/imbdfjps.lil]
(0xd3527410+0x00000070) run__27ImbThreadPoolThreadFunctionFP11ImbOsThread [/var/mqsi/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd351a9b0+0x00000054) threadRun__11ImbOsThreadFv [/var/mqsi/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd351a554+0x00000064) threadBootStrap__11ImbOsThreadFPv [/var/mqsi/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd004b490+0x000000ec) _pthread_body [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0x00000000) <invalid code address>
=== Thread 2571
(0xd0055d8c+0x000000a8) _event_sleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd00563e0+0x00000060) _event_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0061c54+0x00000414) _cond_wait_local [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062478+0x00000054) _cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062f94+0x000001dc) pthread_cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd2fa495c+0x00000050) condvarWait [/usr/java131/jre/bin/libhpi.a]
(0xd2fa35a8+0x000000c4) sysMonitorWait [/usr/java131/jre/bin/libhpi.a]
(0xd3a9515c+0x000001b4) lkMonitorWait [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a07c0c+0x00000140) JVM_MonitorWait [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a34c98+0x0000019c) sysInvokeNative [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a0f088+0x000002a0) mmisInvokeJniMethodHelper [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a35360+0xfffffec4) changeCodes (name probably invalid) [/usr/java131/jre/bin/classic/libjvm.a]
=== Thread 2828
(0xd0055d8c+0x000000a8) _event_sleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd00563e0+0x00000060) _event_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0061c54+0x00000414) _cond_wait_local [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062478+0x00000054) _cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062f94+0x000001dc) pthread_cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd2fa495c+0x00000050) condvarWait [/usr/java131/jre/bin/libhpi.a]
(0xd2fa35a8+0x000000c4) sysMonitorWait [/usr/java131/jre/bin/libhpi.a]
(0xd3a9515c+0x000001b4) lkMonitorWait [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a07c0c+0x00000140) JVM_MonitorWait [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a34c98+0x0000019c) sysInvokeNative [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a0f088+0x000002a0) mmisInvokeJniMethodHelper [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a35360+0xfffffec4) changeCodes (name probably invalid) [/usr/java131/jre/bin/classic/libjvm.a]
=== Thread 3085
(0xd0055d8c+0x000000a8) _event_sleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd00563e0+0x00000060) _event_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0061c54+0x00000414) _cond_wait_local [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062478+0x00000054) _cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062f94+0x000001dc) pthread_cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd2fa495c+0x00000050) condvarWait [/usr/java131/jre/bin/libhpi.a]
(0xd2fa35a8+0x000000c4) sysMonitorWait [/usr/java131/jre/bin/libhpi.a]
(0xd3a9515c+0x000001b4) lkMonitorWait [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a07c0c+0x00000140) JVM_MonitorWait [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a34c98+0x0000019c) sysInvokeNative [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a0f088+0x000002a0) mmisInvokeJniMethodHelper [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a35360+0xfffffec4) changeCodes (name probably invalid) [/usr/java131/jre/bin/classic/libjvm.a]
=== Thread 3342
(0xd0055d8c+0x000000a8) _event_sleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd00563e0+0x00000060) _event_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0061c54+0x00000414) _cond_wait_local [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062478+0x00000054) _cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062f94+0x000001dc) pthread_cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd2fa495c+0x00000050) condvarWait [/usr/java131/jre/bin/libhpi.a]
(0xd2fa35a8+0x000000c4) sysMonitorWait [/usr/java131/jre/bin/libhpi.a]
(0xd3a9515c+0x000001b4) lkMonitorWait [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a07c0c+0x00000140) JVM_MonitorWait [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a34c98+0x0000019c) sysInvokeNative [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a0f088+0x000002a0) mmisInvokeJniMethodHelper [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a35360+0xfffffec4) changeCodes (name probably invalid) [/usr/java131/jre/bin/classic/libjvm.a]
=== Thread 3599
(0xd0055d8c+0x000000a8) _event_sleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd00563e0+0x00000060) _event_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0061c54+0x000002e0) _cond_wait_local [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062478+0x00000054) _cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062c08+0x000002dc) pthread_cond_timedwait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd2fa4530+0x0000019c) condvarTimedWaitUpTo248Days [/usr/java131/jre/bin/libhpi.a]
(0xd2fa4890+0x000000a0) condvarTimedWait [/usr/java131/jre/bin/libhpi.a]
(0xd2fa35a8+0x000000e0) sysMonitorWait [/usr/java131/jre/bin/libhpi.a]
(0xd3a9515c+0x000001b4) lkMonitorWait [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a07c0c+0x00000140) JVM_MonitorWait [/usr/java131/jre/bin/classic/libjvm.a]
(0x00000000+0x00000000) <no name available> [/usr/java131/jre/bin/classic/libjvm.a]
=== Thread 3856
(0xd0055d8c+0x000000a8) _event_sleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd00563e0+0x00000060) _event_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0061c54+0x000002e0) _cond_wait_local [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062478+0x00000054) _cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062c08+0x000002dc) pthread_cond_timedwait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd2fa4530+0x0000019c) condvarTimedWaitUpTo248Days [/usr/java131/jre/bin/libhpi.a]
(0xd2fa4890+0x000000a0) condvarTimedWait [/usr/java131/jre/bin/libhpi.a]
(0xd2fa35a8+0x000000e0) sysMonitorWait [/usr/java131/jre/bin/libhpi.a]
(0xd3a9515c+0x000001b4) lkMonitorWait [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a07c0c+0x00000140) JVM_MonitorWait [/usr/java131/jre/bin/classic/libjvm.a]
(0x00000000+0x00000000) <no name available> [/usr/java131/jre/bin/classic/libjvm.a]
=== Thread 4113
(0xd0055d8c+0x000000a8) _event_sleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd00563e0+0x00000060) _event_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0061c54+0x00000414) _cond_wait_local [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062478+0x00000054) _cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062f94+0x000001dc) pthread_cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd2fa495c+0x00000050) condvarWait [/usr/java131/jre/bin/libhpi.a]
(0xd2fa35a8+0x000000c4) sysMonitorWait [/usr/java131/jre/bin/libhpi.a]
(0xd3a9515c+0x000001b4) lkMonitorWait [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a07c0c+0x00000140) JVM_MonitorWait [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a34c98+0x0000019c) sysInvokeNative [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a0f088+0x000002a0) mmisInvokeJniMethodHelper [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a35360+0xfffffec4) changeCodes (name probably invalid) [/usr/java131/jre/bin/classic/libjvm.a]
=== Thread 4370
(0xd0055d8c+0x000000a8) _event_sleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd00563e0+0x00000060) _event_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0061c54+0x000002e0) _cond_wait_local [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062478+0x00000054) _cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062c08+0x000002dc) pthread_cond_timedwait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd2fa4530+0x0000019c) condvarTimedWaitUpTo248Days [/usr/java131/jre/bin/libhpi.a]
(0xd2fa4890+0x000000a0) condvarTimedWait [/usr/java131/jre/bin/libhpi.a]
(0xd2fa35a8+0x000000e0) sysMonitorWait [/usr/java131/jre/bin/libhpi.a]
(0xd3a5416c+0x00000168) xmThreadSleep [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a04298+0x00000118) JVM_Sleep [/usr/java131/jre/bin/classic/libjvm.a]
(0x00000000+0x00000000) <no name available> [/usr/java131/jre/bin/classic/libjvm.a]
=== Thread 4627
(0xd0055d8c+0x000000a8) _event_sleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd00563e0+0x00000060) _event_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0061c54+0x000002e0) _cond_wait_local [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062478+0x00000054) _cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062c08+0x000002dc) pthread_cond_timedwait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd2fa4530+0x0000019c) condvarTimedWaitUpTo248Days [/usr/java131/jre/bin/libhpi.a]
(0xd2fa4890+0x000000a0) condvarTimedWait [/usr/java131/jre/bin/libhpi.a]
(0xd2fa35a8+0x000000e0) sysMonitorWait [/usr/java131/jre/bin/libhpi.a]
(0xd3a5416c+0x00000168) xmThreadSleep [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a04298+0x00000118) JVM_Sleep [/usr/java131/jre/bin/classic/libjvm.a]
(0x00000000+0x00000000) <no name available> [/usr/java131/jre/bin/classic/libjvm.a]
=== Thread 4884
(0xd0055d8c+0x000000a8) _event_sleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd00563e0+0x00000060) _event_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0061c54+0x000002e0) _cond_wait_local [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062478+0x00000054) _cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062c08+0x000002dc) pthread_cond_timedwait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd2fa4530+0x0000019c) condvarTimedWaitUpTo248Days [/usr/java131/jre/bin/libhpi.a]
(0xd2fa4890+0x000000a0) condvarTimedWait [/usr/java131/jre/bin/libhpi.a]
(0xd2fa35a8+0x000000e0) sysMonitorWait [/usr/java131/jre/bin/libhpi.a]
(0xd3a5416c+0x00000168) xmThreadSleep [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a04298+0x00000118) JVM_Sleep [/usr/java131/jre/bin/classic/libjvm.a]
(0x00000000+0x00000000) <no name available> [/usr/java131/jre/bin/classic/libjvm.a]
=== Thread 5141
No stack information available
=== Thread 5398
(0xd02f12f4+0x00000080) semop [/usr/java131/jre/bin/classic/libjvm.a]
(0xd6ec6918+0x00000000) <no name available> [/usr/mqm/lib/libmqmcs_r.a(shr.o)]
(0xd70e3a7c+0x00000000) <no name available> [/usr/mqm/lib/libmqz_r.a(libmqz_r.o)]
(0xd70e0dd0+0x00000000) <no name available> [/usr/mqm/lib/libmqz_r.a(libmqz_r.o)]
(0xd70e6a44+0x00000000) <no name available> [/usr/mqm/lib/libmqz_r.a(libmqz_r.o)]
(0xd70b1db4+0x00000000) <no name available> [/usr/mqm/lib/libmqz_r.a(libmqz_r.o)]
(0xd44c6af4+0x00000000) <no name available> [/usr/mqm/lib/libmqm_r.a(libmqm_r.o)]
(0xd6b70620+0x000027c8) readQueue__14ImbMqInputNodeFP11ImbOsThread [/var/mqsi/lib/libMQLibrary.a(libMQLibrary.a.so)]
(0xd6b7a314+0x00000048) run__Q2_14ImbMqInputNode10ParametersFP11ImbOsThread [/var/mqsi/lib/libMQLibrary.a(libMQLibrary.a.so)]
(0xd3527410+0x00000070) run__27ImbThreadPoolThreadFunctionFP11ImbOsThread [/var/mqsi/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd351a9b0+0x00000054) threadRun__11ImbOsThreadFv [/var/mqsi/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd351a554+0x00000064) threadBootStrap__11ImbOsThreadFPv [/var/mqsi/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd004b490+0x000000ec) _pthread_body [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0x00000000) <invalid code address>
=== Thread 5655
(0xd02f12f4+0x00000080) semop [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd6ec6918+0x00000000) <no name available> [/usr/mqm/lib/libmqmcs_r.a(shr.o)]
(0xd70e3a7c+0x00000000) <no name available> [/usr/mqm/lib/libmqz_r.a(libmqz_r.o)]
(0xd70e0dd0+0x00000000) <no name available> [/usr/mqm/lib/libmqz_r.a(libmqz_r.o)]
(0xd70e6a44+0x00000000) <no name available> [/usr/mqm/lib/libmqz_r.a(libmqz_r.o)]
(0xd70b1db4+0x00000000) <no name available> [/usr/mqm/lib/libmqz_r.a(libmqz_r.o)]
(0xd44c6af4+0x00000000) <no name available> [/usr/mqm/lib/libmqm_r.a(libmqm_r.o)]
(0xd6b70620+0x000027c8) readQueue__14ImbMqInputNodeFP11ImbOsThread [/var/mqsi/lib/libMQLibrary.a(libMQLibrary.a.so)]
(0xd6b7a314+0x00000048) run__Q2_14ImbMqInputNode10ParametersFP11ImbOsThread [/var/mqsi/lib/libMQLibrary.a(libMQLibrary.a.so)]
(0xd3527410+0x00000070) run__27ImbThreadPoolThreadFunctionFP11ImbOsThread [/var/mqsi/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd351a9b0+0x00000054) threadRun__11ImbOsThreadFv [/var/mqsi/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd351a554+0x00000064) threadBootStrap__11ImbOsThreadFPv [/var/mqsi/lib/libCommonServices.a(libCommonServices.a.so)]
(0xd004b490+0x000000ec) _pthread_body [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0x00000000) <invalid code address>
=== Thread 5912
(0xd0055d8c+0x000000a8) _event_sleep [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd00563e0+0x00000060) _event_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0061c54+0x000002e0) _cond_wait_local [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062478+0x00000054) _cond_wait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd0062c08+0x000002dc) pthread_cond_timedwait [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd2fa4530+0x0000019c) condvarTimedWaitUpTo248Days [/usr/java131/jre/bin/libhpi.a]
(0xd2fa4890+0x000000a0) condvarTimedWait [/usr/java131/jre/bin/libhpi.a]
(0xd2fa35a8+0x000000e0) sysMonitorWait [/usr/java131/jre/bin/libhpi.a]
(0xd3a5416c+0x00000168) xmThreadSleep [/usr/java131/jre/bin/classic/libjvm.a]
(0xd3a04298+0x00000118) JVM_Sleep [/usr/java131/jre/bin/classic/libjvm.a]
(0x00000000+0x00000000) <no name available> [/usr/java131/jre/bin/classic/libjvm.a]
=== Thread 6169
No stack information available
----------------------------------------------------------------------
We do also get the occassional entry about lost DB connections (but these dont seem to match up with the abended execution groups - i.e. we might get one lost DB connection error reported in a day but 12 abended execution groups) :
Jul 25 12:00:00 MQSIUAT1 MQSIv500[74076]: (MQSBK01A.EX15)[5141]BIP2232E: Error detected whilst handling a previous error in node 'MID_Get_Cargo.CH_XXX_HandleException1.Throw 3001'. : MQSBK01A.a9de5bb3-fd00-0000-0080-cab9a07f464c: /build/S500_P/src/DataFlowEngine/ImbDataFlowNode.cpp: 387: ImbDataFlowNode::logExceptionList: ComIbmThrowNode: MID_Get_Cargo#FCMComposite_1_2.CH_XXX_HandleException#FCMComposite_1_3
Jul 25 12:00:00 MQSIUAT1 MQSIv500[74076]: (MQSBK01A.EX15)[5141]BIP2230E: Error detected whilst processing a message in node 'MID_Get_Cargo.Save Reply Details'. : MQSBK01A.a9de5bb3-fd00-0000-0080-cab9a07f464c: /build/S500_P/src/DataFlowEngine/ImbComputeNode.cpp: 471: ImbComputeNode::evaluate: ComIbmComputeNode: MID_Get_Cargo#FCMComposite_1_6
Jul 25 12:00:00 MQSIUAT1 MQSIv500[74076]: (MQSBK01A.EX15)[5141]BIP2488E: (.MID_Get_Cargo_Save Reply Details.main, 89.1) Error detected whilst executing the SQL statement 'PASSTHRU('INSERT INTO MID_Request_Reply (MsgId,: MQSBK01A.a9de5bb3-fd00-0000-0080-cab9a07f464c: /build/S500_P/src/DataFlowEngine/ImbRdl/ImbRdlStatementGroup.cpp: 158: SqlStatementGroup::execute: ComIbmComputeNode: MID_Get_Cargo#FCMComposite_1_6
Jul 25 12:00:00 MQSIUAT1 MQSIv500[74076]: (MQSBK01A.EX15)[5141]BIP2519E: (.MID_Get_Cargo_Save Reply Details.main, 89.1) : Error executing SQL statement 'INSERT INTO MID_Request_Reply (MsgId,: MQSBK01A.a9de5bb3-fd00-0000-0080-cab9a07f464c: /build/S500_P/src/DataFlowEngine/ImbRdl/ImbRdlExternalDb.cpp: 277: SqlExternalDbStmt::executeStmt: ComIbmComputeNode: MID_Get_Cargo#FCMComposite_1_6
Jul 25 12:00:00 MQSIUAT1 MQSIv500[74076]: (MQSBK01A.EX15)[5141]BIP2321E: Database error: ODBC return code '-1'. : MQSBK01A.a9de5bb3-fd00-0000-0080-cab9a07f464c: /build/S500_P/src/DataFlowEngine/ImbOdbc.cpp: 185: ImbOdbcHandle::checkRcInner: :
Jul 25 12:00:00 MQSIUAT1 MQSIv500[74076]: (MQSBK01A.EX15)[5141]BIP2322E: Database error: SQL State '40003'; Native Error Code '-30081'; Error Text '[IBM][CLI Driver] SQL30081N A communication error has been detected. Communication protocol being used: "TCP/IP". Communication API being used: "SOCKETS". Location where the error was detected: "10.109.234.17". Communication function detecting the error: "send". Protocol specific error code(s): "32", "*", "0". SQLSTATE=08001 '. : MQSBK01A.a9de5bb3-fd00-0000-0080-cab9a07f464c: /build/S500_P/src/DataFlowEngine/ImbOdbc.cpp: 327: ImbOdbcHandle::checkRcInner: :
Jul 25 12:00:00 MQSIUAT1 MQSIv500[74076]: (MQSBK01A.EX15)[5141]BIP2628E: Exception condition detected on input node 'MID_Get_Cargo.Request In'. : MQSBK01A.a9de5bb3-fd00-0000-0080-cab9a07f464c: /build/S500_P/src/DataFlowEngine/ImbMqInputNode.cpp: 1781: ImbMqInputNode::readQueue: ComIbmMQInputNode: MID_Get_Cargo#FCMComposite_1_19
Jul 25 12:00:00 MQSIUAT1 MQSIv500[74076]: (MQSBK01A.EX15)[5141]BIP3001I: Exception thrown by throw node 'MID_Get_Cargo.CH_XXX_HandleException1.Throw 3001'; text is 'To back out queue'. : MQSBK01A.a9de5bb3-fd00-0000-0080-cab9a07f464c: /build/S500_P/src/DataFlowEngine/BasicNodes/ImbThrowNode.cpp: 259: ImbThrowNode::evaluate: ComIbmThrowNode: MID_Get_Cargo#FCMComposite_1_2.CH_XXX_HandleException#FCMComposite_1_3
Jul 25 12:00:00 MQSIUAT1 MQSIv500[94052]: (MQSBK01A.EX14)[5141]BIP2648E: Message backed out to a queue; node 'MID_Get_Cargo.Request In'. : MQSBK01A.7cb95bb3-fd00-0000-0080-cab9a07f464c: /build/S500_P/src/DataFlowEngine/ImbMqInputNode.cpp: 3096: ImbMqInputNode::eligibleForBackout: ComIbmMQInputNode: MID_Get_Cargo#FCMComposite_1_19 |
|
| Back to top |
|
|
| m00300 |
| Posted: Wed Jul 28, 2004 2:00 am Post subject: |
|
|
Apprentice
Joined: 01 May 2002
Posts: 31
|
There was a DB connection error in the syslog (similar to the one i've included above) just before the original abend in my first post.
Could it be that we get only one DB connection error reported and then this causes a number of execution groups to abend (without further DB connection errors reported) as and when they process messages next ?
I have raised this connection error with our IBM DB team but they couldnt resolve it and then we went out of support  .....but we're back in support now so i've re-raised it again. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Jul 28, 2004 4:40 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| m00300 wrote: |
The abend file contains :
abend record for pid 54730 tid 5141 time in seconds since 01/01/1970: 1090484035
File: /build/S500_P/src/CommonServices/Unix/ImbAbend.cpp
Line: 458
Function: signal received
---- Inserts ----
11
@(#) 1.33.2.2 CommonServices/Unix/ImbAbend.cpp, CommonServices, S500, S500-L31218 03/11/25 15:51:20 [12/18/03 17:10:51]
1044085984 |
I'm not big on abend files, but that looks like a signal 11 to the execution group. Signal 11 is a segmentation fault.
Did you set up all the kernel resources to meet the needs of your broker system?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
