| Author |
Message
|
| rohank84 |
 Posted: Tue Apr 16, 2013 8:04 am Post subject: Need Message Flow name from core dump |
 |
|
Centurion
Joined: 31 Dec 2008
Posts: 109
|
Hi Guys,
There is a issue which has come up in our UAT environment. We are using broker version 8.0.0.1 with MQ 7
There are bunch of message flows and execution group which are deployed onto a broker. The things were working fine last week. After the weekly reboot of the system and starting of broker we found that there in some message flow/flows which is causing some issues and throwing core dump on the home2 directory. We identified this throw the PID which depicts DataFlowEngine.
Now the core dump is pretty big and we need to know which message flow is throwing that. Reading a core dump is not helping anyway.
Do anyone know how to identify a message flow/flows from a core dump.
Thanks in advance |
|
| Back to top |
|
 |
| vmcgloin |
| Posted: Tue Apr 16, 2013 1:15 pm Post subject: |
|
|
Knight
Joined: 04 Apr 2002
Posts: 560
Location: Scotland
|
The short answer is probably to raise a PMR and ask the experts (not disparaging the support you'll get here but they do know best  )
I don't know the long answer but I'd ask the following questions...
How do you know it is just one message flow and not a combination of factors? How many flows are in the execution group? Can you recreate it elsewhere? What testing is happening when the core happens? Which flows have been modified recently - can you isolate them and/or do some tracing to look for problems? |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Apr 16, 2013 1:31 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
There should be specific messages in the system log that accompany the creation of a coredump.
There should also be an FDC associated with the coredump as well. |
|
| Back to top |
|
|
| rohank84 |
| Posted: Tue Apr 16, 2013 9:27 pm Post subject: |
|
|
Centurion
Joined: 31 Dec 2008
Posts: 109
|
There are 20 Egs and around 100 message flows .... i rule out the recent changes as the message flows were working fine before the system reboot. The reboot is a weekly activity and happens on Sunday. There were no issues till Saturday evening and hence we decided to carry on with the reboot.
I guess there could be bunch of message flows which are causing the issue or may be some other factors. To begin debugging the issue i only have core dump to start with.
We checked the unix system logs and nothing specific was there regarding the core dump. Regarding the FDC i will check once and update.
Are there anything else that could trigger such a behavior ? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Apr 17, 2013 3:37 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Did you check /var/mqsi/commons/errors? or any similar path in your environment?? There should be an abend file somewhere there that will tell you which eg abended... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Apr 17, 2013 4:41 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Please do open a PMR. IBM wants to know about any abnormal termination of the DataFlowEngine process so that we can fix the problem.
Sounds as if you already have a core file - that's good, because it is probably one of the first things that the IBM service team will request. Please keep all core files, and all abend files, and zip up the contents of the stdout/stderr log files. |
|
| Back to top |
|
|
| rohank84 |
| Posted: Fri Apr 19, 2013 7:10 am Post subject: |
|
|
Centurion
Joined: 31 Dec 2008
Posts: 109
|
Hi Guys ..... Below is what is there in the abend file .... i did not understand it fully  (((
And we have broker 6.0 on this environment...sorry for misinforming earlier......
abend record for pid 614598 tid 1 time in seconds since 01/01/1970: 1366293314
File: /build/S000_P/src/CommonServices/Unix/ImbAbend.cpp
Line: 536
Function: signal received
---- Inserts ----
6
@(#) 1.33.1.16 CommonServices/Unix/ImbAbend.cpp, CommonServices, S000 05/02/17 13:14:39 [2/17/05 14:29:09]
804379760
-----------------
----------------------------- Stack dump for current thread ( 1)
(0x00000000) <invalid code address>
(0xd0062ccc+0x00000058) _p_raise [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd01fc070+0x00000034) raise [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd021c6a0+0x000000b8) abort [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd8bfc8a8+0x00000094) jni_FatalError [/opt/IBM/mqsi/6.0/jre/bin/classic/libjvm.a]
(0xd8c1e370+0x00000258) xeRunJniMethod [/opt/IBM/mqsi/6.0/jre/bin/classic/libjvm.a]
(0xd8bfb7d8+0x00000104) invokeJniMethod [/opt/IBM/mqsi/6.0/jre/bin/classic/libjvm.a]
(0xd8bf8f88+0x0000003c) jni_CallLongMethodV [/opt/IBM/mqsi/6.0/jre/bin/classic/libjvm.a]
(0xdafc1a28+0x00000078) mbBrokerExceptionToNativeException__21ImbJavaExceptionUtilsFP7JNIEnv_P11_jthrowable [/opt/IBM/mqsi/6.0
/lib/libimbjplg.a]
(0xdafbbe68+0x000000e8) handleJavaException__21ImbJavaExceptionUtilsFP7JNIEnv_iP10ImbWstringT2P11_jthrowableP8_jobject [/opt/I
BM/mqsi/6.0/lib/libimbjplg.a]
(0xdb06c440+0x00000014) handleJavaException__21ImbJavaExceptionUtilsFP7JNIEnv_iP10ImbWstringT2P11_jthrowable [/opt/IBM/mqsi/6.
0/lib/libimbjplg.a]
(0xdb096868+0x00000388) bipGetJavaPluginNodeFactory [/opt/IBM/mqsi/6.0/lib/libimbjplg.a]
(0xdaf9e128+0x00000010) bipGetMessageflowNodeFactory [/opt/IBM/mqsi/6.0/lil/imbjplug2.lil]
(0xd77a4254+0x0000083c) __ct__10ImbLibraryFRC10ImbWstring [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)
]
(0xd77a39b8+0x000000dc) loadLibrary__18ImbResourceManagerFRC10ImbWstring [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessag
eServices.a.so)]
(0xd9a09b88+0x00002b3c) start__7ImbMainFR18ImbStartParameters [/opt/IBM/mqsi/6.0/lib/libbipmain.a]
(0x10002058+0x00000408) main [DataFlowEngine]
(0x10000128+0x0000008c) __start [DataFlowEngine]
----------------------------------------------------------------------
abend record for pid 614598 tid 1 time in seconds since 01/01/1970: 1366302119
File: /build/S000_P/src/CommonServices/Unix/ImbAbend.cpp
Line: 536
Function: signal received
---- Inserts ----
6
@(#) 1.33.1.16 CommonServices/Unix/ImbAbend.cpp, CommonServices, S000 05/02/17 13:14:39 [2/17/05 14:29:09]
804379760
-----------------
----------------------------- Stack dump for current thread ( 1)
(0x00000000) <invalid code address>
(0xd0062ccc+0x00000058) _p_raise [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd01fc070+0x00000034) raise [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd021c6a0+0x000000b8) abort [/usr/lib/libpthreads.a(shr_xpg5.o)]
(0xd8bfc8a8+0x00000094) jni_FatalError [/opt/IBM/mqsi/6.0/jre/bin/classic/libjvm.a]
(0xd8c1e370+0x00000258) xeRunJniMethod [/opt/IBM/mqsi/6.0/jre/bin/classic/libjvm.a]
(0xd8bfb7d8+0x00000104) invokeJniMethod [/opt/IBM/mqsi/6.0/jre/bin/classic/libjvm.a]
(0xd8bf8f88+0x0000003c) jni_CallLongMethodV [/opt/IBM/mqsi/6.0/jre/bin/classic/libjvm.a]
(0xdafc1a28+0x00000078) mbBrokerExceptionToNativeException__21ImbJavaExceptionUtilsFP7JNIEnv_P11_jthrowable [/opt/IBM/mqsi/6.0
/lib/libimbjplg.a]
(0xdafbbe68+0x000000e8) handleJavaException__21ImbJavaExceptionUtilsFP7JNIEnv_iP10ImbWstringT2P11_jthrowableP8_jobject [/opt/I
BM/mqsi/6.0/lib/libimbjplg.a]
(0xdb06c440+0x00000014) handleJavaException__21ImbJavaExceptionUtilsFP7JNIEnv_iP10ImbWstringT2P11_jthrowable [/opt/IBM/mqsi/6.
0/lib/libimbjplg.a]
(0xdb096868+0x00000388) bipGetJavaPluginNodeFactory [/opt/IBM/mqsi/6.0/lib/libimbjplg.a]
(0xdaf9e128+0x00000010) bipGetMessageflowNodeFactory [/opt/IBM/mqsi/6.0/lil/imbjplug2.lil]
(0xd77a4254+0x0000083c) __ct__10ImbLibraryFRC10ImbWstring [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessageServices.a.so)
]
(0xd77a39b8+0x000000dc) loadLibrary__18ImbResourceManagerFRC10ImbWstring [/opt/IBM/mqsi/6.0/lib/libMessageServices.a(libMessag
eServices.a.so)]
(0xd9a09b88+0x00002b3c) start__7ImbMainFR18ImbStartParameters [/opt/IBM/mqs |
|
| Back to top |
|
|
| rohank84 |
| Posted: Fri Apr 19, 2013 7:12 am Post subject: |
|
|
Centurion
Joined: 31 Dec 2008
Posts: 109
|
| the FDC files are not created ....checked almost all the directories linked to wmb .... |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Apr 19, 2013 7:21 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| rohank84 wrote: |
Hi Guys ..... Below is what is there in the abend file .... i did not understand it fully (((
And we have broker 6.0 on this environment...sorry for misinforming earlier...... |
Then you can't open a PMR, and very likely not many people can help you.
There may be a javacore or java heapdump or other java crash file instead of an FDC. I don't know that reading them will provide any help.
The file you've posted says that something received a Signal 6. this is a SIGABRT.
I don't know that that provides you any help.
Perform troubleshooting. Review all events before and during and after each abend. Determine if you can recreate the cause of an abend. Enable additional monitoring to get a better idea of what may be contributing to the abend. |
|
| Back to top |
|
|
|
|
