| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Analyzing Message broker Core Files. |
« View previous topic :: View next topic » |
| Author |
Message
|
| JohnSmith |
 Posted: Wed Apr 21, 2010 7:39 pm Post subject: Analyzing Message broker Core Files. |
 |
|
Voyager
Joined: 17 Mar 2010
Posts: 86
|
Hello All,
Recently we faced a problem with one of our message flow. When we deploy that flow to the execution group, after some time it started writing core dump files into the user home directory(we are running broker on AIX).
many large files with names starting with core.****.**** started generating in directory. Now, we know the root cause of the problem lies somewhere in those core dumps files. But, we dont know how to analyze them, is there any command or tool provided by IBM which can help us to analyze these logs??
Following is our environment -
Message Broker V6.1
OS - AIX
Any help, experts?
Regards, |
|
| Back to top |
|
 |
| smdavies99 |
| Posted: Wed Apr 21, 2010 10:20 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
If this is a repeating problem then I'd raise a PMR and get the experts to do it for you.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Apr 22, 2010 1:35 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
IBM Support through a PMR is the only way to analyze those core files. You *are* running a recent fixpack of a supported version of broker, right?
If you have identified a specific message flow that, when deployed, causes broker to take core dumps, then you should revisit the design and implementation of the message flow for the following things:
- user defined nodes and custom user exits
- nodes wired back upstream in a loop
- extensive use of field[i] instead of reference variables when working with message trees with more than about 10 children
- very large database result sets
These are the most likely suspects, and you should review all of these cases anyway.
A core file in general is just a dump of the contents of "core memory" at the time of the abend. The abend file and or the ffst file are much more likely to have useful information about the cause of the error.
And of course the BIP errors in the system log. And the contents of a trace taken during an occurance of the core dump. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Apr 22, 2010 5:27 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mqjeff wrote: |
| You *are* running a recent fixpack of a supported version of broker, right? |
He said v6.1, which I think is still in support.
But I agree completely with your diagnostic advice.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| JohnSmith |
| Posted: Thu Apr 22, 2010 4:20 pm Post subject: |
|
|
Voyager
Joined: 17 Mar 2010
Posts: 86
|
mqjeff wrote:
| Quote: |
If you have identified a specific message flow that, when deployed, causes broker to take core dumps, then you should revisit the design and implementation of the message flow for the following things:
user defined nodes and custom user exits
nodes wired back upstream in a loop
extensive use of field[i] instead of reference variables when working with message trees with more than about 10 children
very large database result sets |
we found after code review, there were one infinite while loop in ESQL programming  . We have fixed that and deploy the code again, now it is working fine.
smdavies wrote:
| Quote: |
| If this is a repeating problem then I'd raise a PMR and get the experts to do it for you. |
Yes, we also thought of raising a PMR, but then it is a slow approach as we have realized in the past, if IBM can provide a tool handy to analyze logs then we can come to a bit of conclusion ourselves. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Apr 23, 2010 3:26 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| JohnSmith wrote: |
mqjeff wrote:
| Quote: |
If you have identified a specific message flow that, when deployed, causes broker to take core dumps, then you should revisit the design and implementation of the message flow for the following things:
user defined nodes and custom user exits
nodes wired back upstream in a loop
extensive use of field[i] instead of reference variables when working with message trees with more than about 10 children
very large database result sets |
we found after code review, there were one infinite while loop in ESQL programming . We have fixed that and deploy the code again, now it is working fine. |
Yes, an infinite loop in ESQL would also cause this. Very well done for finding the issue. The first place in finding issues is to always identify the change that was made between "when things worked" and "when things didn't". In your case, it was that you deployed the flow. So the problem must be in the flow.
smdavies wrote:
| JohnSmith wrote: |
| Quote: |
| If this is a repeating problem then I'd raise a PMR and get the experts to do it for you. |
Yes, we also thought of raising a PMR, but then it is a slow approach as we have realized in the past, if IBM can provide a tool handy to analyze logs then we can come to a bit of conclusion ourselves. |
While I agree that working a PMR can take time, I also repeat my earlier statement that the *contents* of core files are almost irrelevant in most situations. The existance of a core file is a useful fact, but the contents are only useful in very specific circumstances.
In your case, a user trace would have shown you the infinite loop. And user trace is already human readable (after mqsiformatlog, that is).
You might also look at IBM Support Assistant. It has a lot of tools for analyzing and gathering data. |
|
| Back to top |
|
|
| JohnSmith |
| Posted: Fri Apr 23, 2010 8:02 am Post subject: |
|
|
Voyager
Joined: 17 Mar 2010
Posts: 86
|
mqjeff wrote:
| Quote: |
| You might also look at IBM Support Assistant. It has a lot of tools for analyzing and gathering data. |
Thanks Jeff, I would definitely have a look at the same.
Regards,
Smith |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
