| Author |
Message
|
| saurabh867 |
 Posted: Thu Apr 12, 2012 3:27 am Post subject: Message failing at MQInput node |
 |
|
Voyager
Joined: 13 Jun 2010
Posts: 78
|
Hi,
In one of our integrations, we receive a lot of big messages (around 40 MB). These messages are sent as a batch job from SAP so we receive a set of 15-17 such big messages and produce a delimited file for each of these messages.
We have a general error handling subflow which is connected to the Catch terminal, the component extracts the ExceptionList and put the message to an Error queue. Also the exception is thrown back using Throw node and we have the Failure terminal connected to MQOutput to store the failed message.
The situation now is that some of these big messages are going to Failure queue but nothing comes up in the Error queue (connected to catch terminal).
From this I could think that the message might not have gone out of the MQInput node and would have failed the node itself. But there is no validation on the node. Message comes as BLOB to the node and the message does not have <mcd> folder.
What might be the reason where the messages are directly routed to theFailure terminal instead of going to Catch terminal.
Also when we replay the failed message, it goes fine. Now where should I look specifically.
I could not find anything on the syslog either.
Broker version : 6.1.0.8 |
|
| Back to top |
|
 |
| McueMart |
| Posted: Thu Apr 12, 2012 3:36 am Post subject: |
|
|
Chevalier
Joined: 29 Nov 2011
Posts: 490
Location: UK...somewhere
|
| Any chance the inbound message could have a Backout Count > 0 already set on it? Its a long shot I know. |
|
| Back to top |
|
|
| Esa |
| Posted: Thu Apr 12, 2012 3:41 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| This can happen for example if the execution group abends while processing the message. The message gets backed out and MQInput propagates it to failure when it gets it. Depending on backout settings for the input queue object. |
|
| Back to top |
|
|
| saurabh867 |
| Posted: Thu Apr 12, 2012 3:44 am Post subject: |
|
|
Voyager
Joined: 13 Jun 2010
Posts: 78
|
Ok. What might be reason for the E.G. abend state?
Should it be due to the large size of the message and insufficient memory for E.G at that time?
I am asking so that I can look at these areas and make some changes.
Regards,
Saurabh |
|
| Back to top |
|
|
| Esa |
| Posted: Thu Apr 12, 2012 4:04 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| saurabh867 wrote: |
Ok. What might be reason for the E.G. abend state?
Should it be due to the large size of the message and insufficient memory for E.G at that time?
I am asking so that I can look at these areas and make some changes.
Regards,
Saurabh |
You did not mention your operating system. Somehow I guess it is AIX.
Processing large messages can be the cause, especially prosessing several of them simultaneously. But your input messages come from SAP, so I suppose the flow has no additional instances.
When you are processing large messages, small mistakes can be expensive. For example unnecessary copying of the message, copying the message to and worth to strip and put back some in-house header, etc. Yes, they can cause it. But it can as well be some other execution group that has reserved too much memory. Usually it is an unhappy coincidence of several things. But bad design, if present, tends to play the leading role.
Is the eg 32-bit? |
|
| Back to top |
|
|
| saurabh867 |
| Posted: Thu Apr 12, 2012 4:28 am Post subject: |
|
|
Voyager
Joined: 13 Jun 2010
Posts: 78
|
Yes, the operating system is AIX.
Message comes from SAP but via old stand alone SAP adapter but still there are no additional instance for the message flow.
E.G. are 64 bit.
Should the increase in memory be an option apart for the design changes?
If yes, can I selectively increase the stack memory for the particular E.G. ?
Regards,
Saurabh |
|
| Back to top |
|
|
| Esa |
| Posted: Thu Apr 12, 2012 4:45 am Post subject: Re: Message failing at MQInput node |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| saurabh867 wrote: |
The situation now is that some of these big messages are going to Failure queue but nothing comes up in the Error queue (connected to catch terminal). |
How often? Does this happen several times in a row or just every now and then?
Do you run mqsireaload for all execution groups regularly?
There may be some other explanation than eg abend. If it is caused by an abend, you should find something in the syslog or abend files. |
|
| Back to top |
|
|
| saurabh867 |
| Posted: Thu Apr 12, 2012 9:07 pm Post subject: |
|
|
Voyager
Joined: 13 Jun 2010
Posts: 78
|
Hi,
This is happening quite often now due to the large traffic of big messages coming recently.
I checked with Admin team and they do not run mqsireload frequently.
I checked the syslog and found the issue was due to the E.G. abend state. I could locate the abend files and all of these are saying "Failed to allocate memory".
I am attaching one of the abend files here:
Probably your analysis of the messages going to Failure queue is correct. In this situation I am thinking of taking these steps:
1. Tune the current design of the message flows to avoid excessive parsing and unnecessary use of extra nodes.
2. Allocate more memory to broker. (How do I do that? Should adding a disk space be sufficient or I should run some commands to allocate the added space to broker or E.G.)
3. Increase the backout count of the Input queue so that it can back-out the message again(If E.G. can process after the start from abend).
Regards,
Saurabh |
|
| Back to top |
|
|
|
|
