| Author |
Message
|
| narki |
 Posted: Wed Aug 11, 2010 9:43 am Post subject: Failed to allocate memory |
 |
|
Acolyte
Joined: 19 Dec 2005
Posts: 67
|
| I am having stage problem. I have 2MB message, that hase 22,000 records. I have to parse each record and transform it to o/p format. All the output record should be appened one after another. I am using the create parse and asbit stream function. My flow works fine upto 1MB message but it breaks on 2MB. I am getting failed to allocate memory error in abend file. There is enough memeory , onlt cpu usage is going to 93%. Please help. |
|
| Back to top |
|
 |
| Gaya3 |
| Posted: Wed Aug 11, 2010 9:54 am Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
usually the MQ can handle 4 MB of message, neither we touch the CSTACK to do the things...
hope your MQ is fine in this case to send messages more than 1 MB of messages
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Aug 11, 2010 9:58 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Using Create Field PARSE and ASBITSTREAM is not the preferred method of doing this.
A properly constructed MRM model will handle all of this without significant effort.
What specific reason do you have for using CREATE FIELD and ASBITSTREAM?
The most likely reason for a high CPU usage is a loop in your message flow, for example by wiring nodes into a loop form.
Also are you using indexed field references - field[i] rather than using REFERENCE variables? That would also explain your situation. |
|
| Back to top |
|
|
| narki |
| Posted: Wed Aug 11, 2010 10:41 am Post subject: |
|
|
Acolyte
Joined: 19 Dec 2005
Posts: 67
|
Generally create parse and asbit stream statement are good for handling very large messages. It is failing for 2MB message record. I may be getting message which will be more than 10MB. Don't you think if I parsing the complete message is good idea.
Also can you please explain this??
Also are you using indexed field references - field[i] rather than using REFERENCE variables? That would also explain your situation. |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Aug 11, 2010 11:27 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
|
| Back to top |
|
|
| narki |
| Posted: Wed Aug 11, 2010 11:31 am Post subject: |
|
|
Acolyte
Joined: 19 Dec 2005
Posts: 67
|
| yes I am using same kind of technique, but I am not propagating it because it has to go as a single messages. Also I am using the reference variables. But while loop also look very aggressive, basically it is looping through 22,000 times |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Aug 11, 2010 11:37 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
It's also in the article you mention.
So presumably, if narki is using these techniques, then using references is NOT the cause of the issue. |
|
| Back to top |
|
|
| narki |
| Posted: Wed Aug 11, 2010 1:26 pm Post subject: |
|
|
Acolyte
Joined: 19 Dec 2005
Posts: 67
|
| Please help, any sugesstion will be helpfull. |
|
| Back to top |
|
|
| kimbert |
| Posted: Wed Aug 11, 2010 2:17 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
| but I am not propagating it because it has to go as a single messages |
That's fair enough - you need to assemble a single output message. But you should still be DELETEing each part of the parsed input data and if you are building a message tree ( under OutputRoot or anywhere else ) you must manually DELETE that tree fragment each time around the loop ( because PROPAGATE will not be doing it for you ). |
|
| Back to top |
|
|
| narki |
| Posted: Wed Aug 11, 2010 2:22 pm Post subject: |
|
|
Acolyte
Joined: 19 Dec 2005
Posts: 67
|
Kimbert,
I am doing this , I have done it several times and was able to process more than 2MB of message. I am not sure why this is abending here. I am guessing some machine configurationtion needs to be changed . If any one of you have come across this situation please let me know . |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Aug 11, 2010 2:53 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
|
| Back to top |
|
|
| narki |
| Posted: Wed Aug 11, 2010 4:28 pm Post subject: |
|
|
Acolyte
Joined: 19 Dec 2005
Posts: 67
|
| I know where it is taking time but what is solution, ultimately message is backing out. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Aug 11, 2010 5:36 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Where is it spending the time?
In a *loop*?
There isn't *A* solution, at least from the data you have provided here. There are many *possible* solutions, but there's no way to narrow it down.
You *might* find relief if you ensure you are at the most recent fix level of the version of the product you are using, which you haven't mentioned. |
|
| Back to top |
|
|
| narki |
| Posted: Thu Aug 12, 2010 4:21 am Post subject: |
|
|
Acolyte
Joined: 19 Dec 2005
Posts: 67
|
For WMB
/opt/IBM/mqsi/6.0=6.0.0.3
For MQ
Name: WebSphere MQ
Version: 6.0.1.1
CMVC level: p600-101-060504
BuildType: IKAP - (Production)
For AIX
Version : 5.3.0.0
Number Of Processors: 1
Processor Clock Speed: 4400 MHz
CPU Type: 64-bit
Kernel Type: 64-bit
LPAR Info: 10 soaint
Memory Size: 4096 MB
Good Memory Size: 4096 MB
Yeat it is taking more time where looping is happening. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Aug 12, 2010 4:28 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
The most recent fixpack of Message Broker 6.0 is FixPack 10. You are at FixPack 3.
How have you constructed the loop that is taking up all of the time? Can we see the ESQL - at least the portions relevant to the question. |
|
| Back to top |
|
|
|
|
