| Author |
Message
|
| mqbrk_newbie |
 Posted: Thu Jun 23, 2005 7:55 am Post subject: xml tranformation node! |
 |
|
Acolyte
Joined: 13 Sep 2004
Posts: 70
|
If I try to process the XML file of 10 MB size through xml transformation node, it is throwing me a XML parsing error. If I cut down the XML file size from 10 MB to a few KB it is going through the node. I have checked the validity if XML file as well as the XSLT stylsheet I had, which was correct. Is there anything that broker is taking huge amount of memory while processing such a big XML file and unable to process it or is that something I doing wrong.....
Can anyone help me regarding this. |
|
| Back to top |
|
 |
| chenulu |
| Posted: Mon Jun 27, 2005 11:31 am Post subject: |
|
|
Voyager
Joined: 27 Mar 2002
Posts: 87
Location: Research Triangle Park, NC
|
Hello,
What is the WBI-MB version / FP level?
And what platform?
What is the error you are getting when processing 10MB message?
It would be helpful if you set the Detail Trace to On and paste the conents of the XMLT node trace.
Regards, Chenulu |
|
| Back to top |
|
|
| mqbrk_newbie |
| Posted: Mon Jun 27, 2005 1:58 pm Post subject: xml trasformation |
|
|
Acolyte
Joined: 13 Sep 2004
Posts: 70
|
chenulu,
thanks for the response . we are having WBIMB 5.0.4. I tried in different way by breaking a big xml file into a separate xml file. When I tried to Send 150 xml records through XML transformation node by looping each individual record , I am getting "STACK OVERFLOW ERROR" and if i send a single larger xml file i am getting out of memory error. Do i need to increase the heap size or something in the broker execution group. Can you help me out regarding this...
thanks
[/b] |
|
| Back to top |
|
|
| JT |
| Posted: Mon Jun 27, 2005 4:24 pm Post subject: |
|
|
Padawan
Joined: 27 Mar 2003
Posts: 1564
Location: Hartford, CT.
|
|
| Back to top |
|
|
| mgk |
| Posted: Mon Jun 27, 2005 7:19 pm Post subject: |
|
|
Padawan
Joined: 31 Jul 2003
Posts: 1642
|
Hi,
Have you tried using the ESQL PROPAGATE statement to send one message at a time down to the node?
_________________
MGK
The postings I make on this site are my own and don't necessarily represent IBM's positions, strategies or opinions. |
|
| Back to top |
|
|
| mqbrk_newbie |
| Posted: Mon Jun 27, 2005 7:35 pm Post subject: xml tranformation node! |
|
|
Acolyte
Joined: 13 Sep 2004
Posts: 70
|
I tried using propagate statement . It could able to separate 500 records , after that it blows up. And I could not combine all the messages to one message . My enironment gets cleared up when i tried to that ...
So if I set the enivronment variable for MQSI_THREAD_STACK_SIZE to 1 MB on unix , will that solve my problem. where should I set that environment variable on both windows and unix platform. Please pardon my ignorance.... |
|
| Back to top |
|
|
| JT |
| Posted: Tue Jun 28, 2005 5:18 am Post subject: |
|
|
Padawan
Joined: 27 Mar 2003
Posts: 1564
Location: Hartford, CT.
|
| Quote: |
| So if I set the enivronment variable for MQSI_THREAD_STACK_SIZE to 1 MB on unix, will that solve my problem. |
Did you click on the hilighted text/link in my post? Within the 1st paragraph it clearly states that 1MB is the default size allocated to a message flow on a UNIX platforms:
| Quote: |
The default stack size allocated to a message flow thread depends on the platform used.
Windows: Each message flow thread is allocated 2MB of stack space.
Unix: Each message flow thread is allocated 1MB of stack space.
z/OS: Each message flow thread is allocated 512 KB of downward stack space and 50 KB of upward stack space. |
And, according to the following excerpt from that same link, you need to calculate how many nodes your 500+ message segments will traverse during the course of your message flow, and multiply that number by 2KB to arrive at the approximate value for the MQSI_THREAD_STACK_SIZE setting.
| Quote: |
| In a message flow, a node typically uses about 2KB of the stack space. A typical message flow can therefore include roughly 250 nodes on z/OS, 500 nodes on UNIX platforms and 1000 nodes on Windows. This amount can be higher or lower depending on the type of nodes used and the processing they perform. |
| Quote: |
| where should I set that environment variable on both windows and unix platform |
It's a run-time parameter, so you need to set it on the same box where your brokers are running, based on previous statements I assume that's the Unix server. |
|
| Back to top |
|
|
| mqbrk_newbie |
| Posted: Tue Jun 28, 2005 5:22 am Post subject: xml transformation! |
|
|
Acolyte
Joined: 13 Sep 2004
Posts: 70
|
| How can I set the stack size to MB on both unix and windows platform. can anyone please let me know how can i set that? |
|
| Back to top |
|
|
| mqbrk_newbie |
| Posted: Tue Jun 28, 2005 5:52 am Post subject: xml transformation! |
|
|
Acolyte
Joined: 13 Sep 2004
Posts: 70
|
| sorry I did not read the last post that I got . THANKS A LOT FOR YOUR HELP. |
|
| Back to top |
|
|
| mgk |
| Posted: Tue Jun 28, 2005 11:35 am Post subject: |
|
|
Padawan
Joined: 31 Jul 2003
Posts: 1642
|
Hi,
PROPAGATE is the right way to solve this. If it is blowing up, this is for one of two reasons.
1: You are not correctly following the Large Message handing guidelines in the documentation, and are not deleting the old message properly before you PROPAGATE the new one.
2: Assuming your ESQL code is correct, you could have hit a known problem with the PROPAGATE statement not correctly freeing parsers after each useage. There is a fix, and I'm not sure which csd (if any) it is in. You should raise a PRM to get an efix if you think this is the problem.
regards
_________________
MGK
The postings I make on this site are my own and don't necessarily represent IBM's positions, strategies or opinions. |
|
| Back to top |
|
|
|
|
