| Author |
Message
|
| Nandinii |
 Posted: Fri Nov 17, 2017 7:24 am Post subject: Issue with large XML message |
 |
|
Newbie
Joined: 17 Nov 2017
Posts: 3
Location: Texas, USA
|
Hi Friends,
I am facing issue with following scenario. I am getting large XML message and i need to insert it into DB. I am having around 3000 tags in my XML which needs to insert into DB. I created a while loop and inserting my 3000 records per each message into DB. But my execution group is getting crashed ?
I came to know I can use LASTMOVE and MOVE and change reference and DELETE previous reference which can fix this issue. Is there any other way of doing that with out using reference.
Your help is greatly appreciated,
Thanks,
Nandinii |
|
| Back to top |
|
 |
| timber |
| Posted: Fri Nov 17, 2017 2:55 pm Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
| Quote: |
| I came to know I can use LASTMOVE and MOVE and change reference and DELETE previous reference which can fix this issue. Is there any other way of doing that with out using reference. |
Not that I know of. If your input XML is very large, you need to design your message flow to manage memory carefully. Why don't you want to use references? |
|
| Back to top |
|
|
| souciance |
| Posted: Fri Nov 17, 2017 4:00 pm Post subject: Re: Issue with large XML message |
|
|
Disciple
Joined: 29 Jun 2010
Posts: 169
|
| Nandinii wrote: |
Hi Friends,
I am facing issue with following scenario. I am getting large XML message and i need to insert it into DB. I am having around 3000 tags in my XML which needs to insert into DB. I created a while loop and inserting my 3000 records per each message into DB. But my execution group is getting crashed ?
I came to know I can use LASTMOVE and MOVE and change reference and DELETE previous reference which can fix this issue. Is there any other way of doing that with out using reference.
Your help is greatly appreciated,
Thanks,
Nandinii |
Large message means you need to stay away from parsing the entire message, instead parse as much as you need. Hence, you need to use references. |
|
| Back to top |
|
|
| Nandinii |
| Posted: Mon Nov 20, 2017 7:12 am Post subject: |
|
|
Newbie
Joined: 17 Nov 2017
Posts: 3
Location: Texas, USA
|
| timber wrote: |
| Quote: |
| I came to know I can use LASTMOVE and MOVE and change reference and DELETE previous reference which can fix this issue. Is there any other way of doing that with out using reference. |
Not that I know of. If your input XML is very large, you need to design your message flow to manage memory carefully. Why don't you want to use references? |
Hi,
I want to check if there is any other way of parsing message with out using reference.
Thanks,
Nandinii |
|
| Back to top |
|
|
| timber |
| Posted: Mon Nov 20, 2017 8:37 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
| Quote: |
| I want to check if there is any other way of parsing message with out using reference. |
Yes, you said that. But why do you want to avoid using references? |
|
| Back to top |
|
|
| abhi_thri |
| Posted: Tue Nov 21, 2017 2:00 am Post subject: |
|
|
Knight
Joined: 17 Jul 2017
Posts: 516
Location: UK
|
Hi Nandinii...when dealing with repeatable structures reference is the efficient way to access it. Not sure how you are using Move/Lastmove in regards to the While loop but have you considered using a For loop instead which may be a more cleaner approach.
Also, I suppose the broker is creating heap/core dumps when the EG crashes? if so it is worth looking whether it is reporting any specific issue, eg:- OOM(out of memory) errors, if so you need to tune the heap size accordingly. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Tue Nov 21, 2017 5:24 pm Post subject: Re: Issue with large XML message |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| Nandinii wrote: |
| I came to know I can use LASTMOVE and MOVE and change reference and DELETE previous reference which can fix this issue. Is there any other way of doing that with out using reference. |
Please note the difference between "reference" and "reference variable". It is just about impossible to do anything at all to a message tree without using references.
However, it is easy to access even very large messages without using reference variables; just replace the variables with whatever references they would otherwise resolve to.
Reference variables can significantly improve the performance associated with message tree navigation, but should not necessarily have an effect on memory usage due to message parsing.
Without knowing why the EG is crashing, or what the message flow is doing, or why it shouldn't use reference variables, ... good luck. |
|
| Back to top |
|
|
|
|
