| Author |
Message
|
| kirankinnu |
 Posted: Wed Aug 09, 2006 6:58 am Post subject: Performance COBOL to XML |
 |
|
Centurion
Joined: 12 Jun 2004
Posts: 128
Location: Chicago, IL
|
Hello Folks,
We r doing transformation from COBOL copybooks to XML. the message can be of size 200-300kb. Daily we might recieve around 50,000 messages. but we are not sure. Our Environment is in V6. Please advice which is the best pratice. All the operations r performed in single Compute node.
1) Whether to Use Move statements or Use lot of references when doing the transformations.
2) what If we do not validate the message, & after the transformation to xml, we use Asbitstream to convert it to bit stream and send it to output. Thatway the message does not need keep lot of references. Does this reduce the Memory usage & CPu usage. & also Is it a good practice.
Is there any other way we could reduce the memory usage as well as CPU usage.
Any Advice would be really appreciated.
Thanks |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Wed Aug 09, 2006 7:03 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Using asbitstream to convert a message tree right before an xxxOutput node is entirely wasteful. The job of the xxxOutput node is to do exactly that, and then send the data using the appropriate protocol.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| kirankinnu |
| Posted: Wed Aug 09, 2006 9:40 am Post subject: |
|
|
Centurion
Joined: 12 Jun 2004
Posts: 128
Location: Chicago, IL
|
Thanks Jeff for the prompt response.
What about the 1st question. Can anyone give some input to this, Whether to use MOVE or use(DECLARE) many references.
Thanks |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Aug 09, 2006 1:09 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| kirankinnu wrote: |
Thanks Jeff for the prompt response.
What about the 1st question. Can anyone give some input to this, Whether to use MOVE or use(DECLARE) many references.
Thanks |
I think the right answer is a mix somewhere.
Remember that even a pointer to data consumes some memory. So having a lot of them active and open could become prohibitive...
When I have a procedure being called by reference with an input and an output argument I usually declare 2 more references (1 to input and 1 to output) and let these walk the tree.
But it does make sense for some cases to use additional refences.
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| kirankinnu |
| Posted: Thu Aug 10, 2006 5:06 am Post subject: |
|
|
Centurion
Joined: 12 Jun 2004
Posts: 128
Location: Chicago, IL
|
Hello Jeff,
When I did a search on "Handling large MRM messages" in toolkkit, I found out this info. Here they even explained with an example as well. here is what they say
" Copy the body of the input message as a bit stream to a special folder in the output message. This creates a modifiable copy of the input message that is not parsed and which therefore uses a minimum amount of memory.
Avoid any inspection of the input message; this avoids the need to parse the message.
Use a loop and a reference variable to step through the message one record at a time. For each record:
Use normal transforms to build a corresponding output subtree in a second special folder.
Use the ASBITSTREAM function to generate a bit stream for the output subtree that is stored in a BitStream element, placed in the position in the tree, that corresponds to its required position in the final bit stream.
Use the DELETE statement to delete both the current input and the output record message trees when you complete their manipulation.
When you complete the processing of all records, detach the special folders so that they do not appear in the output bit stream. "
Did anyone try this way of coding before, Just making sure. If anyone did this, please provide some input to this.
Thanks |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Aug 10, 2006 5:24 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Notice that it says to create the bitstream for EACH RECORD, not for the ENTIRE message.
Also notice that it's talking about BitStream elements. This is very advanced stuff here - it's basically pre-optimizing the message tree for the Output node.
Unless your messages are actually very large (bigger than 10 meg), and your performance requirements are very tight, you probably don't need to do this.
At 200 - 300 kb, this is almost certainly a waste of your time - unless you have a very small broker machine and you need very very fast response time - which 50,000 messages a day is not.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| kirankinnu |
| Posted: Thu Aug 10, 2006 5:37 am Post subject: |
|
|
Centurion
Joined: 12 Jun 2004
Posts: 128
Location: Chicago, IL
|
Hi,
Right now, we are not exactly sure what will be the volume of messages that we will recieve. And also those estimated 50,000 messages might run as batch process. So we might need instant responses as well. I think after developing the code, we have to do performance verification and check whether it is making any difference or not. But for now we want to keep our options open.
Yes, we have multiple records in one single message. So we can create Bitstream for each record.
Thanks for the responses. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Aug 10, 2006 7:37 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Just as a heads up on message manipulation (mapping).
Our input is SAP IDOC the output is XML. Message size is anything from 300K to 2MB.
Mapping is done in ESQL.
Moving the message from the InputRoot to Environment and deleting each segment (EDI_DD40) after having processed it, we are going through the segments by reference, seemed to have yielded something around 20% improvement in processing time....
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
