| Author |
Message
|
| Bala |
 Posted: Mon Jul 07, 2003 10:51 am Post subject: Concatinate multiple XML messgaes into One XML. |
 |
|
Novice
Joined: 14 May 2002
Posts: 12
Location: Chennai, India
|
Hi,
I am exporting data from SAP in the form of XML using an ABAP Adapter to update an extrnal database. I use MQ for messaging and WMQI for inserting the data into the database.
The problem is that my adpater can send only 32kb of XML data at a time.
so the data from SAP is split into a number of XML messages.
I need to concatinate all these mesages into one xml and then update my external database.
could you please give me a solution.
Regards,
Bala
_________________
IBM Certified Specialist MQSeries. |
|
| Back to top |
|
 |
| inder |
| Posted: Mon Jul 07, 2003 12:02 pm Post subject: |
|
|
Apprentice
Joined: 24 Mar 2003
Posts: 49
Location: USA
|
Can you use any MQMD header field to specify for the No of messages that the actual message is split into and if all these messages arrive in the queue in sync, then you may try using a filer node to check this field and use a database table to store this part of XML as blob. All the next messages are concatenated to this field.
once you get the last message and entire xml is stored in table as a blob you can use a reset content descriptor node to convert this blob in to XML and load with highest priority in to the input queue so that the Message flow reads it first before other messages are read from queue.
This involves database storing and retrieval overhead and you need to have a text field setup in the database table to store this data.
Hope this helps
Inder |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Jul 07, 2003 2:17 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
You may also be able to be clever and use the Aggregator nodes to do aggregate your individual message segments for you.
I haven't done it, but it should be possible. |
|
| Back to top |
|
|
| Yanghui |
| Posted: Tue Jul 08, 2003 12:26 am Post subject: |
|
|
Disciple
Joined: 08 May 2002
Posts: 151
Location: Dublin, Ireland
|
Hi,
As far as I know Oracle OPG also has this 32k limition problem. The solution we chose is to use MQ segmentation mechanism, which means the problem is handled in the MQ layer. One logical message consists of many physical messages. On Oracle side, there is a common package to do message segmentation before they are sent to a MQ Q. On the MQ or WMQI side, it's transparent if you tcik "All Messags Available" on the Advanced of MQInput node.
Hope it helps.
Regards
-Yanghui |
|
| Back to top |
|
|
| Yanghui |
| Posted: Tue Jul 08, 2003 12:48 am Post subject: |
|
|
Disciple
Joined: 08 May 2002
Posts: 151
Location: Dublin, Ireland
|
Hi, JeffLowrey,
I think I must not be clever enough.  I couldn't figure out how it can work by using the Aggregators nodes.
Probably we can use the first message to trigger the whole process and use the fan-in mechanism to pick up the rest messages within same group. To me, the problem is that we never know how many physical messages within one logical one. I also couldn't figure out how we can let the Aggregators fan-in part to have a loop.
Have you thought a bit further? Thanks for sharing your ideas.
Regards
-Yanghui |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Jul 08, 2003 6:06 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| Yanghui wrote: |
I think I must not be clever enough. I couldn't figure out how it can work by using the Aggregators nodes.
Probably we can use the first message to trigger the whole process and use the fan-in mechanism to pick up the rest messages within same group. To me, the problem is that we never know how many physical messages within one logical one. I also couldn't figure out how we can let the Aggregators fan-in part to have a loop.
Have you thought a bit further? Thanks for sharing your ideas.
|
I think I'm clever enough to know that it can be done, but I'm not sure I'm clever enough to know HOW to do it. 
What I think needs to happen is something vaguely like this. Incoming messages are transformed into 'request' messages and fanned out to a process that turns around and recasts them as 'reply' messages and fans them back in. The message that marks the end of your logical group is used to send the aggregate control message that indicates that all requests have been made. The fan in process then assembles all the aggregated replies. The main difference here between a normal Aggregate process is that the requests and replies aren't doing anything real with the data - just fooling the aggregate process to work the way you want it to.
Like I said, though, I haven't ACTUALLY done this.  It should work  , but it might not  .
There are examples of using the Aggregate nodes floating around, including one in the Developing Solutions redbook. They may be a bit of help. |
|
| Back to top |
|
|
|
|
