| Author |
Message
|
| shaon123 |
 Posted: Tue Jul 06, 2010 12:21 am Post subject: Segmenting a Message in queue using ESQL and reassembling |
 |
|
Newbie
Joined: 01 Jun 2010
Posts: 3
|
Hi,
I have the following requirement:-
A Message of 100MB will get placed on a path and picked up by the File Input node and then this messages need to be passed through a standard queue (4MB) capacity by segmenting the message through ESQL or in any other way.After passing the message through the queue the message needs to be assembled and passed to the o/p directory through File O/p Node.
Please can anyone suggest how i can go ahead to split any message while placing the message in the queue through the ESQL and MQ properties.
Thanks in advance for the help. |
|
| Back to top |
|
 |
| kimbert |
| Posted: Tue Jul 06, 2010 1:41 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
Please can anyone suggest how i can go ahead to split any message while placing the message in the queue through the ESQL and MQ properties.
|
The documentation for FileInput and FileOutput nodes explains how to do this. Please do one of the following:
a) tell us why the faciliites in those nodes are not suitable or
b) ask a more specific question or
c) tell us what you tried, and why it didn't work. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Jul 06, 2010 1:42 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
so you are using Broker as an FTP system?
I'd look at using WMQ File Transfer Edition if this is all you are going to do.
If you are stuck using broker then... read on.
Have you thought about increasing the size of the MAXMSGL parameter of the queue(s) etc? so that you don't need to do this. Remember that whichever way you proceed you may well have to vastly increase the size of the WMQ Logfiles if you are using persistent messages (Which IMHO, you should be...)
If you really can't do this then you will have to use message grouping. This is a standard WMQ way of associating messages together.
Remember that other messages may well arrive on the destination queue in the middle of this stream of 4Mb messages. Using message Grouping is the way to bind them together.
There is some sample code in the Broker Info Centre that shows how you might send this set of data. (Search for GroupId).
You should also read up on this topic in the WMQ Documentation.
You will need to set the MQMD Version correctly to enable the GroupId part of the MQMD.
Enjoy
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Jul 06, 2010 1:48 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
So the properties of the FileInput and FileOutput will let you break the file into logical records, which can then be passed into MQ without segmenting, and reassembled at the other end.
Otherwise you can configure the MQMD in the message you pass to MQOutput to perform qmgr segmentation, by setting all of the usual properties you would set in any MQ application to allow qmgr segmentation.
Or you can perform application segmentation using a PROPAGATE loop in your flow, but you might not want to bother with that much code. |
|
| Back to top |
|
|
| shaon123 |
| Posted: Wed Jul 07, 2010 2:09 am Post subject: |
|
|
Newbie
Joined: 01 Jun 2010
Posts: 3
|
Thanks to all for the help.
We can segment the message through File i/p node.
However the problem arises if the application directly places a message in the queue which is around 50MB and the queues have a capacity of 4MB so we need to apply a segmentation of messages while placing in the queue and then reassemble them.
Please can you suggest how this is possible .I find that there is a concept of segmentation through Qmanager . Please suggest how we can do the same and where in the qmanager we need to alter the properties. |
|
| Back to top |
|
|
| Amitha |
| Posted: Fri Jul 09, 2010 6:49 am Post subject: |
|
|
Voyager
Joined: 20 Nov 2009
Posts: 80
Location: Newyork
|
| Quote: |
However the problem arises if the application directly places a message in the queue which is around 50MB and the queues have a capacity of 4MB so we need to apply a segmentation of messages while placing in the queue and then reassemble them.
Please can you suggest how this is possible .I find that there is a concept of segmentation through Qmanager . Please suggest how we can do the same and where in the qmanager we need to alter the properties. |
you can segment messages by setting the MQMD headers using esql or Java compute. Also, you can configure MQOUTPUT node to do that.
Here is the Infocenter link:
http://publib.boulder.ibm.com/infocenter/wmbhelp/v6r1m0/topic/com.ibm.etools.mft.doc/ac25720_.htm |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Fri Jul 09, 2010 8:38 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| shaon123 wrote: |
Please can you suggest how this is possible .I find that there is a concept of segmentation through Qmanager . Please suggest how we can do the same and where in the qmanager we need to alter the properties. |
As I said in my previous post, there is sample code in the infocentre that shows you how to do message grouping. You don't need to alter any QMGR properties unless your QMGr is created using all the defaults (ie Circular logging, 3 primary log files of 4mg each) Your WMQ Admins should be able to tell you how your QMGR is configured.
Beware, if you are using the default logfile size then yopu may need to re-create the QMGR but again, your WMQ Admins should be able to help you there.
Why don't you look in the infocentre, and give the sample code a try and then get back to us if you hit problems.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
|
|
