| Author |
Message
|
| Sam Uppu |
 Posted: Sun Feb 22, 2009 5:52 pm Post subject: MQ segmentation setup |
 |
|
Yatiri
Joined: 11 Nov 2008
Posts: 610
|
Hi Guys,
We are using MQ V.6.0 on SunOS.
We need to handle messages of more than 1 GB size. As MQ can handle max msg length of 100 MB at a time, we want to use MQ segmentation(segmentation and reassembly by QMgr) to achieve this.
The below information what we found in IBM manual and what we are following:
| Quote: |
Segmentation and reassembly by queue manager
This is the simplest scenario, in which one application puts a message to be retrieved by another. The message may be large: not too large for either the putting or the getting application to handle in a single buffer, but possibly too large for the queue manager or a queue on which the message is to be put.
The only changes necessary for these applications are for the putting application to authorize the queue manager to perform segmentation if necessary,
PMO.Options = (existing options)
MQPUT MD.MsgFlags = MQMF_SEGMENTATION_ALLOWED
and for the getting application to ask the queue manager to reassemble the message if it has been segmented:
GMO.Options = MQGMO_COMPLETE_MSG | (existing options)
MQGET
The application buffer must be large enough to contain the reassembled message (unless the MQGMO_ACCEPT_TRUNCATED_MSG option is included). |
Here is our scenario:
The application is connecting to a QMgr - QM1 via SVRCONN channel. The source application(Java) is going to use the above mentioned parameters in the PMO(put message options) in side their code and on the destination end in the GMO(get message options).
When the source application is trying to connect to the QMgr via SVRCONN channel and while placing the message(1GB) we are getting an error saying that THE MESSAGE IS TOO LARGE FOR THE CHANNEL.
As MQ version 6 can handle the max msg length of 100 MB, I can only increase the SVRCONN channel's maxmsglength parameter to 100MB. If I use the maxmessagelength of SVRCONN channel to be 100MB and all other objects like QMgr, queue etc to be 4MB, then if I send a message of 80 MB, on the queue I can see the message which is divided into smaller chunks of 4MB each. This is fine.
As I am going to get messages which are of 1GB each, how to handle this?. Please suggest me in this regard.
Thanks much. |
|
| Back to top |
|
 |
| Mr Butcher |
| Posted: Sun Feb 22, 2009 11:06 pm Post subject: |
|
|
Padawan
Joined: 23 May 2005
Posts: 1716
|
do you use a client channel table? afaik without a client channel table only 4mb on srvconn is possible.
_________________
Regards, Butcher |
|
| Back to top |
|
|
| exerk |
| Posted: Mon Feb 23, 2009 1:20 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
If you are going to move 'messages' of that size, have you considered WebSphere MQ File Transfer Edition (FTE) ?
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| Sam Uppu |
| Posted: Mon Feb 23, 2009 8:53 am Post subject: |
|
|
Yatiri
Joined: 11 Nov 2008
Posts: 610
|
| Mr Butcher wrote: |
| do you use a client channel table? afaik without a client channel table only 4mb on srvconn is possible. |
The application is not using CHLTAB to connect to the QMgr, The application is connecting via SYSTEM.DEF.SVRCONN channel.
Thanks for your valuable suggestions. |
|
| Back to top |
|
|
| Sam Uppu |
| Posted: Mon Feb 23, 2009 8:56 am Post subject: |
|
|
Yatiri
Joined: 11 Nov 2008
Posts: 610
|
| exerk wrote: |
| If you are going to move 'messages' of that size, have you considered WebSphere MQ File Transfer Edition (FTE) ? |
We dont want to use File transfer as 99% of the time our messages are less than 1 MB and remaining 1% of the time the message can be upto 1GB. If the message is large(1GB), then the application should use MQ segmentation.
Thanks. |
|
| Back to top |
|
|
| Toronto_MQ |
| Posted: Mon Feb 23, 2009 11:26 am Post subject: |
|
|
Master
Joined: 10 Jul 2002
Posts: 263
Location: read my name
|
| Sam Uppu wrote: |
| Mr Butcher wrote: |
| do you use a client channel table? afaik without a client channel table only 4mb on srvconn is possible. |
The application is not using CHLTAB to connect to the QMgr, The application is connecting via SYSTEM.DEF.SVRCONN channel.
Thanks for your valuable suggestions. |
You can still use a channel table to connect using SYSTEM.DEF.SVRCONN. I assume by your response then you are either using the MQSERVER variable or an MQCONNX.
On a side note, you shouldn't use the SYSTEM.* objects for application use, but that's a whole other thread. |
|
| Back to top |
|
|
| Sam Uppu |
| Posted: Tue Feb 24, 2009 4:26 pm Post subject: |
|
|
Yatiri
Joined: 11 Nov 2008
Posts: 610
|
| Mr Butcher wrote: |
| do you use a client channel table? afaik without a client channel table only 4mb on srvconn is possible. |
You meant to say when we use channel table, we can implement segmentation of messages which are more than 100MB?. With SVRCONN channel, it is not possible. Please correct me?.
Thanks. |
|
| Back to top |
|
|
| Toronto_MQ |
| Posted: Thu Feb 26, 2009 11:49 am Post subject: |
|
|
Master
Joined: 10 Jul 2002
Posts: 263
Location: read my name
|
| Sam Uppu wrote: |
| Mr Butcher wrote: |
| do you use a client channel table? afaik without a client channel table only 4mb on srvconn is possible. |
You meant to say when we use channel table, we can implement segmentation of messages which are more than 100MB?. With SVRCONN channel, it is not possible. Please correct me?.
Thanks. |
Both use a SVRCONN channel. Only the implemenatation is different.
When you create the client channel table (by defining client conn channels), you will need to define the client conns as having a maxmsgl of 100MB. These settings will carry into the table.
You may want to read up in the doc obout client channel tables as I'm not sure you really have a grasp of it. |
|
| Back to top |
|
|
| MQEnthu |
| Posted: Thu Feb 26, 2009 9:18 pm Post subject: |
|
|
Partisan
Joined: 06 Oct 2008
Posts: 329
Location: India
|
| Toronto_MQ wrote: |
| When you create the client channel table (by defining client conn channels), you will need to define the client conns as having a maxmsgl of 100MB. These settings will carry into the table. |
There is option to specify the Max. message length on server connection channel as well where you can specify it to be 100MB...then how will it differ using the client channel table or using server conn channel alone as per the message size is concern??
_________________
-----------------------------------------------
It is good to remember the past,
but don't let past capture your future |
|
| Back to top |
|
|
| vol |
| Posted: Thu Feb 26, 2009 10:21 pm Post subject: |
|
|
Acolyte
Joined: 01 Feb 2009
Posts: 69
|
Using a client channel table means that you create a CLNTCONN channel and so can increase the MAXMSGL attribute of the channel. Using MQSERVER the MAXMSGL of the client channel is fixed at 4Mb regardless of the MAXMSGL attribute of the SVRCONN.
With regard to the original subject of the post, you can put msgs of any size into a queue and the qmgr will perform automatic segmentation to MAXMSGL; I am a little hazy as to which MAXMSGL is used if the channel and queue are different. However, with the MQGMO_COMPLETE_MSG option you can only get complete msgs up to 100Mb in size, so your getting app will have to get the individual segments of the msgs and reassemble the complete msg outside WMQ. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Feb 27, 2009 7:31 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| vol wrote: |
| Using MQSERVER the MAXMSGL of the client channel is fixed at 4Mb regardless of the MAXMSGL attribute of the SVRCONN. |
Up until MQ 7.0, where this restriction is finally gone. In MQ 7.0, you can send 100 MB messages using MQSERVER.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Sam Uppu |
| Posted: Tue Mar 03, 2009 6:31 am Post subject: |
|
|
Yatiri
Joined: 11 Nov 2008
Posts: 610
|
| PeterPotkay wrote: |
| vol wrote: |
| Using MQSERVER the MAXMSGL of the client channel is fixed at 4Mb regardless of the MAXMSGL attribute of the SVRCONN. |
Up until MQ 7.0, where this restriction is finally gone. In MQ 7.0, you can send 100 MB messages using MQSERVER. |
We are on MQ version 6 and we are able to send the message to the Qmgr via SVRCONN channel if we set the MaxMessageLength attribute to 100MB. If the message is <=100 MB, I can do segmentation(QMgr segmentation).
Now the question is how to allow a message > 100 MB onto the queue manager via SVRCONN channel or channel table?. The QMgr will do the segmentation once the message comes onto the QMgr.
I believe the channel table and SVRCONN channel functionality is one and the same except the advantage of using channel table is useful in implementing SSL which is not possible with SVRCONN channel. Correct me if I am wrong.
Thanks for your thoughts. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Mar 03, 2009 6:40 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Sam Uppu wrote: |
| I believe the channel table and SVRCONN channel functionality is one and the same except the advantage of using channel table is useful in implementing SSL which is not possible with SVRCONN channel. Correct me if I am wrong. |
They're not one and the same.Though I agree both methods employ a SVRCONN channel.
The channel table has a number of advantages. Using SSL is one, allowing for client failover in the event of error is another, putting large messages from client is a third.
Check the Clients manual for full details.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Sam Uppu |
| Posted: Tue Mar 03, 2009 8:49 am Post subject: |
|
|
Yatiri
Joined: 11 Nov 2008
Posts: 610
|
| Vitor wrote: |
| Sam Uppu wrote: |
| I believe the channel table and SVRCONN channel functionality is one and the same except the advantage of using channel table is useful in implementing SSL which is not possible with SVRCONN channel. Correct me if I am wrong. |
They're not one and the same.Though I agree both methods employ a SVRCONN channel.
The channel table has a number of advantages. Using SSL is one, allowing for client failover in the event of error is another, putting large messages from client is a third.
Check the Clients manual for full details. |
| Quote: |
| putting large messages from client is a third. |
Here you mean larger than 100 MB?. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Mar 03, 2009 9:13 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Sam Uppu wrote: |
| Here you mean larger than 100 MB?. |
No, here I mean larger than 4Mb on WMQv6 or below.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
