| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Large messages and MQ segmentation |
« View previous topic :: View next topic » |
| Author |
Message
|
| goffinf |
 Posted: Wed Jun 20, 2012 12:39 am Post subject: Large messages and MQ segmentation |
 |
|
Chevalier
Joined: 05 Nov 2005
Posts: 401
|
MBv6.1.0.9 / MQv7.0 / AIX
I have a flow that reads emails from our email server and eventually stores both the body text and any attachments in a content management application. At at least one point in this processing, the messages are written to an MQ queue.
The question is, sometimes the attachments can run to between 2 and 10MB. I can set the max message size to accommodate that, but at what point should I consider segmented messages and if I did, what would be the impact to my message broker flows that write to and read from that queue ?
The number of emails per day is modest, something around 7000, although that could pick up I suppose.
Fraser. |
|
| Back to top |
|
 |
| Esa |
| Posted: Wed Jun 20, 2012 3:00 am Post subject: Re: Large messages and MQ segmentation |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| goffinf wrote: |
The question is, sometimes the attachments can run to between 2 and 10MB. I can set the max message size to accommodate that, but at what point should I consider segmented messages and if I did, what would be the impact to my message broker flows that write to and read from that queue ?
|
I would say that you need to consider segmenting the message only when you cannot set the max message size of the queue large enough to accomodate the single message. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Jun 20, 2012 5:50 am Post subject: Re: Large messages and MQ segmentation |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| Esa wrote: |
| goffinf wrote: |
The question is, sometimes the attachments can run to between 2 and 10MB. I can set the max message size to accommodate that, but at what point should I consider segmented messages and if I did, what would be the impact to my message broker flows that write to and read from that queue ?
|
I would say that you need to consider segmenting the message only when you cannot set the max message size of the queue large enough to accomodate the single message. |
Sorry but there are other considerations here that might not have been expressed, like the ultimate destination and the quality of the network.
I'd rather send more messages with a lower size through a network where the "noise level" is high and the channel might initiate a rollback because the CRC does not match... I'd also reduce the batch size of the channel...
This being said, if the message does not leave the box, go for the max size (currently 100MB)... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Esa |
| Posted: Wed Jun 20, 2012 5:59 am Post subject: Re: Large messages and MQ segmentation |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| fjb_saper wrote: |
| Esa wrote: |
| goffinf wrote: |
The question is, sometimes the attachments can run to between 2 and 10MB. I can set the max message size to accommodate that, but at what point should I consider segmented messages and if I did, what would be the impact to my message broker flows that write to and read from that queue ?
|
I would say that you need to consider segmenting the message only when you cannot set the max message size of the queue large enough to accomodate the single message. |
Sorry but there are other considerations here that might not have been expressed, like the ultimate destination and the quality of the network.
I'd rather send more messages with a lower size through a network where the "noise level" is high and the channel might initiate a rollback because the CRC does not match... I'd also reduce the batch size of the channel...
This being said, if the message does not leave the box, go for the max size (currently 100MB)... |
I agree with you. But from Message Broker point of view it's perhaps sufficient just to set the segmentation allowed flag? And leave the rest for MQ admins? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Jun 20, 2012 6:04 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
That'll work for outbound messages. For inbound messages I'd review the relevant options on the MQInput node... like (sorry from memory) entire logical message, all messages in group etc...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| goffinf |
| Posted: Wed Jun 20, 2012 7:00 am Post subject: Re: Large messages and MQ segmentation |
|
|
Chevalier
Joined: 05 Nov 2005
Posts: 401
|
| Esa wrote: |
I would say that you need to consider segmenting the message only when you cannot set the max message size of the queue large enough to accomodate the single message. |
OK.
| fjb_saper wrote: |
Sorry but there are other considerations here ...
This being said, if the message does not leave the box, go for the max size (currently 100MB)... |
The message won't leave the box in this particular case. I'll be reading from the mail server and writing to a local queue in one flow, then reading from that queue in another flow and reformatting the message for the Content Mgt app.
| Esa wrote: |
I agree with you. But from Message Broker point of view it's perhaps sufficient just to set the segmentation allowed flag? And leave the rest for MQ admins? |
Sounds like a good idea
| fjb_saper wrote: |
That'll work for outbound messages. For inbound messages I'd review the relevant options on the MQInput node |
Not 100% sure I understand that, but I'll take a look at what the InfoCentre has to say and see if I can see where you're going.
Thanks for both sets of comments |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
