| Author |
Message
|
| Jo_0303 |
 Posted: Wed Jun 23, 2004 3:47 am Post subject: Message Segmentation |
 |
|
Newbie
Joined: 10 May 2004
Posts: 6
|
Hi there
I'm using MQ (5.3) with a VB6.0 application. Of late, the size of messages passing through the queues has been getting bigger and bigger (>4MB). The application is also memory intensive (continous polling to read messages from the queue, using DOM parser, etc). This has created all sorts of problems.
I'm contemplating Segmentation to avoid allocating large buffer sizes to accomodate big messages. For those who have used/are using Segmentation, what are your thoughts on it? Are there any negative impacts to be expected?
I understand Segmentation shouldn't cause problems as long as " if multiple servers are processing the same queue, each server should execute similar code, so that one server never fails to find a message or segment that it expects to be there (because it had specified MQGMO_ALL_MSGS_AVAILABLE or MQGMO_ALL_SEGMENTS_AVAILABLE earlier). "
 |
|
| Back to top |
|
 |
| EddieA |
| Posted: Wed Jun 23, 2004 7:40 am Post subject: |
|
|
Jedi
Joined: 28 Jun 2001
Posts: 2453
Location: Los Angeles
|
| Quote: |
| continous polling to read messages from the queue |
That shouldn't cause your application to be any more memory intensive. Unless, of course, you've got a leaky program.  Also, MQ has a perfectly good GET with WAIT.
Segmenting your messages, either by application or by MQ, is not going to help. This is designed to reduce the size of the mesages as they are transmitted.
Your sending application still going to have to allocate a buffer large enough to hold the message. And so is the receiving application.
The "only" way this could have helped, is if the sending application could segment the messages into smaller pieces, that the receiver could process individually. But, you mention a DOM parser, which means the messages are XML, and therefore cannot be processed in "pieces".
Cheers,
_________________
Eddie Atherton
IBM Certified Solution Developer - WebSphere Message Broker V6.1
IBM Certified Solution Developer - WebSphere Message Broker V7.0 |
|
| Back to top |
|
|
| Jo_0303 |
| Posted: Thu Jun 24, 2004 2:03 am Post subject: RESOLVED - Message Segmentation |
|
|
Newbie
Joined: 10 May 2004
Posts: 6
|
Thanks a lot Eddie
Now I'm convinced segmentation wont work. I'll have to find other ways of efficiently managing memory.
 |
|
| Back to top |
|
|
| PGoodhart |
| Posted: Thu Jun 24, 2004 9:59 am Post subject: |
|
|
Master
Joined: 17 Jun 2004
Posts: 278
Location: Harrisburg PA
|
Segmentation works! You just have to have decent servers with a decent amount of memory. If you need to deal with messages larger then 100 MBs it's the best way to go.
However the DOM parser is a dog perfomance and memory wise and if you are making use of it in your program that may be hurting you. You may want to look at SAX or optimizing your transformations. Do you have the Broker? If so you can send a non-XML message through it and have it turn the message into XML. The broker is much better(quicker) at flat data to XML changes then transforming a message through the DOM.
I'd also suggest using triggering rather then GET wait or polling.
Good luck!
_________________
Patrick Goodhart
MQ Admin/Web Developer/Consultant
WebSphere Application Server Admin |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Jun 24, 2004 10:08 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| PGoodhart wrote: |
| Segmentation works! You just have to have decent servers with a decent amount of memory. If you need to deal with messages larger then 100 MBs it's the best way to go. |
I don't think you can use segmentation for messages larger than 100MBs.
I think you can only use grouping for messages larger than 100MBs.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| PGoodhart |
| Posted: Thu Jun 24, 2004 10:15 am Post subject: |
|
|
Master
Joined: 17 Jun 2004
Posts: 278
Location: Harrisburg PA
|
On 5.2 you may be right. On 5.3 it works! I had to do a proof of concept and test on it to prove it about three months ago.
_________________
Patrick Goodhart
MQ Admin/Web Developer/Consultant
WebSphere Application Server Admin |
|
| Back to top |
|
|
|
|
