| Author |
Message
|
| akir |
 Posted: Tue Jul 01, 2008 5:08 am Post subject: handling large size message greater than 500 mb in broker |
 |
|
Acolyte
Joined: 28 Jun 2007
Posts: 51
|
Hello All,
Can any one tell me how to handle a large size message greater than 500 mb in message broker. i have no idea regarding this......please provide me information on various ways to do it such that i could know how to handle large messages....
_________________
Thanks and Regards,
Akir
Tell everyone what you want to do and someone will want to help you do it.-W. Clement Stone |
|
| Back to top |
|
 |
| mrgate |
| Posted: Tue Jul 01, 2008 6:21 am Post subject: |
|
|
Centurion
Joined: 28 Feb 2007
Posts: 141
Location: India
|
Hello akir,
It is possible by increasing jvm heap size of the broker in runtime. try issuing mqsichangeproperties command.
_________________
MQSeries terrorist |
|
| Back to top |
|
|
| mrgate |
| Posted: Tue Jul 01, 2008 6:24 am Post subject: |
|
|
Centurion
Joined: 28 Feb 2007
Posts: 141
Location: India
|
sorry for posting one more query. can you please post the block diagram of integration(message flow) on which you are working, so that if possible, you can get response.
_________________
MQSeries terrorist |
|
| Back to top |
|
|
| akir |
| Posted: Tue Jul 01, 2008 6:45 am Post subject: |
|
|
Acolyte
Joined: 28 Jun 2007
Posts: 51
|
THANKS FOR REPLY MR Gate Beela ....i heard the max size that a broker can handle is 100 mb.if we increase jvm heap size will accept for 500mb.
_________________
Thanks and Regards,
Akir
Tell everyone what you want to do and someone will want to help you do it.-W. Clement Stone |
|
| Back to top |
|
|
| KeeferG |
| Posted: Tue Jul 01, 2008 7:06 am Post subject: |
|
|
Master
Joined: 15 Oct 2004
Posts: 215
Location: Basingstoke, UK
|
100MB is the maximum message size for WMQ and not Broker.
_________________
Keith Guttridge
-----------------
Using MQ since 1995 |
|
| Back to top |
|
|
| AkankshA |
| Posted: Tue Jul 01, 2008 8:50 pm Post subject: |
|
|
Grand Master
Joined: 12 Jan 2006
Posts: 1494
Location: Singapore
|
| akir wrote: |
| i heard the max size that a broker can handle is 100 mb |
gone are those days..... 
_________________
Cheers |
|
| Back to top |
|
|
| WBI_user |
| Posted: Wed Jul 02, 2008 4:47 am Post subject: |
|
|
Partisan
Joined: 07 Aug 2001
Posts: 386
|
Can you provide more information on what do you plan to do with the 500M of data
1. What version of WMB and what platform?
2. How do you plan to get it into the borker? e.g. using MQ messages with segmentation? Using a file node ?...
3. What format of is the data in (XML or mrm or TDS).i.e. What kind of parser will be invoked ?
4. Once the data is in the borker, what do you want to do with it ? i.e. what will your output looks like? Will it be bigger than the input or smaller and in what format ? What node are you using, compute node or Java?
Just use 'large message' as an argument and search the info center, you will have some information to get started.
Your question is too generic, you need to provide more information before others can help you. |
|
| Back to top |
|
|
| mqmatt |
| Posted: Wed Jul 02, 2008 5:49 am Post subject: |
|
|
Grand Master
Joined: 04 Aug 2004
Posts: 1213
Location: Hursley, UK
|
As other people have commented, the maximum message size is determined by the node and protocol being used, and not enforced by the broker infrastructure. For example, the file nodes in 6.1 work with messages (files) gigabytes in size, so long as you can dissect the contents into manageable chunks.
(This said, most node types read an entire message into memory at once, which means that your maximum message size is generally based on available memory.) |
|
| Back to top |
|
|
| KeeferG |
| Posted: Wed Jul 02, 2008 5:57 am Post subject: |
|
|
Master
Joined: 15 Oct 2004
Posts: 215
Location: Basingstoke, UK
|
I have had to adjust AIX memory management to allow a single execution group to run with 2GB of memory before to enable it to parse very large XML messages. Once we had proved that it would work I then got them to redesign the system to break up the data into smaller chunks and dirstibut the data across multiple execution groups to be rebuilt at the other end. They were much happier with the performance and resilience of the latter implementation.
Just because you can do something doesn't mean you should. I would look at the requirements in a bit more detail and break the message up if you want good performance and resilience.
_________________
Keith Guttridge
-----------------
Using MQ since 1995 |
|
| Back to top |
|
|
| kimbert |
| Posted: Mon Jul 07, 2008 4:32 am Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
|
| Back to top |
|
|
|
|
