| Author |
Message
|
| giorginus80 |
 Posted: Fri Jan 02, 2009 6:53 am Post subject: Aggregation error on oversized split |
 |
|
Centurion
Joined: 08 Jul 2008
Posts: 124
Location: Rome, Italy
|
Hello,
I have a node esql that split an MRM message (the message inside have many occurence of a type).
All works good, very nice node, I split the message , I do my work on single split, and I reaggregate all the message to the end. Till today all works good, since I tried with a big Message with more than 6000 split. Is there any limit with aggregation node? I have the message going on backout queue if I have too much request. |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Fri Jan 02, 2009 7:52 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Check the default max queue depth on SYSTEM.BROKER.AGG* queues.
Make sure the maxqdepth will accommodate what you are trying to do.
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Jan 02, 2009 7:53 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Also you may feel it useful to increase the broker EG heap size to accomodate that many messages.
It's not clear why the message is going to the backout queue - there should hopefully be some error message that explains it. That message should better help you resolve this. |
|
| Back to top |
|
|
| giorginus80 |
| Posted: Fri Jan 02, 2009 8:46 am Post subject: |
|
|
Centurion
Joined: 08 Jul 2008
Posts: 124
Location: Rome, Italy
|
| fjb_saper wrote: |
Check the default max queue depth on SYSTEM.BROKER.AGG* queues.
Make sure the maxqdepth will accommodate what you are trying to do.
Enjoy |
I checked this, it's set to the maximum value, may be another value on environment? I use WMB 6.1 on linux |
|
| Back to top |
|
|
| giorginus80 |
| Posted: Fri Jan 02, 2009 8:48 am Post subject: |
|
|
Centurion
Joined: 08 Jul 2008
Posts: 124
Location: Rome, Italy
|
| mqjeff wrote: |
Also you may feel it useful to increase the broker EG heap size to accomodate that many messages.
It's not clear why the message is going to the backout queue - there should hopefully be some error message that explains it. That message should better help you resolve this. |
How can I do this? |
|
| Back to top |
|
|
| giorginus80 |
| Posted: Fri Jan 02, 2009 9:58 am Post subject: |
|
|
Centurion
Joined: 08 Jul 2008
Posts: 124
Location: Rome, Italy
|
| fjb_saper wrote: |
Check the default max queue depth on SYSTEM.BROKER.AGG* queues.
Make sure the maxqdepth will accommodate what you are trying to do.
Enjoy |
Sorry if I ask you, but when you say SYSTEM.BROKER.AGG* queues, you don't mean the queues defined in the node of the flow? The queues System.broker.agg* are important for the aggregation flow? I mean the messages pass throw this queues too? Anyways these queues the max queue depth for default is 100000. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri Jan 02, 2009 2:16 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
With max depth of 100000 on system.broker.agg* queues you should be covered. Do you run circular or linear logging? How much space (kB, MB) will those 6000 messages need? What is your logfile size and your primary and secondary number of files? (see qm.ini)
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| giorginus80 |
| Posted: Mon Jan 05, 2009 1:08 am Post subject: |
|
|
Centurion
Joined: 08 Jul 2008
Posts: 124
Location: Rome, Italy
|
| fjb_saper wrote: |
| With max depth of 100000 on system.broker.agg* queues you should be covered. Do you run circular or linear logging? How much space (kB, MB) will those 6000 messages need? What is your logfile size and your primary and secondary number of files? (see qm.ini) |
Sorry If I answer today I was out. Now I check this values
The logtype is circular in qm.ini I didn't check the space when the flow is running. Now I try it again. another thing I saw that on system.broker.agg* queues depth is 1000000 but on SYSTEM.BROKER.CONFIG.REPLY, SYSTEM.BROKER.CONTROL.QUEUE, SYSTEM.BROKER.ADMIN.REPLY is 5000. These system queues are used from the node aggregate?
Last edited by giorginus80 on Wed Jan 07, 2009 12:59 am; edited 1 time in total |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Jan 05, 2009 11:34 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
AFAIK only SYSTEM.BROKER.AGG* queues are used for aggregation.
However you have to factor in the 6000 messages with the max uncommitted (default 10,000) and the log space (see number of prim+sec logs * log file size * 2)
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| giorginus80 |
| Posted: Wed Jan 07, 2009 12:56 am Post subject: |
|
|
Centurion
Joined: 08 Jul 2008
Posts: 124
Location: Rome, Italy
|
| fjb_saper wrote: |
AFAIK only SYSTEM.BROKER.AGG* queues are used for aggregation.
However you have to factor in the 6000 messages with the max uncommitted (default 10,000) and the log space (see number of prim+sec logs * log file size * 2)
Have fun |
Here is my qm.ini
| Code: |
#*******************************************************************#
ExitPath:
ExitsDefaultPath=/var/mqm/exits/
ExitsDefaultPath64=/var/mqm/exits64/
#* *#
#* *#
Log:
LogPrimaryFiles=3
LogSecondaryFiles=2
LogFilePages=1024
LogType=CIRCULAR
LogBufferPages=0
LogPath=/var/mqm/log/BROKER_QM/
LogWriteIntegrity=TripleWrite
Service:
Name=AuthorizationService
EntryPoints=13
ServiceComponent:
Service=AuthorizationService
Name=MQSeries.UNIX.auth.service
Module=/opt/mqm/lib/amqzfu
ComponentDataSize=0
|
|
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Jan 07, 2009 3:06 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
So you have a max of 20 MB in log space. Is this sufficient with the rest of the qmgr's load?
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| giorginus80 |
| Posted: Wed Jan 07, 2009 3:08 am Post subject: |
|
|
Centurion
Joined: 08 Jul 2008
Posts: 124
Location: Rome, Italy
|
| fjb_saper wrote: |
| So you have a max of 20 MB in log space. Is this sufficient with the rest of the qmgr's load? |
May be isn't sufficient, how can I change these values? is it sufficient stop mq and change qm.ini? Which are the best values?
thanks in advance |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Jan 07, 2009 3:16 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| giorginus80 wrote: |
| fjb_saper wrote: |
| So you have a max of 20 MB in log space. Is this sufficient with the rest of the qmgr's load? |
May be isn't sufficient, how can I change these values? is it sufficient stop mq and change qm.ini? Which are the best values?
thanks in advance |
You can change the number of log files.
Setting the LogSecondaryFiles=38 will allow you to get a lot more log space but will only use the physical space on the disk when you need it. You better have 164 MB available though...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
