| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| Determine best value for MAXUMSGS |
« View previous topic :: View next topic » |
| Author |
Message
|
| JPDS |
 Posted: Wed Jun 23, 2010 3:28 am Post subject: Determine best value for MAXUMSGS |
 |
|
Newbie
Joined: 06 Aug 2008
Posts: 7
|
IBM recommends 100 for MAXUMSGS which would cause 2042 for applications that have larger units of work than the recommended 100.
However we have applications that put 1000s of messageI don't think there is any guarantee of not hitting maxumsgs if it was set to 100 and I would imagine it could impact performance if it started doing checkpoint processing every 50 messages on SYSTEM.CLUSTER.REPOSITORY.QUEUE.
I suppose we could increase it if we needed to do a REFRESH cluster.
Is there any way to measure the UOW sizes for applications ?
I would think any of them could fail if we reduced maxumsgs but I don't know how to test this.
IBM say
"There is logic in place to periodically check the size of the SYSTEM.CLUSTER.REPOSITORY.QUEUE and if it has reached 5000 or half the queue manager's MAXUMSGS, whichever is the lower, to compress this queue's contents into checkpoint records just as is carried out at channel initiator shutdown. The process that carries out this check runs periodically, so the queue depth may creep slightly above this number before processing takes place".
IBM recommend not making it too large a value, they say 100, but I am concerned that this is not realistic and would impact many existing applications. |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Wed Jun 23, 2010 4:06 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| However we have applications that put 1000s of messages ... |
1000s of messages in a single unit of work?
Most applications use one message (like a payroll time-card) for a single unit of work (one pay check).
What type of application? Why so many messages in a single UofW?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| JPDS |
| Posted: Wed Jun 23, 2010 8:15 am Post subject: MAXUMSGS |
|
|
Newbie
Joined: 06 Aug 2008
Posts: 7
|
| These messages are related to financial transactions, there may be thousands put in seconds. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Jun 23, 2010 8:19 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| These messages are related to financial transactions, there may be thousands put in seconds. |
Yes, but do the 1000s of messages all relate to one business transaction? Or are they all individual transactions?
Transaction = unit of work = syncpoint.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Jun 23, 2010 8:19 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
thousands of messages put in one single second doesn't mean that all messages are in the same logical unit of work.
Are there really thousands of separate transactions for the same unitary entity that all need to be handled as a logical unit of work - that is committed or rolled back as a whole rather than individually? |
|
| Back to top |
|
|
| JPDS |
| Posted: Wed Jul 07, 2010 9:01 am Post subject: |
|
|
Newbie
Joined: 06 Aug 2008
Posts: 7
|
| There will be a mixture - some applications are coded to do a big UOW where it is not necessary because they are actually individual transactions that can be changed to commit more often. I guess my question is how to identify these - is there a way to identify the larger Units Of Work so I can go back to the applications support to ask if they are necessary ? I don't want to just reduce the value of maxumsgs on the qmgr in case a number of apps then fail. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Jul 07, 2010 12:36 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
How many thousands? 2,000? 4,000? 20,000? 100,000?
It's usually badly-written apps that create large (greater than 100 message) UofWs. Each time, it was the developer misunderstanding what a transaction is.
However, since this post is in the mainframe forum, I'm going to speculate that you can set maxumsgs to a substantially higher value with little impact. Ask the developers if 1000 (or 5000) msgs/uofw will meet their needs. Lots of horsepower and other resources in z/OS. I'd be far more concerned if this was on a midrange box.
Since it's z/OS, I'd suspect that your z/OS sysprog can turn capture of the appropriate SMF data for the qmgr in question.
Again, what type of applications? What type of transactions?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| JPDS |
| Posted: Thu Jul 08, 2010 1:50 am Post subject: |
|
|
Newbie
Joined: 06 Aug 2008
Posts: 7
|
Our setting for most queue managers is 15000 and this needs to be increased for a particular application for peak processing periods in which case I put it up and then back down after the batch finishes.
This particular application will be fixed.
After that I want to know whether to try and reduce aiming for IBMs recommendation of 100.
Is this excerise worthwhile ?
I would like to flush out other bad apps but the crucial thing is not to break any production applications by making it too low before applications are changed.
So if the SMF can tell me the sizes of UOW that I can tie to apps that could be very useful - is this the case ?
If the only reason for having it low is to deal with looping applications it may be that we should just leave as is.
I also know that doing more frequent commits can affect performance badly and also that IBM code itself in MQ puts more than 100 messages on SYSTEM.CLUSTER.REPOSITORY.QUEUE without committing so there are conflicting steers. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Jul 08, 2010 4:56 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| I also know that doing more frequent commits can affect performance badly... |
Commits (and nearly all other app design issues) should be based on the business requirement. On z, with its usually abundant resources and WLM, performance is this is not so much an issue for any specific application.
Using a payroll or inventory inquiry app as examples: one time-card = one paycheck = one transaction = one commit; one inventory inquiry = one response = one transaction = one commit.
Conversely (and a horrible example), updating every inventory record in the inventory DB should not be one transaction and one commit. Why? It is unlikely that all inventory records in the DB are related.
| Quote: |
| and also that IBM code itself in MQ puts more than 100 messages on SYSTEM.CLUSTER.REPOSITORY.QUEUE without committing so there are conflicting steers. |
IBM code does this on behalf of your applications, as your app puts messages in a UofW.
Yes, I believe SMF stats and accounting data can give you this kind of information.
[edit] The purpose of many of the object attributes is to put some constraints on run-away applications: maxqdepth, maxmsglength, maxhandles, maxchannels, and others. And most can be altered dynamically.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| JPDS |
| Posted: Thu Jul 08, 2010 6:28 am Post subject: |
|
|
Newbie
Joined: 06 Aug 2008
Posts: 7
|
Do you mean the Bytes if UOWID field in the MQMQUEUE record ?
I ran a test report but that field was always zero, I would expect 1 at least for each transaction - but this may be an unrepresentative sample.
If the only reason for haveing low maxumsgs is to stop looping applications I believe we should not introduce risk to the environment by lowering it.
It could be argued to make it 9s as MQ is not a monitor for detecting looping transactions. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Jul 08, 2010 6:49 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| Do you mean the Bytes if UOWID field in the MQMQUEUE record ? |
I don't have the SMF manuals right now...
| Quote: |
| I ran a test report but that field was always zero, I would expect 1 at least for each transaction - but this may be an unrepresentative sample. |
I'd expect that SMF would have captured uowid, but you will need to do more research.
| Quote: |
| If the only reason for haveing low maxumsgs is to stop looping applications I believe we should not introduce risk to the environment by lowering it. |
Nearly all applications loop (get a message, process it, produce output, repeat). Detecting a misbehaving looping application takes some calculations and assumptions.
For example: If there are 1 million items in inventory, AND the app is producing a report of all items in inventory, then the loop will be repeated 1 million times. If the app developer (mis)designed the app to do this in one UofW, smack the developer.
| Quote: |
| It could be argued to make it 9s as MQ is not a monitor for detecting looping transactions. |
In no case, is setting maxumsgs (or maxmsglength or maxqdepth or ...) intended to detect; rather, it is designed to restrict the abuse of limited resources by an application. (If you consider the app receiving a fail CompletionCode as detecting a loop, then I guess it does.)
Notice that 999,999,999 is max for many of these constraints. Note also that the as-shipped values are dramatically lower. You can change these to meet the requirements of your organization and its apps.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Jul 08, 2010 1:47 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| JPDS wrote: |
Our setting for most queue managers is 15000 and this needs to be increased for a particular application for peak processing periods in which case I put it up and then back down after the batch finishes.
This particular application will be fixed.
After that I want to know whether to try and reduce aiming for IBMs recommendation of 100.
Is this excerise worthwhile ?
I would like to flush out other bad apps but the crucial thing is not to break any production applications by making it too low before applications are changed.
So if the SMF can tell me the sizes of UOW that I can tie to apps that could be very useful - is this the case ?
If the only reason for having it low is to deal with looping applications it may be that we should just leave as is.
I also know that doing more frequent commits can affect performance badly and also that IBM code itself in MQ puts more than 100 messages on SYSTEM.CLUSTER.REPOSITORY.QUEUE without committing so there are conflicting steers. |
This number of 100 uncommitted message for the qmgr sounds very suspect to me as distributed qmgrs come with a default of 10,000. I believe what might have been the intent of IBM there was to suggest to set a max size of 100 messages per UOW for the apps creating a UOW.
More discrete packets of a lesser size may actually favor throughput. It all depends on the context and what else is happening in the qmgr and the network...
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Jul 08, 2010 2:07 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| IBM recommends 100 for MAXUMSGS |
Please post the URL or other reference where you found this recommendation.
[edit]
OK. Here's what I found in the z/OS WMQ Concepts and Planning guide:
-------------------------------------------------------------------------
Syncpoints
One of the roles of the queue manager is syncpoint control within an application. An application constructs a unit of work containing any number of MQPUT or MQGET calls terminated with an MQCMIT call. However, as the number of MQPUT or MQGET calls within the scope of one MQCMIT increases, the performance cost of the commit increases significantly.
You can limit the number of messages within any single syncpoint by using the MAXUMSGS queue manager attribute. You should not normally exceed 100 MQPUTs within a single MQCMIT.
Set a fairly low MAXUMSGS value, such as 100, to avoid performance problems, and to protect against looping applications. If you have a need for a larger value, change MAXUMSGS temporarily while the application that requires the larger value is run, rather than using a permanently high value. However, be aware that reducing the value of MAXUMSGS may cause problems to existing applications and queue manager processes such as clustering if they are already using a higher
value.
-----------------------------------------------------------------------------
As with most IBM doc, there are things you need to consider before you make a decision and then change some setting or other. You should clearly establish a goal when you make any configuration change. You should capture performance (and other) data before and after you make any change; then compare results to ensure that you achieved whatever goal you established.
As delivered from IBM, WMQ has settings and initial values that may not meet your organizations needs. As-shipped, WMQ is typically configured just fine for initial testing.
So, 100 is a good number for maxumsgs - IF it makes sense for your organization. If not, change it. 15,000 is a good number if it makes sense for your organization - IF not, change it.
Be very clear on your understanding of what impacts your choices will have on the qmgr, the o/s, other applications, etc..
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Jul 08, 2010 2:33 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| Quote: |
You can limit the number of messages within any single syncpoint by using the MAXUMSGS queue manager attribute. You should not normally exceed 100 MQPUTs within a single MQCMIT.
Set a fairly low MAXUMSGS value, such as 100, to avoid performance problems, and to protect against looping applications. If you have a need for a larger value, change MAXUMSGS temporarily while the application that requires the larger value is run, rather than using a permanently high value. However, be aware that reducing the value of MAXUMSGS may cause problems to existing applications and queue manager processes such as clustering if they are already using a higher
value. |
Bruce, I have my problems with the understanding of this low value.
My understanding is that setting the value to 100 forces the max uncommitted messages for the qmgr at any moment to be under less or equal to 100. This, across all existing units of work. So if you run 10 concurrent applications with 100 messages per UOW each, this will create some trouble and you should bump it up to 1000 in the same spirit right?
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Jul 08, 2010 3:27 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
I'm pretty sure MAXUMSGS is for a single UOW, not for any and all UOWs combined.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
|
|
|
  |
Goto page 1, 2 Next |
Page 1 of 2 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
