| Author |
Message
|
| shashivarungupta |
 Posted: Tue Apr 05, 2016 1:39 pm Post subject: Additional Instances at Node/Mflow Level? |
 |
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
Hi,
As per the given details in this page of IBM Info Center, I did some testing related to threads/instances by setting additional Instances at Node and Message Flow level of a broker application.
The results of tests were somehow misleading and confusing related to the facts given in the Info center.
1. Number of Input Nodes (count = 1) in the Message Flow (count = 1) of the deployed message broker application.
2. Deployed application with setting 'Start additional instances when flow starts' (configurable property), Message flow level, at the BAR file.
3. Deployed application (bar file) with 'Pool Associated with Node' in an execution group of a local Broker via toolkit.
4. If additional instances at input node level set to 50 (with 'Pool Associated with Node') and additional instances at Message flow level is set to 256, total instances are shown 257. Matter of fact, should expect larger value than 257?
5. If additional instances at input node level set to 300 (with 'Pool Associated with Node') and additional instances at Message flow level is set to 256, total instances are shown 257. Matter of fact, I was still able to deploy the bar file successfully with these settings and values. Total instances count was still misleading.
Questions:
(a): In points 4 and 5, Message Broker would internally choose some precedence between Addtional instances pool setting (i.e. Pool Associate with Node and/or Pool Associated with Message Flow)? As mentioned above, while deployment of the bar file following 'Pool Associated with Node' was used.
(b): If broker doesn't do magical selection of 'additional instances pool' internally than what value is expected in scenarios mentioned in points 4 and 5, above?
(c): Does Broker ignore the configurable setting 'Pool Associated with Node', if additional instances are set to some non-zero value at Message flow level (with atleast one input node, like SOAPInput node)?
(d): Is that 308 and 558 total instances count valid for scenarios mentioned in points 4 and 5, respectively ?
(e): As per the point 5 (above), the deployment of bar would have failed cause additional instances at node level were set to 300 (i.e. 'n'>256, where 'n' is additional instances count). But actually deployment didn't fail. Question is, additional instances count at node level doesn't impact the deployment of bar in an execution group?
Software/Tools used:
IS03 has been used to display the total instances in pool during these tests, on Windows 7 with WMB v8.0.0.4.
Reference of an old post.
_________________
*Life will beat you down, you need to decide to fight back or leave it. |
|
| Back to top |
|
 |
| timber |
| Posted: Wed Apr 06, 2016 2:18 am Post subject: |
|
|
Grand Master
Joined: 25 Aug 2015
Posts: 1292
|
| I agree that there are some valid questions to be answered here. I'm not sure that anybody on this forum can answer those questions definitively. I think your best option is to open a PMR and get IBM's official answers. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Apr 06, 2016 5:13 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Additional instances are allocated when needed. Apparently this doesn't apply to additional instances of a message flow ?
So 257 is a perfectly reasonable number. It will increase if you send more than one message more or less simultaneously (within the time period while a single instance is processing the message)
In general, 256 instances of a flow is very very large, and you will only expect to use that many under very very very very high volume. Again, additional instances are only used when the primary instance is still processing a message. You would have to be sending more than 256 messages within that period for any of the node level instances to be used. And you will only use whatever portion of the 256 based on the rate of message arrival.
You might also wait five minutes and see what happens - the 257 instances may go down to 2. Might not, just a guess.
As the warning of falling logs says, though. A PMR will get you reliable information.
Behavior of all of this at v9 and v10 may be entirely different.
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| shashivarungupta |
| Posted: Wed Apr 06, 2016 1:35 pm Post subject: |
|
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
| mqjeff wrote: |
| Additional instances are allocated when needed. Apparently this doesn't apply to additional instances of a message flow ? |
As per InfoCenter "These instances can be started as you need them or you can choose to start all of them at once when the message flow starts by selecting the Start additional instances when the flow starts property. "
Start additional instances when the flow starts was enabled in the above tests (as mentioned earlier) and all the instances (Message Flow and Node Level) should have been started at the start of flow or first deployment of it.
| mqjeff wrote: |
| So 257 is a perfectly reasonable number. It will increase if you send more than one message more or less simultaneously (within the time period while a single instance is processing the message) |
I would rethink about it, when Start additional instances when the flow starts was enabled while bar override.
| mqjeff wrote: |
In general, 256 instances of a flow is very very large, and you will only expect to use that many under very very very very high volume. Again, additional instances are only used when the primary instance is still processing a message. You would have to be sending more than 256 messages within that period for any of the node level instances to be used. And you will only use whatever portion of the 256 based on the rate of message arrival.
You might also wait five minutes and see what happens - the 257 instances may go down to 2. Might not, just a guess. |
I might agree with you here because 256 is a big number in terms of additional instances of a message flow. Recently in one of the very busy web service application, we set 50+ instances at MFlow level and that looks reasonable, based on and after regression/performance testing on an application to anticipate and knowing the facts that how many users would access an application simultaneously.
| mqjeff wrote: |
| A PMR will get you reliable information. |
Certainly. Cause setting some parameters with some finite reasonable values should give some reasonable results, to prove a statement/theory.
_________________
*Life will beat you down, you need to decide to fight back or leave it. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Apr 07, 2016 4:28 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Message flow performance generally scales better with fewer instances in more than one EG, than with more instances in a single EG.
"simultaneous users' doesn't mean much. Those users could be hitting the same request at 10 millisecond different times. If your flow takes 5 milliseconds to process a single request, you can probably get away with 5 instances.
In addition, based on the input transport, you may already be dealing with a thread pool at the http server. So, again, there's a difference between 'simultaneous users' and 'instances of the message flow'.
If your performance testing showed 50 instances worked, you should still test it separately with all instances started versus starting instances as needed. You may find that only 10 instances ever get started.
It will take either a really slow message flow, or a very very very high number of messages hitting a flow at the same time for 256 instances to ever make sense.
You should spend a lot of time tuning the performance of the message flow before you start to ramp up instances.
And again, you're better off putting a much smaller number of instances in more EGs.
It's good to see that you have actually run performance testing!  
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Apr 07, 2016 1:05 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| mqjeff wrote: |
| Message flow performance generally scales better with fewer instances in more than one EG, than with more instances in a single EG. |
Why is that? (curious)
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| shashivarungupta |
| Posted: Thu Apr 07, 2016 4:02 pm Post subject: |
|
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
| mqjeff wrote: |
| Message flow performance generally scales better with fewer instances in more than one EG, than with more instances in a single EG. |
It also depends on the kind of purchased license of the product (above case would not be applicable if one has standard license of WMBv8.x with only 1 EG, I know that's awful.)
| mqjeff wrote: |
"simultaneous users' doesn't mean much. Those users could be hitting the same request at 10 millisecond different times. If your flow takes 5 milliseconds to process a single request, you can probably get away with 5 instances.
In addition, based on the input transport, you may already be dealing with a thread pool at the http server. So, again, there's a difference between 'simultaneous users' and 'instances of the message flow'. |
_________________
*Life will beat you down, you need to decide to fight back or leave it. |
|
| Back to top |
|
|
| shashivarungupta |
| Posted: Thu Apr 07, 2016 4:05 pm Post subject: |
|
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
| mqjeff wrote: |
| Message flow performance generally scales better with fewer instances in more than one EG, than with more instances in a single EG. |
Also... Isn't that tedious task to manage and maintain such kind of deployment strategy in a busy and complex environment where there are many deployed applications ? 
_________________
*Life will beat you down, you need to decide to fight back or leave it. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Apr 07, 2016 4:56 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| shashivarungupta wrote: |
| mqjeff wrote: |
| Message flow performance generally scales better with fewer instances in more than one EG, than with more instances in a single EG. |
Also... Isn't that tedious task to manage and maintain such kind of deployment strategy in a busy and complex environment where there are many deployed applications ? |
Yes. And like all tedious tasks, you automate it and get a machine with a high boredom threshold (or at least poor union representation) to do it.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| shashivarungupta |
| Posted: Thu Apr 07, 2016 6:58 pm Post subject: |
|
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
| Vitor wrote: |
| ... like all tedious tasks, you automate it and get a machine with a high boredom threshold (or at least poor union representation) to do it. |
I would say that not everybody is lucky to get the perfect environment to work on even if they wished to have it. 
..cost..cost..cost..cost cutting.. at the end of the day. 
_________________
*Life will beat you down, you need to decide to fight back or leave it. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Apr 07, 2016 7:00 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| shashivarungupta wrote: |
| Vitor wrote: |
| ... like all tedious tasks, you automate it and get a machine with a high boredom threshold (or at least poor union representation) to do it. |
I would say that not everybody is lucky to get the perfect environment to work on even if they wished to have it. |
ANT is free. Shell script is free.
Automation actually saves money. It also saves you having to perform simple tasks even if they're not that tedious (I don't know about you, but the fun went out of typing "mqsideploy" into a Prod command prompt sometime in 2004).
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
