| Author |
Message
|
| RadhamaniRamadoss |
 Posted: Thu Dec 20, 2012 6:23 am Post subject: Scada thread pooling and Additional instances |
 |
|
Apprentice
Joined: 08 Oct 2009
Posts: 42
|
My message flow design uses Scada node.
Now,I am confused with number of Additional instances in the message flow and the ScadaInput node's "Max number of connections" property with and without Use Thread pool option.
The design goes like this ....
In Message flow:
No of Addtional instances = 100
In scadainput node:
Max number of connections =40 (With Use Thread pool option selected)
As per number of additional instances,100 threads can run parallely to process incoming messages.
Does that mean 100 client connections can be connected to Scada port?
And where is this Max number of connections at Scadanode coming into picture?
As per Info manual,
"Select Use thread pooling if you want the node to use a pool of threads to
service clients. If you select this option, the number of threads that are
available to the MQIsdp server is limited by Max threads, which is most
effective when set to a value between 20 and 40. If you do not select this
option, a new thread is created for each client that connects. The check box is cleared initially.
Use this option only if you expect a large number of clients (greater than
200) to connect."
1)If only 40 threads can be active from scada thread pool at a time,then how come 100 additonal instances defined as per design?
2)As per user manual,it is said if more number of clients are there,then use thread pooling.Then how can we define only 40 in Max connections?
3)I have 2500 clients connecting to the Scada port .Then how the below design works?
In Message flow:
No of Addtional instances = 100
In scadainput node:
Max number of connections =40 (With Use Thread pool option selected)
There are lot of confusions in understanding the above logic.I could not understand clearly though I did lot of surfing.
Can someone please throw some light on these? |
|
| Back to top |
|
 |
| kimbert |
| Posted: Thu Dec 20, 2012 3:58 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
| Quote: |
As per number of additional instances,100 threads can run parallely to process incoming messages.
Does that mean 100 client connections can be connected to Scada port? |
OK- I'll bite. I imagine that it works like this, although I'm prepared to be corrected by someone who knows better:
It means that ( up to ) 100 message flow instances, each running on a separate thread, will share ( up to ) 40 connections.
If all 40 connections are active, then a flow instance that needs a new connection must wait for one of the 40 to become available. That may or may not be a problem - it depends on the complexity of your message flow. If your message flow interacts with various back-end systems then other flow instances will be able to utilise a connection while it is completing those tasks. If your message flow is very simple then you will probably not get any performance benefit from increasing the additional instances beyond 40. |
|
| Back to top |
|
|
| RadhamaniRamadoss |
| Posted: Fri Dec 21, 2012 12:14 am Post subject: |
|
|
Apprentice
Joined: 08 Oct 2009
Posts: 42
|
We have 2500 clients to be connected to Scada port.How come just 40 connections will server the purpose then?
Again,Whats the connection between Scada thread pool and Additional instances....?
This design is already in production and I am just trying to understand the logic behind it. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Dec 21, 2012 9:36 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Each instance will use a connection only as long as it has to, and this does not necessarily require that the instance will use the connection for the entire lifetime of processing a given message.
particularly if SCADA is non-transaction (which I think you'll find is true....) then the connection is only needed long enough for the instance to actually read the message.
So 40 connections shared by 2500 instances means that each instance holds onto a connection just long enough to *start* processing. 40 messages get pulled at a time, into 40 instances which then release the connections for 40 more instances to pull messages which then release the connections for 40 more instances to pull messages which then...
and so on and so on and so on. |
|
| Back to top |
|
|
| RadhamaniRamadoss |
| Posted: Sat Dec 22, 2012 1:58 am Post subject: |
|
|
Apprentice
Joined: 08 Oct 2009
Posts: 42
|
Thanks for the reply...
As per mqjeff,40 connections is active at a time.
So,during peak hours,when all 2500 connections are busy,how would it be sufficient if just 40 connections pulling the messages at a time?
Then,where is this 100 additional instances for the message flow coming into picture?
Please explain...I am still not very clear about the entire picture. |
|
| Back to top |
|
|
| RadhamaniRamadoss |
| Posted: Mon Dec 24, 2012 12:59 am Post subject: |
|
|
Apprentice
Joined: 08 Oct 2009
Posts: 42
|
| Hi...Can someone please help me on the above query? |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Mon Dec 24, 2012 4:19 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
Please be aware that many of the regular posters to this forum
1) don't post at weekends
2) are probably on vacation until the new year.
Remember that any opinions expressed here are not the official IBM response.
If you realy want an official IBM response please open a PMR. That way the right people in IBM will get to look at your problem and you will get a far better answer.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Dec 24, 2012 4:22 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| RadhamaniRamadoss wrote: |
| Hi...Can someone please help me on the above query? |
Which part of
| mqjeff wrote: |
So 40 connections shared by 2500 instances means that each instance holds onto a connection just long enough to *start* processing. 40 messages get pulled at a time, into 40 instances which then release the connections for 40 more instances to pull messages which then release the connections for 40 more instances to pull messages which then...
and so on and so on and so on. |
did you not understand?? 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| RadhamaniRamadoss |
| Posted: Thu Dec 27, 2012 2:37 am Post subject: |
|
|
Apprentice
Joined: 08 Oct 2009
Posts: 42
|
As per the explanation,40 threads will be active at a time and 40 threads will be pulling messages at a time for SCADA node to process.So,when 2500 clients are there and during busy/peak hours all 2500 clients need the response without timeouts,how is this possible with just 40 threads active at a time?And what is the calculation behind running 100 message flow instances wrt to the 40 threads in single message flow?
Is that like 40*100 =400?
Or
For 100 message flow instances,40 threads is equally divided amongst themselves?
And,why dont we go for Max threads as 1000 or some maximum value without choosing thread pool?
Sorry if its a basic question.But please clear me... |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Thu Dec 27, 2012 5:59 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| RadhamaniRamadoss wrote: |
| that like 40*100 =400? |
No, 40 * 100 = 4,000 not 400.
You should execute your own performance testing and determine what exactly is needed. Or hire a WMB Developer consultant to do it for you if you are unsure how to do it.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| RadhamaniRamadoss |
| Posted: Fri Dec 28, 2012 4:14 am Post subject: |
|
|
Apprentice
Joined: 08 Oct 2009
Posts: 42
|
Sorry...Thats was a typo.
So,whether 40*100 =4000 threads are active at a time?
Or
For 100 message flow instances,40 threads are shared amongst themselves?
When do we go for thread pooling with 20 - 40 as a count why not we choose Max thread value without thread pooling as >500 ? |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Fri Dec 28, 2012 6:01 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| RadhamaniRamadoss wrote: |
| When do we go for thread pooling with 20 - 40 as a count why not we choose Max thread value without thread pooling as >500 ? |
When your own performance testing and your own expert analysis indicate it's time to do so.
These are knobs to turn. Keep tweaking the knobs till you find a sweet spot.
With seven of nine, I could draw you a map to the sweet spot, but since you are after twenty of forty, I'd have to play with the knobs to find it.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
|
|
