|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
Lots of Execution Groups? |
« View previous topic :: View next topic » |
Author |
Message
|
PeterPotkay |
Posted: Fri Jun 17, 2005 2:40 pm Post subject: Lots of Execution Groups? |
|
|
 Poobah
Joined: 15 May 2001 Posts: 7722
|
Having fun designing our new Broker environment. I am curious what you guys do out there.
How many Execution Groups does your Broker have?
How many flows on average does each EG have?
How many CPUs does your Broker run with?
Anyone running on Linux on Intel? Or maybe z/Linux? _________________ Peter Potkay
Keep Calm and MQ On |
|
Back to top |
|
 |
jefflowrey |
Posted: Fri Jun 17, 2005 3:24 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002 Posts: 19981
|
Generally, I aim for one execution group per CPU in the box.
Another approach is to have one execution group per logically connected set of flows/sets, to think in terms of "if something breaks and locks up the eg, what can I stand having impacted". For instance, on a single CPU box, then I'd do EGs this way.
I've done linux on intel, but only in a POC sense and haven't seen it in production. It seemed reasonable, though - especially now that there's support for some more dbs. _________________ I am *not* the model of the modern major general. |
|
Back to top |
|
 |
zpat |
Posted: Fri Jun 17, 2005 10:53 pm Post subject: |
|
|
 Jedi Council
Joined: 19 May 2001 Posts: 5866 Location: UK
|
We aim for around twice the EGs as CPUs - because otherwise CPUs could be idle when flows are waiting for i/o to complete.
But it depends how much i/o or other waits your flows perform. |
|
Back to top |
|
 |
JohnMetcalfe |
Posted: Wed Jun 22, 2005 11:30 am Post subject: |
|
|
 Apprentice
Joined: 02 Apr 2004 Posts: 40 Location: Edinburgh, Scotland
|
We have numerous execution groups, our approach is to to create execution groups aligned to the area/application/project. However, due to memory limitation on the execution groups, we limit the number of flows within an execution geroup to 20 or so - if we go over this we would tend to split out into _1, _2, _3 etc suffixed execution groups. In very unusual cases, where a flow processes needed to process large (up to 20MB) messages, we can have an execution group per flow. We have several hundred flows in prod currently.
Each execution group has an overhead of some 40- 50MB, so this is worth bearing in mind.
We run with 15GB of memory and 5CPU in our prod broker environment, but currently this is shared with WAS (Websphere application server) as well. |
|
Back to top |
|
 |
PeterPotkay |
Posted: Wed Jun 22, 2005 2:50 pm Post subject: |
|
|
 Poobah
Joined: 15 May 2001 Posts: 7722
|
John, I am curious what O/S you run on. How many EGs do you have? It looks like well over 50 if I do the math righ on your numbers. _________________ Peter Potkay
Keep Calm and MQ On |
|
Back to top |
|
 |
JohnMetcalfe |
Posted: Fri Jun 24, 2005 7:33 am Post subject: |
|
|
 Apprentice
Joined: 02 Apr 2004 Posts: 40 Location: Edinburgh, Scotland
|
Peter, we run on AIX UNIX, running on an IBM p690 server. We have about 30 or so execution groups in Prod.
Another reason we use multiple execution group is because we have multiple teams across the company all developing stuff in WMQI, so the execution groups act as run time 'containers' for a particular bunch of flows developed by the same team. |
|
Back to top |
|
 |
Carl Bloy |
Posted: Wed Jun 29, 2005 1:48 am Post subject: |
|
|
Acolyte
Joined: 16 Dec 2003 Posts: 69 Location: England
|
Also there appears to be a limit on the number of connection handles to queues within an execution group, we hit this problem a few months back.
You may want to check the Open input and output counts add up to what you exepct for your flows.
In our situation this didn't happen, it didn't effect the flow operation but i'm assuming there's an overhead on connecting to the queue. Once we split the flows across execution groups the open input and output counts were at the correct levels. _________________ Regards
Carl |
|
Back to top |
|
 |
PeterPotkay |
Posted: Wed Jul 27, 2005 5:26 pm Post subject: |
|
|
 Poobah
Joined: 15 May 2001 Posts: 7722
|
So I have 16 EGs, and about 140 flows spread unevenly throughout them. The EGs align with applications. Some of the EGs contain only flows that process messages on the weekend. But those EGs take up memory 24x7,
And we see more and more apps coming onboard. Clearly we can't keep creating new EGs indefinitly.
We could keep going the way we have, and just try and have hardware with gobs of Memory, like 12 GB+
-or-
We change everything, and consolidate.
We create some random number of EGs, and divide the flows in some random fashion between those. Looks like 2x the number of CPUs would make sense, so on my servers, 4 EGs. Name them EG1, EG2, EG3 and EG4. CPU intensive flows would be segragated between the EGs as much as possible. Going forward we would end up with 4 EGs with 50 or so flows in each.
But John's post indicates 20+ flows in an EG is not smart. But at the MQ Tech conferance people seem surprised that I had that many EGs.
I am worried if we switch to have only 4 EGs overall performance may suffer. I don't think there is anyway to guarantee that 2 EGs will get assigned to CPU#1 and the other 2 go to CPU#2. What if by the luck of the draw 3 or 4 get assigned to the same CPU? At least with lots of EGs, you can assume the load will be evenly split. Is this something to worry about? Does it matter if it is Windows or Unix?
Other than they way it "looks" IS there any reason to have more than 1 EG per cpu? I mean, whose to say EGs are these fragile animals and we need to split all our flows up between lots of them. I'm sure there are lots of components under the covers that could crash just as easily as an EG, and we don't worry about making multiple copies of them.
I looked in the WB-IMB manuals and at the various articles written by Tim Dunn, eg:
http://www-128.ibm.com/developerworks/websphere/library/techarticles/0211_dunn/dunn.html
and there is no recomendation for the # of EGs to use.
Surely there must be an official guideline?
The closest I found was this quote in the WB-IMb Concepts manual. If I don't have these security considerations, it would seem to indicate 1 EG is all you need!
Quote: |
An execution group is a named grouping of message flows that have been assigned to a broker. The broker enforces a degree of isolation between message flows in distinct execution groups by ensuring that they execute in separate address spaces, or as unique processes. Each execution group is started as a separate operating system process, providing an isolated runtime environment for a set of deployed message flows. A single default execution group is set up ready for use when you create a broker in the workbench. By setting up additional execution groups, you can isolate message flows that handle sensitive data such as payroll records, or security information, or unannounced product information, from other non-sensitive message flows. Within an execution group, the assigned message flows run in different thread pools. You can specify the size of the thread pool (that is, the number of threads) that are assigned for each message flow by specifying the number of additional instances of each message flow. If you create additional execution groups, you must give each group a name that is unique within the broker, and assign and deploy one or more message flows to each one. Execution groups are created and deployed in the workbench.
|
_________________ Peter Potkay
Keep Calm and MQ On |
|
Back to top |
|
 |
Bill.Matthews |
Posted: Wed Jul 27, 2005 6:16 pm Post subject: |
|
|
 Master
Joined: 23 Sep 2003 Posts: 232 Location: IBM (Retired)
|
Just a comment - several years ago I did a PoC (using 2.1) on message aggregation - with a Windows machine feeding messages to a 6 way AIX machine with SAA drives, a pretuned broker database and 5 flows to handle the simulated backends (also on the windows machine). I had a single execution group with most of the flows having between 3 and 6 additional instances. During the "peak" test period, I was running all six engines at 98% utilization.
These message flows were also given to Hursley and then generalized and used as a basis for the IP05 performance report. Husley also worked closley with me on building more effective message flows.
Also note that IP04 and IP11 are a couple of Tim Dunn's presentations on designing for performance.
I can "argue" both sides of the question. There have been several very good examples in this topic for making decisions concerning multiple EGS.
Another reason might be: All new or modified flows are allocated to a special EG for a period of time in order to prove their realibility. So, if one should fail, the "stable" flows (in other EGs) will not be affected.
Cheers
Bill Matthews _________________ Bill Matthews |
|
Back to top |
|
 |
PeterPotkay |
Posted: Wed Jul 27, 2005 8:08 pm Post subject: |
|
|
 Poobah
Joined: 15 May 2001 Posts: 7722
|
Bill.Matthews wrote: |
Just a comment - several years ago I did a PoC (using 2.1) on message aggregation - with a Windows machine feeding messages to a 6 way AIX machine with SAA drives, a pretuned broker database and 5 flows to handle the simulated backends (also on the windows machine). I had a single execution group with most of the flows having between 3 and 6 additional instances. During the "peak" test period, I was running all six engines at 98% utilization.
|
If you had only 1 EG, then you had only one DataFlowEngine exe (process), meaning you were only using 1 of the 6 CPUs, right? It is my understanding that flows in a single EG are just threads of that EG process, and they all compete for the same CPU. To utilize all 6 CPUs by the flows, you would have needed 6 EGs at a minimum, and made the assumption that the O/S would have been smart enough to assign each of the 6 EGs to a separate CPU ?
Bill.Matthews wrote: |
Another reason might be: All new or modified flows are allocated to a special EG for a period of time in order to prove their realibility. So, if one should fail, the "stable" flows (in other EGs) will not be affected.
|
Playing devil's advocate here, what's so special about the EG? Couldn't a contact admin flow take down the server, QM, broker, DB, etc even if it was running in a separate EG? The only true isolation is another server all together, no? _________________ Peter Potkay
Keep Calm and MQ On |
|
Back to top |
|
 |
fschofer |
Posted: Wed Jul 27, 2005 11:06 pm Post subject: |
|
|
 Knight
Joined: 02 Jul 2001 Posts: 524 Location: Mainz, Germany
|
Hi,
Quote: |
If you had only 1 EG, then you had only one DataFlowEngine exe (process), meaning you were only using 1 of the 6 CPUs, right? It is my understanding that flows in a single EG are just threads of that EG process, and they all compete for the same CPU. To utilize all 6 CPUs by the flows, you would have needed 6 EGs at a minimum, and made the assumption that the O/S would have been smart enough to assign each of the 6 EGs to a separate CPU ? |
1 EG can use more than one CPU
http://www-128.ibm.com/developerworks/websphere/library/techarticles/0301_dunn/dunn.html
http://www-128.ibm.com/developerworks/websphere/library/techarticles/0311_dunn/dunn.html
Quote: |
Each additional instance of a message flow is implemented as an operating system thread on Windows and UNIX, or as a Task Control Block (TCB) on z/OS. Each additional instance has the potential to use an additional processor. |
Greetings
Frank |
|
Back to top |
|
 |
Bill.Matthews |
Posted: Thu Jul 28, 2005 4:31 am Post subject: |
|
|
 Master
Joined: 23 Sep 2003 Posts: 232 Location: IBM (Retired)
|
PeterPotkay wrote: |
Playing devil's advocate here, what's so special about the EG? Couldn't a contact admin flow take down the server, QM, broker, DB, etc even if it was running in a separate EG? The only true isolation is another server all together, no? |
One would hope that a very badley misbehaved flow would be caught in a Systems Test or a Quality Assurance broker(s) prior to going into production. But we all know that not every situation can be found in a test environment.
Its basically a matter of understanding the risks.. and thats true regardless of what system is being used.
Taking everything into consideration - this is a good discussion top since it shows many different viewpoints. As a generalization, I perfer a fewer number of EGs. As an advisor - I would want to understand the justification in a particular situation since there are many factors that can come into play. I also depend on the experts (i.e. Tim Dunn) for their recommendations.
(As an aside, I also encourage new customers to also come here for peer advice.)
Cheers
Bill
Cheers to all _________________ Bill Matthews |
|
Back to top |
|
 |
PeterPotkay |
Posted: Thu Jul 28, 2005 8:17 am Post subject: |
|
|
 Poobah
Joined: 15 May 2001 Posts: 7722
|
Frank, your quote certainly is definitive in saying 1 EG can use more than 1 CPU.
Maybe I was/am badly misinformed, but isn't a single EG a single process (with threads), and a single process (regardless of what it is) can only utilize a single CPU? _________________ Peter Potkay
Keep Calm and MQ On |
|
Back to top |
|
 |
fschofer |
Posted: Thu Jul 28, 2005 12:05 pm Post subject: |
|
|
 Knight
Joined: 02 Jul 2001 Posts: 524 Location: Mainz, Germany
|
|
Back to top |
|
 |
Michael Dag |
Posted: Thu Jul 28, 2005 1:01 pm Post subject: |
|
|
 Jedi Knight
Joined: 13 Jun 2002 Posts: 2607 Location: The Netherlands (Amsterdam)
|
fschofer wrote: |
I see this often when i run top on one of my solaris boxes. |
Interesting. I thought the additional instances wasn't supported on solaris,
or at least they did not run concurrently...
something fixed in 5.0 ? _________________ Michael
MQSystems Facebook page |
|
Back to top |
|
 |
|
|
 |
Goto page 1, 2, 3 Next |
Page 1 of 3 |
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|
|
|