ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Lots of Execution Groups?

Post new topic  Reply to topic Goto page 1, 2, 3  Next
 Lots of Execution Groups? « View previous topic :: View next topic » 
Author Message
PeterPotkay
PostPosted: Fri Jun 17, 2005 2:40 pm    Post subject: Lots of Execution Groups? Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7722

Having fun designing our new Broker environment. I am curious what you guys do out there.

How many Execution Groups does your Broker have?
How many flows on average does each EG have?
How many CPUs does your Broker run with?
Anyone running on Linux on Intel? Or maybe z/Linux?
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
jefflowrey
PostPosted: Fri Jun 17, 2005 3:24 pm    Post subject: Reply with quote

Grand Poobah

Joined: 16 Oct 2002
Posts: 19981

Generally, I aim for one execution group per CPU in the box.

Another approach is to have one execution group per logically connected set of flows/sets, to think in terms of "if something breaks and locks up the eg, what can I stand having impacted". For instance, on a single CPU box, then I'd do EGs this way.

I've done linux on intel, but only in a POC sense and haven't seen it in production. It seemed reasonable, though - especially now that there's support for some more dbs.
_________________
I am *not* the model of the modern major general.
Back to top
View user's profile Send private message
zpat
PostPosted: Fri Jun 17, 2005 10:53 pm    Post subject: Reply with quote

Jedi Council

Joined: 19 May 2001
Posts: 5866
Location: UK

We aim for around twice the EGs as CPUs - because otherwise CPUs could be idle when flows are waiting for i/o to complete.

But it depends how much i/o or other waits your flows perform.
Back to top
View user's profile Send private message
JohnMetcalfe
PostPosted: Wed Jun 22, 2005 11:30 am    Post subject: Reply with quote

Apprentice

Joined: 02 Apr 2004
Posts: 40
Location: Edinburgh, Scotland

We have numerous execution groups, our approach is to to create execution groups aligned to the area/application/project. However, due to memory limitation on the execution groups, we limit the number of flows within an execution geroup to 20 or so - if we go over this we would tend to split out into _1, _2, _3 etc suffixed execution groups. In very unusual cases, where a flow processes needed to process large (up to 20MB) messages, we can have an execution group per flow. We have several hundred flows in prod currently.

Each execution group has an overhead of some 40- 50MB, so this is worth bearing in mind.

We run with 15GB of memory and 5CPU in our prod broker environment, but currently this is shared with WAS (Websphere application server) as well.
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Wed Jun 22, 2005 2:50 pm    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7722

John, I am curious what O/S you run on. How many EGs do you have? It looks like well over 50 if I do the math righ on your numbers.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
JohnMetcalfe
PostPosted: Fri Jun 24, 2005 7:33 am    Post subject: Reply with quote

Apprentice

Joined: 02 Apr 2004
Posts: 40
Location: Edinburgh, Scotland

Peter, we run on AIX UNIX, running on an IBM p690 server. We have about 30 or so execution groups in Prod.

Another reason we use multiple execution group is because we have multiple teams across the company all developing stuff in WMQI, so the execution groups act as run time 'containers' for a particular bunch of flows developed by the same team.
Back to top
View user's profile Send private message
Carl Bloy
PostPosted: Wed Jun 29, 2005 1:48 am    Post subject: Reply with quote

Acolyte

Joined: 16 Dec 2003
Posts: 69
Location: England

Also there appears to be a limit on the number of connection handles to queues within an execution group, we hit this problem a few months back.

You may want to check the Open input and output counts add up to what you exepct for your flows.

In our situation this didn't happen, it didn't effect the flow operation but i'm assuming there's an overhead on connecting to the queue. Once we split the flows across execution groups the open input and output counts were at the correct levels.
_________________
Regards
Carl
Back to top
View user's profile Send private message Visit poster's website
PeterPotkay
PostPosted: Wed Jul 27, 2005 5:26 pm    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7722

So I have 16 EGs, and about 140 flows spread unevenly throughout them. The EGs align with applications. Some of the EGs contain only flows that process messages on the weekend. But those EGs take up memory 24x7,

And we see more and more apps coming onboard. Clearly we can't keep creating new EGs indefinitly.

We could keep going the way we have, and just try and have hardware with gobs of Memory, like 12 GB+

-or-

We change everything, and consolidate.
We create some random number of EGs, and divide the flows in some random fashion between those. Looks like 2x the number of CPUs would make sense, so on my servers, 4 EGs. Name them EG1, EG2, EG3 and EG4. CPU intensive flows would be segragated between the EGs as much as possible. Going forward we would end up with 4 EGs with 50 or so flows in each.

But John's post indicates 20+ flows in an EG is not smart. But at the MQ Tech conferance people seem surprised that I had that many EGs.

I am worried if we switch to have only 4 EGs overall performance may suffer. I don't think there is anyway to guarantee that 2 EGs will get assigned to CPU#1 and the other 2 go to CPU#2. What if by the luck of the draw 3 or 4 get assigned to the same CPU? At least with lots of EGs, you can assume the load will be evenly split. Is this something to worry about? Does it matter if it is Windows or Unix?

Other than they way it "looks" IS there any reason to have more than 1 EG per cpu? I mean, whose to say EGs are these fragile animals and we need to split all our flows up between lots of them. I'm sure there are lots of components under the covers that could crash just as easily as an EG, and we don't worry about making multiple copies of them.

I looked in the WB-IMB manuals and at the various articles written by Tim Dunn, eg:
http://www-128.ibm.com/developerworks/websphere/library/techarticles/0211_dunn/dunn.html
and there is no recomendation for the # of EGs to use.

Surely there must be an official guideline?

The closest I found was this quote in the WB-IMb Concepts manual. If I don't have these security considerations, it would seem to indicate 1 EG is all you need!
Quote:

An execution group is a named grouping of message flows that have been assigned to a broker. The broker enforces a degree of isolation between message flows in distinct execution groups by ensuring that they execute in separate address spaces, or as unique processes. Each execution group is started as a separate operating system process, providing an isolated runtime environment for a set of deployed message flows. A single default execution group is set up ready for use when you create a broker in the workbench. By setting up additional execution groups, you can isolate message flows that handle sensitive data such as payroll records, or security information, or unannounced product information, from other non-sensitive message flows. Within an execution group, the assigned message flows run in different thread pools. You can specify the size of the thread pool (that is, the number of threads) that are assigned for each message flow by specifying the number of additional instances of each message flow. If you create additional execution groups, you must give each group a name that is unique within the broker, and assign and deploy one or more message flows to each one. Execution groups are created and deployed in the workbench.

_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
Bill.Matthews
PostPosted: Wed Jul 27, 2005 6:16 pm    Post subject: Reply with quote

Master

Joined: 23 Sep 2003
Posts: 232
Location: IBM (Retired)

Just a comment - several years ago I did a PoC (using 2.1) on message aggregation - with a Windows machine feeding messages to a 6 way AIX machine with SAA drives, a pretuned broker database and 5 flows to handle the simulated backends (also on the windows machine). I had a single execution group with most of the flows having between 3 and 6 additional instances. During the "peak" test period, I was running all six engines at 98% utilization.

These message flows were also given to Hursley and then generalized and used as a basis for the IP05 performance report. Husley also worked closley with me on building more effective message flows.

Also note that IP04 and IP11 are a couple of Tim Dunn's presentations on designing for performance.

I can "argue" both sides of the question. There have been several very good examples in this topic for making decisions concerning multiple EGS.

Another reason might be: All new or modified flows are allocated to a special EG for a period of time in order to prove their realibility. So, if one should fail, the "stable" flows (in other EGs) will not be affected.

Cheers
Bill Matthews
_________________
Bill Matthews
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Wed Jul 27, 2005 8:08 pm    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7722

Bill.Matthews wrote:
Just a comment - several years ago I did a PoC (using 2.1) on message aggregation - with a Windows machine feeding messages to a 6 way AIX machine with SAA drives, a pretuned broker database and 5 flows to handle the simulated backends (also on the windows machine). I had a single execution group with most of the flows having between 3 and 6 additional instances. During the "peak" test period, I was running all six engines at 98% utilization.

If you had only 1 EG, then you had only one DataFlowEngine exe (process), meaning you were only using 1 of the 6 CPUs, right? It is my understanding that flows in a single EG are just threads of that EG process, and they all compete for the same CPU. To utilize all 6 CPUs by the flows, you would have needed 6 EGs at a minimum, and made the assumption that the O/S would have been smart enough to assign each of the 6 EGs to a separate CPU ?

Bill.Matthews wrote:

Another reason might be: All new or modified flows are allocated to a special EG for a period of time in order to prove their realibility. So, if one should fail, the "stable" flows (in other EGs) will not be affected.

Playing devil's advocate here, what's so special about the EG? Couldn't a contact admin flow take down the server, QM, broker, DB, etc even if it was running in a separate EG? The only true isolation is another server all together, no?
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
fschofer
PostPosted: Wed Jul 27, 2005 11:06 pm    Post subject: Reply with quote

Knight

Joined: 02 Jul 2001
Posts: 524
Location: Mainz, Germany

Hi,
Quote:
If you had only 1 EG, then you had only one DataFlowEngine exe (process), meaning you were only using 1 of the 6 CPUs, right? It is my understanding that flows in a single EG are just threads of that EG process, and they all compete for the same CPU. To utilize all 6 CPUs by the flows, you would have needed 6 EGs at a minimum, and made the assumption that the O/S would have been smart enough to assign each of the 6 EGs to a separate CPU ?

1 EG can use more than one CPU

http://www-128.ibm.com/developerworks/websphere/library/techarticles/0301_dunn/dunn.html
http://www-128.ibm.com/developerworks/websphere/library/techarticles/0311_dunn/dunn.html

Quote:
Each additional instance of a message flow is implemented as an operating system thread on Windows and UNIX, or as a Task Control Block (TCB) on z/OS. Each additional instance has the potential to use an additional processor.


Greetings
Frank
Back to top
View user's profile Send private message Send e-mail
Bill.Matthews
PostPosted: Thu Jul 28, 2005 4:31 am    Post subject: Reply with quote

Master

Joined: 23 Sep 2003
Posts: 232
Location: IBM (Retired)

PeterPotkay wrote:
Playing devil's advocate here, what's so special about the EG? Couldn't a contact admin flow take down the server, QM, broker, DB, etc even if it was running in a separate EG? The only true isolation is another server all together, no?


One would hope that a very badley misbehaved flow would be caught in a Systems Test or a Quality Assurance broker(s) prior to going into production. But we all know that not every situation can be found in a test environment.

Its basically a matter of understanding the risks.. and thats true regardless of what system is being used.

Taking everything into consideration - this is a good discussion top since it shows many different viewpoints. As a generalization, I perfer a fewer number of EGs. As an advisor - I would want to understand the justification in a particular situation since there are many factors that can come into play. I also depend on the experts (i.e. Tim Dunn) for their recommendations.

(As an aside, I also encourage new customers to also come here for peer advice.)

Cheers
Bill

Cheers to all
_________________
Bill Matthews
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Thu Jul 28, 2005 8:17 am    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7722

Frank, your quote certainly is definitive in saying 1 EG can use more than 1 CPU.

Maybe I was/am badly misinformed, but isn't a single EG a single process (with threads), and a single process (regardless of what it is) can only utilize a single CPU?
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
fschofer
PostPosted: Thu Jul 28, 2005 12:05 pm    Post subject: Reply with quote

Knight

Joined: 02 Jul 2001
Posts: 524
Location: Mainz, Germany

Back to top
View user's profile Send private message Send e-mail
Michael Dag
PostPosted: Thu Jul 28, 2005 1:01 pm    Post subject: Reply with quote

Jedi Knight

Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)

fschofer wrote:
I see this often when i run top on one of my solaris boxes.

Interesting. I thought the additional instances wasn't supported on solaris,
or at least they did not run concurrently...

something fixed in 5.0 ?
_________________
Michael



MQSystems Facebook page
Back to top
View user's profile Send private message Visit poster's website MSN Messenger
Display posts from previous:   
Post new topic  Reply to topic Goto page 1, 2, 3  Next Page 1 of 3

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Lots of Execution Groups?
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.