| Author |
Message
|
| rknox |
 Posted: Wed May 21, 2008 11:31 am Post subject: Message Broker best practices |
 |
|
Apprentice
Joined: 28 Aug 2007
Posts: 34
|
We have implemented Message Broker and have been using it successfully for nearly a year. We run version 6.1.0.1 on AIX. We are beginning to see signs of overloading each execution group with too many flows. Some signs include contention on deployments, taking nearly 5 minutes to stop or start the Broker, etc. It hasn’t become a problem situation yet. Since we have had success with the environment we are revisiting our standards. Initially we separated execution groups by business units but some execution groups are getting overloaded with flows and memory constraints since we are using quite a few java compute nodes. I have read (somewhere) that IBM recommends a 1-1 ratio of execution group to application, but I don’t think that is feasible from a cost perspective to our business. We are in a good position right now to re-establish standards and best practices and I would like to gather some input from this group.
My question is to anyone who has implemented or been involved in standard setting for your enterprise.
What is the stance that you employ regarding the number of brokers and/or execution groups that you use in relation to the number of applications deployed?
How has that worked for you?
Would you consider yours a large Message Broker shop?
Do you have any other words of wisdom from experience?
Thanks for your input. |
|
| Back to top |
|
 |
| mqmatt |
| Posted: Thu May 22, 2008 12:48 am Post subject: |
|
|
Grand Master
Joined: 04 Aug 2004
Posts: 1213
Location: Hursley, UK
|
| I'd be interested in hearing why you think an execution group not cost effective from a business point-of-view; it's an operating system process. |
|
| Back to top |
|
|
| rknox |
| Posted: Thu May 22, 2008 5:19 am Post subject: |
|
|
Apprentice
Joined: 28 Aug 2007
Posts: 34
|
| An Execution group itself is cost effective since it simply runs as an OS process. Our management has made a decision to charge our business partners by execution group. That decision has been made and it's the way billing is done. So, from that perspective our businesses have decided to pack as many flows and applications on each EG that they can. I am doing two things right now, 1) Since we have this chargeback in place, I am trying to figure out how many flows and processes we can allow on each execution group. 2) Trying to convince our management that we need to change our chargeback process. |
|
| Back to top |
|
|
| rknox |
| Posted: Thu May 22, 2008 5:56 am Post subject: |
|
|
Apprentice
Joined: 28 Aug 2007
Posts: 34
|
I thought I had read that IBM's recommendation is a 1 application per 1 execution group but now I can't find it. What actually constitutes an application? A number of flows that perform like functions so logically they are one application? If that is the case, how likely is it that an application could contain many flows and be too large to fit into a single execution group? I'm thinking of an app that may contain 50-100 flows.
As you may be able to tell, I'm not a developer, I work on the infrastructure portion of Broker.
thanks |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu May 22, 2008 6:01 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| rknox wrote: |
| I thought I had read that IBM's recommendation is a 1 application per 1 execution group but now I can't find it. |
It's not something I remember reading (which could just prove I've forgotten it).
Where I've split things across execution groups it's either been to share them across OS processes for load balancing reasons or because it maps onto a view of the applications that the business has.
Other strategies for mapping flows to groups are equally valid, I'm just quoting examples.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu May 22, 2008 6:02 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
An execution group can start no more than 255 threads.
This can be spread out over no more than 255 flows. It can be spread out over no fewer than 1 flow.
This is the only measure that's reasonable to use to determine how many EGs you need, and what is deployed to them. At least in production.
In development, every developer gets their own EG.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| rknox |
| Posted: Thu May 22, 2008 6:04 am Post subject: |
|
|
Apprentice
Joined: 28 Aug 2007
Posts: 34
|
| I recall where I heard about the 1-1 ratio recommendation. We had a consultant who was hired through IBM to come and help us implement Broker. He told us about IBM's recommendations. I heard it second hand. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu May 22, 2008 6:09 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| rknox wrote: |
| We had a consultant who was hired through IBM to come and help us implement Broker. |
Well there you go - those people are unprincipled brigands who'll say the first thing that comes into their heads if they can charge for it! 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| rknox |
| Posted: Thu May 22, 2008 6:09 am Post subject: |
|
|
Apprentice
Joined: 28 Aug 2007
Posts: 34
|
| jefflowrey, is that max thread count of 255 documented somewhere so I could read a little more about it? I would investigate more about execution group threads in a larger context if it is available. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu May 22, 2008 7:01 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
I may be misreading something. I'd thought there was a note that a single EG could only hold 255 (and I'm off by one there, it's 256) thread, and a second note that single flow can only use 256 additional instances.
But I can only find the second note.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu May 22, 2008 7:20 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Another formula I've heard is that you should have as many EGs as you have CPU cores on the server.
I created 10 EGs for my Brokers years ago and kept it that way. 1 EG holds batch style flows, so that if a big batch os messages come thru only that one process / dataflowengine.exe / EG goes nuts with CPU. Another EG holds all my flows that use HTTP nodes. The remaining 8 share the load for the rest of the flows, about 250 flows in total.
If you have very sensitive data I've heard of people using EGs to segregate the flows, although in my opinion if you need that seperation use seperate brokers on seperate QMs / servers.
I've poked at this question for years. There is no official answer. Its more art than science. It would be nice to have a section in the InfoCenter specific to this topic.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| rknox |
| Posted: Thu May 22, 2008 7:42 am Post subject: |
|
|
Apprentice
Joined: 28 Aug 2007
Posts: 34
|
| Peter are you saying that you have over 250 flows defined to your execution groups within a single broker instance? Do you run into problems by getting too many flows defined to a single broker? I am starting to see contention, but it's not at an OS level. Basically just latency problems are cropping up now and again since we have so many flows assigned to a single Broker/EG. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu May 22, 2008 12:00 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
250 flows on a Broker, spread across 10 EGs.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
|
|
