| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| Consolidating multiple QM's to a single Solaris Zone |
« View previous topic :: View next topic » |
| Author |
Message
|
| LouML |
 Posted: Tue Jan 08, 2013 4:25 am Post subject: Consolidating multiple QM's to a single Solaris Zone |
 |
|
Partisan
Joined: 10 Nov 2005
Posts: 305
Location: Jersey City, NJ / Bethpage, NY
|
Our current Production MQ environment consists of multiple application servers (mostly Solaris, with a few Windows), each running their own Queue Manager. The Queue manager versions range from 6.0.2.x up to 7.5.0.0. We currently have about 25 Production queue managers. This number will drop considerably as 8 of the queue managers are for apps scheduled for decommissioning at some point this year. Of the remaining queue managers, some have as little as one pair for sdr/rcvr channels and a few queues with minimal volume. Even the busiest of the queue managers, is not terribly busy by MQ standards.
We are planning to migrate these queue managers to a single Solaris Zone server, mqm1p (host1p/host2p) with mqm1dr (host1dr/host2dr) as DR in a different data center as a way to cut our MQ costs and simplify maintenance. We initially were going to use MQ Multi-Instance Queue Manager, but after considerable testing, abandoned this idea, since we have many legacy applications and a few external connections that cannot support it.
In Development, we’ve already migrated all of the queue managers to a new Solaris zone, mqm1d (host1d/host2d), running 7.5.0.0 and each queue manager runs with its own port number. This is working extremely well.
Currently, some of these Queue Managers are connected to some of the others with sdr/rcvr channels. So, we have a QRemote on QM-A sending a message to a QLocal on QM-B, even though they both reside on the same server. Ideally, the application that does the PUT to the QRemote on QM-A, could just PUT to the QLocal on QM-B directly.
Is anyone doing something similar? Any issues or concerns? As a long term design change, does this scenario lend itself well to Queue Manager Clustering?
_________________
Yeah, well, you know, that's just, like, your opinion, man. - The Dude |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Tue Jan 08, 2013 5:19 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
As a long term design, I would tend to start from scratch.
Create some kind of diagram that consists of all of your qlocals and applications. Draw arrows that connect the relevant apps to the queues they access. Do something to indicate the volume of work being performed on each queue.
Then sort and organize these groups of queues into a set of queue managers that will be built to handle the volume - making at least some consideration for reducing the number of messages that have to travel over channels (i.e. collocate queues where it makes sense).
Then determine if you want to spread the given volume of PUTs to some queue across an MQ cluster.
Then determine how to connect the queue managers together. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Jan 08, 2013 5:56 pm Post subject: Re: Consolidating multiple QM's to a single Solaris Zone |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| LouML wrote: |
Ideally, the application that does the PUT to the QRemote on QM-A, could just PUT to the QLocal on QM-B directly.
|
If making this change, I would vote having the app put to a new Alias Queue on QM-B that then points to the same local queue on QM-B. This will allow you more flexibility related to things like put/get disabling the Alias queue to control access from the original app (the one that used to connect to QM-A) without impacting the app that always was using this queue on QM-B.
Some people advocate fronting every single app queue with an Alias Queue anyway.
| LouML wrote: |
In Development, we’ve already migrated all of the queue managers to a new Solaris zone, mqm1d (host1d/host2d), running 7.5.0.0 and each queue manager runs with its own port number. This is working extremely well. |
Is this how you will do it in Production? If yes, then I understand why you did it like this in DEV. But I wonder, why have 10 queues spread across 10 Queue Managers on the same server versus 100 queues on a single QM? What does this buy you, on current versions of MQ?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| LouML |
| Posted: Wed Jan 09, 2013 3:53 am Post subject: Re: Consolidating multiple QM's to a single Solaris Zone |
|
|
Partisan
Joined: 10 Nov 2005
Posts: 305
Location: Jersey City, NJ / Bethpage, NY
|
| PeterPotkay wrote: |
| LouML wrote: |
Ideally, the application that does the PUT to the QRemote on QM-A, could just PUT to the QLocal on QM-B directly.
|
If making this change, I would vote having the app put to a new Alias Queue on QM-B that then points to the same local queue on QM-B. This will allow you more flexibility related to things like put/get disabling the Alias queue to control access from the original app (the one that used to connect to QM-A) without impacting the app that always was using this queue on QM-B.
Some people advocate fronting every single app queue with an Alias Queue anyway.
| LouML wrote: |
In Development, we’ve already migrated all of the queue managers to a new Solaris zone, mqm1d (host1d/host2d), running 7.5.0.0 and each queue manager runs with its own port number. This is working extremely well. |
Is this how you will do it in Production? If yes, then I understand why you did it like this in DEV. But I wonder, why have 10 queues spread across 10 Queue Managers on the same server versus 100 queues on a single QM? What does this buy you, on current versions of MQ? |
Yes, Peter, this is the plan for now. The queue managers were all created before I came to this firm. While I believe some of the queue managers could be consolidated, there are a number of business groups involved which might not allow for a single queue manager. We’re still discussing our options.
We’ve got a real mess here and this is my chance to get this right. I don’t want to swing and miss.
_________________
Yeah, well, you know, that's just, like, your opinion, man. - The Dude |
|
| Back to top |
|
|
| LouML |
| Posted: Wed Jan 09, 2013 4:03 am Post subject: |
|
|
Partisan
Joined: 10 Nov 2005
Posts: 305
Location: Jersey City, NJ / Bethpage, NY
|
| mqjeff wrote: |
As a long term design, I would tend to start from scratch.
Create some kind of diagram that consists of all of your qlocals and applications. Draw arrows that connect the relevant apps to the queues they access. Do something to indicate the volume of work being performed on each queue.
Then sort and organize these groups of queues into a set of queue managers that will be built to handle the volume - making at least some consideration for reducing the number of messages that have to travel over channels (i.e. collocate queues where it makes sense).
Then determine if you want to spread the given volume of PUTs to some queue across an MQ cluster.
Then determine how to connect the queue managers together. |
Thanks Jeff. The easiest thing would just be to copy all of the queue managers, as is, to the new server. But if we do that, we’d never get around to consolidating or making any needed changes.
I’m thinking about creating a new queue manager from scratch. Then, migrating apps as they’re ready to the new QM. This would allow me to put as many of the existing apps as possible to this new QM. Obviously, the in-house QM's that communicate with each other would require the most changes. The QM's talking to external QM's would be very straightforward.
_________________
Yeah, well, you know, that's just, like, your opinion, man. - The Dude |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Jan 09, 2013 4:12 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| LouML wrote: |
| I’m thinking about creating a new queue manager from scratch. Then, migrating apps as they’re ready to the new QM. This would allow me to put as many of the existing apps as possible to this new QM. Obviously, the in-house QM's that communicate with each other would require the most changes. The QM's talking to external QM's would be very straightforward. |
Yes, this is a reasonably good idea. Except for the fact that you really should include some notion of volume in this planning, instead of just saying "shove everything I can into one queue manager until it's full, then shove as much of the rest into another queue manager until it's full, and then... "
You might find that you can shove everything into, say, two queue managers and that would be fine. But if you did the volume analysis, you might find you could put everything into 4 queue managers, spread the load of some applications across two clustered queues, and have an overall better performing system *with room to grow*. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Jan 09, 2013 7:48 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
mqjeff, does your premise of multiple QMs for load reasons consider that the single QM or multiple QM would still all be on the same server (if I understand The Dude's proposed topology correctly)?
I don't know of any performance gains sending Load A to QMA and Load B to QMB of both QMA and QMB are on the same server. Due to the overhead an even idle QM takes, one could assume there would be a performance loss splitting it this way.
I don't know....I've never seen this scenario played out in the performance reports.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Jan 09, 2013 7:50 am Post subject: Re: Consolidating multiple QM's to a single Solaris Zone |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| LouML wrote: |
..there are a number of business groups involved which might not allow for a single queue manager. |
I assume you are talking about strictly political reasons and nothing technical, assuming the new multiple QMs would share the same O/S?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Jan 09, 2013 9:16 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| PeterPotkay wrote: |
mqjeff, does your premise of multiple QMs for load reasons consider that the single QM or multiple QM would still all be on the same server (if I understand The Dude's proposed topology correctly)?
I don't know of any performance gains sending Load A to QMA and Load B to QMB of both QMA and QMB are on the same server. Due to the overhead an even idle QM takes, one could assume there would be a performance loss splitting it this way.
I don't know....I've never seen this scenario played out in the performance reports. |
I'm thinking particularly of things like issues with deep queues. If you have shallower queues on more than one queue manager, that will perform better than a single deep queue on a single queue manager, regardless of how many boxes they are running on.
considerations of log management also apply here. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Jan 09, 2013 11:01 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| mqjeff wrote: |
| PeterPotkay wrote: |
mqjeff, does your premise of multiple QMs for load reasons consider that the single QM or multiple QM would still all be on the same server (if I understand The Dude's proposed topology correctly)?
I don't know of any performance gains sending Load A to QMA and Load B to QMB of both QMA and QMB are on the same server. Due to the overhead an even idle QM takes, one could assume there would be a performance loss splitting it this way.
I don't know....I've never seen this scenario played out in the performance reports. |
I'm thinking particularly of things like issues with deep queues. If you have shallower queues on more than one queue manager, that will perform better than a single deep queue on a single queue manager, regardless of how many boxes they are running on.
considerations of log management also apply here. |
Sure, a 2nd qmgr in the same o/s instance will consume resources in competition with the 1st qmgr; but this presumes a poorly provisioned hardware platform or LPAR.
The decision to have a 2nd, or nth, qmgr should be based on the business requirement; and not on the current hardware configuration. Or so I tell my clients.
Tuning should address low-hanging fruit first - those efforts that will yield the most return on investment. Once those have been addressed, it may be time to move on to tuning efforts only likely to yield little (or no) improvement.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Jan 09, 2013 12:39 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
What is a business requirement that drives a second QM on the same server versus a second queue on the first and only QM on that server?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Jan 09, 2013 12:43 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| mqjeff wrote: |
I'm thinking particularly of things like issues with deep queues. If you have shallower queues on more than one queue manager, that will perform better than a single deep queue on a single queue manager, regardless of how many boxes they are running on.
|
No need to go from multipel shallow queues on multiple QM to a single deep queue on a single QM. Go to multipel shallow queues on the single QM.
| mqjeff wrote: |
considerations of log management also apply here. |
Ah, OK. Segragating an app that has long, loooooooong, running Unit of Works to its own set of logs (meaning its own QM). Interesting....thinking...
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Jan 09, 2013 1:31 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| PeterPotkay wrote: |
| mqjeff wrote: |
I'm thinking particularly of things like issues with deep queues. If you have shallower queues on more than one queue manager, that will perform better than a single deep queue on a single queue manager, regardless of how many boxes they are running on.
|
No need to go from multipel shallow queues on multiple QM to a single deep queue on a single QM. Go to multipel shallow queues on the single QM. |
Hard to do if the app is putting to a single queue name... |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Jan 09, 2013 1:54 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Shall we define "shallow" vs. "deep" queue?
Shall we define "small" vs. "large" message size?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Jan 09, 2013 1:56 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| bruce2359 wrote: |
| Shall we define "shallow" vs. "deep" queue? |
a "shallow" queue is one that has fewer messages on it than cause the queue indexing algorithm to experience slowdowns fetching messages from the queue to meet pending GETS.
A deep queue is one that has more messages than the indexing algorithm can retrieve without causing GETS to slow down.
| bruce2359 wrote: |
| Shall we define "small" vs. "large" message size? |
A small message is less than 10 MB. A large message is one more than 50 MB. |
|
| Back to top |
|
|
|
|
|
  |
Goto page 1, 2, 3 Next |
Page 1 of 3 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
