| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Load Balance Design Question |
View previous topic :: View next topic |
| Author |
Message
|
| PeterPotkay |
 Posted: Thu Jun 07, 2012 11:23 am Post subject: Load Balance Design Question |
 |
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
QM1 is on Server1, QM2 is on Server2.
QM1 and QM2 are in an MQ cluster.
InputQ is defined on both QM1 and QM2, and both instances of that queue are clustered. CLWLUSEQ is set to LOCAL.
ServiceApp is an application has multiple instances, half pulling from InputQ on QM1, and half pulling from InputQ on QM2.
New request messages are load balanced across these 2 queues coming from other QMs in the same cluster.
An existing app needs to start sending messages to InputQ. ExistingApp already has MQ Client connections to another QM (QM3) in the company but that other QM is NOT in the same MQ Cluster as QM1 and QM2. While anything is technically possible, there is no desire to add QM3 into the cluster that QM1 and QM2 are in. QM3 is a highly available, hardware clustered QM, fronted by a virtual name that follows the QM as the hardware cluster is moved between nodes. ExistingApp connects to QM3 by populating the fields needed for an MQCONNX style call – its does not use client channel tables. MQ Client version 6. This app has code to connect to a primary QM and if N/A to try a secondary QM by feeding in from a config file the info for the next QM and issuing another MQCONNX.
ExistingApp now needs to send requests to InputQ. ExistingApp has multiple instances.
What’s better? I think F (ironically “F”), but what say you?
A. ExistingApp upgrades to MQ 7 Client. For the new section of code to reach out to InputQ it uses MQ 7 Client Channel tables and gets its connections load balanced between QM1 and QM2. CLWLUSEQ stays set to LOCAL on InputQ. But this means ExistingApp needs to upgrade its MQ Client libraries and learn how to use client channel tables for the new connection. The current connection to QM3 is maintained, although optionally it can be tweaked to rely on the channel table. Additional code changes and unplanned upgrades – everyone hit the panic button.
B. ExistingApp uses the method it always has (no client channel tables), makes another MQCONNX call in parallel with the one it already does to QM3, and picks QM1 as its primary, and QM2 as it secondary for this new MQCONNX. CLWLUSEQ is changed to ANY for InputQ, so that while ExistingApp will only be connected to EITHER QM1 or QM2 at any given time, MQ clustering will load balance the messages between both instances of InputQ, thus keeping the load balanced for ServiceApp. ExistingApp continues to make separate MQCONNX calls to QM3 for its pre-existing MQ needs.
C. Basically the same as Option B. But half of its instances will run with a config file that has QM1 as primary and QM2 as secondary, and the other half use a QM2-then-QM1 config file. This will load balance the connections better than Option B and potentially better than Option A. CLWLUSEQ is left as LOCAL.
D. Same as option C, but CLWLUSEQ is set to ANY. Better load balancing of the messages in case the connection load balancing didn’t work out 100%. This would be the case if Server1 gets restarted for maintenance and all of ExistingApps connections go to Server2 / QM2. Until ExistingApp restarts itself, the connections are all on QM2. BUT, while QM connections are correctly split for ExistingApp to QM1 and QM2, does having the CLWLUSEQ attribute set to ANY cause unnecessary MQ traffic between QM1 and QM2?
E. Same as option A, but CLWLUSEQ is set to ANY. With the same Pros and Cons as described in Option D.
F. We suck it up and add QM3 to the cluster, and ExistingApp uses its existing connections to QM3 to load balance to InputQ on QM1 and QM2 via MQ clustering. Get the app to upgrade client to v7, but not a hard requirement. I really don’t like adding QM3 into that cluster. That cluster currently only holds QMs for a particular set of apps, its nicely segregated.
G. We make regular SNDR / RCVR channels between QM3 and one of the QMs in the cluster. But since none of the QMs in the cluster are Highly Available, I do not consider this a viable option.
Assumption – ExistingApp will connect to QM1 or QM2 once at startup and maintain that connection until it shutdown or something abnormal happens. Otherwise the same connection to the QM is used for multiple transactions.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
 |
| Vitor |
| Posted: Thu Jun 07, 2012 11:32 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
IMHO I'd go with option F but presenting a client with an option subtitled "suck it up" is not ideal, especially when you're breaking an existing segregated arrangement. You don't mention why it's segregted; if it's because the owners have issues sharing their toys that's one thing, if it's because they're business sensitive that's another.
I'd plump for D as a 2nd choice. Other opinions are equally valid and may be better.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| exerk |
| Posted: Thu Jun 07, 2012 1:10 pm Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
Why isn't a virtual IP used to front the highly available QM3, thereby removing the need for ExistingApp to step through connection possibilities? Apart from using a gateway queue manager, if you really don't want to put QM3 in the cluster, programmatic change of the application is the only way round it I can see.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Jun 08, 2012 1:29 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| exerk wrote: |
| Why isn't a virtual IP used to front the highly available QM3, thereby removing the need for ExistingApp to step through connection possibilities? Apart from using a gateway queue manager, if you really don't want to put QM3 in the cluster, programmatic change of the application is the only way round it I can see. |
I think you'll see on review that's not the question.
Peter - Option G is the right choice. The considerations for high availability do not seem meaningful to me. |
|
| Back to top |
|
|
| exerk |
| Posted: Fri Jun 08, 2012 1:31 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| mqjeff wrote: |
| exerk wrote: |
| Why isn't a virtual IP used to front the highly available QM3, thereby removing the need for ExistingApp to step through connection possibilities? Apart from using a gateway queue manager, if you really don't want to put QM3 in the cluster, programmatic change of the application is the only way round it I can see. |
I think you'll see on review that's not the question.
Peter - Option G is the right choice. The considerations for high availability do not seem meaningful to me. |
I was just curious! And I'm with you on the choice.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Jun 08, 2012 1:34 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| exerk wrote: |
| I was just curious! |
Well, okay, but...
| peterPotkay wrote: |
| QM3 is a highly available, hardware clustered QM, fronted by a virtual name that follows the QM as the hardware cluster is moved between nodes. |
|
|
| Back to top |
|
|
| exerk |
| Posted: Fri Jun 08, 2012 1:37 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| mqjeff wrote: |
| exerk wrote: |
| I was just curious! |
Well, okay, but...
| peterPotkay wrote: |
| QM3 is a highly available, hardware clustered QM, fronted by a virtual name that follows the QM as the hardware cluster is moved between nodes. |
|
In the cold light of day I think I misinterpreted this:
| Quote: |
| ...ExistingApp connects to QM3 by populating the fields needed for an MQCONNX style call – its does not use client channel tables. MQ Client version 6. This app has code to connect to a primary QM and if N/A to try a secondary QM by feeding in from a config file the info for the next QM and issuing another MQCONNX... |
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Jun 08, 2012 4:47 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Last night I'm thinking about this, and all of the sudden I realize that one of the QMs in that cluster is in fact a z/OS Queue Manager. Highly Available not an issue with that guy*. So having the request message go from QM3, SNDR channel to z/OS_QM, which then load balances to QM1 and QM2 to get to InputQ should do the trick. On the z/OS QM I will need to add a clustered QM3 Alias queue that resolves to the XMITQ back to QM3 for the replies. And another alias defined on the z/OS QM that will blank out the destination QM from the incoming request messages so that can load balance. Simple.
Pros
The app can follow best practices and continue to connect to only one QM. Much simpler from the app's perspective. in all regards. Just another put to another queue, and "the MQ" will give us an answer.
*Cons
A. The mainframe is IPL'ed on Saturday nights, so that z/OS QM is not always up. But those IPLs are happening when this app is not up anyway, so not an issue. For now.
B. This design has the request message and the reply message hopping across three QMs (QM3 to/from z/OS QM to/from QM1orQM2). If the app connected to QM1 or QM2 directly these QM to QM hops would be eliminated. Don't know how well this sits with me. It makes it a bit more complicated from an Infrastructure perspective. The overall time for the app between submitting the request and getting the reply will be longer, but likely only on the order of hundredths of a second extra to make those 2 QM hops. It for an online app, only about 25K requests a day.
| Quote: |
Peter - Option G is the right choice. The considerations for high availability do not seem meaningful to me. |
Jeff, this was surprising to read. Its not that much of an issue now that I have a z/OS QM to use as a Gateway into the cluster. But why wouldn't relying on a non H.A. QM in between an H.A. QM (QM3) and a multinode service (QM1 and QM2) not be an issue in your eyes?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Jun 08, 2012 4:55 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| PeterPotkay wrote: |
| Quote: |
Peter - Option G is the right choice. The considerations for high availability do not seem meaningful to me. |
Jeff, this was surprising to read. Its not that much of an issue now that I have a z/OS QM to use as a Gateway into the cluster. But why wouldn't relying on a non H.A. QM in between an H.A. QM (QM3) and a multinode service (QM1 and QM2) not be an issue in your eyes? |
Mostly because of considerations like
| Quote: |
| But those IPLs are happening when this app is not up anyway, so not an issue. For now. |
The question is, what is the impact if the gateway qmgr is down? And I don't mean, what is the impact to the application in question, I mean in general. It is usually a significant event if any queue manager is unresponsive or down. So it is expected that something will be done anyway. So it's not unreasonable to assume that effort can be taken to redirect QM3 as part of the effort to recover the impacted qmgr.
So, yes, in the abstract, one can clearly show that if one makes a single qmgr HA, one then should make all qmgrs that it talks to HA (and so on and so on and so on). But in practice, you're always going to make tradeoffs from this. So it's not a deal-breaker by any means. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Jun 08, 2012 4:59 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| PeterPotkay wrote: |
*Cons
A. The mainframe is IPL'ed on Saturday nights, so that z/OS QM is not always up. |
This is troubling. There is little/no need to IPL z/OS weekly, monthly, or even quarterly.
One of the places I worked IPL'd twice a year - once during the summer, and once again around the holidays. It was a festive event, with pizza and soft drinks; and everyone was invited. The purpose was to refresh the IPL script. Tech support and operations folks were required to attend at least one of these each year.
A z/OS qmgr should be the least of your issues.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Jun 08, 2012 5:07 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| bruce2359 wrote: |
| PeterPotkay wrote: |
*Cons
A. The mainframe is IPL'ed on Saturday nights, so that z/OS QM is not always up. |
This is troubling. There is little/no need to IPL z/OS weekly, monthly, or even quarterly. |
You cold start CICS & the other subsystem weekly but not a whole IPL.
| bruce2359 wrote: |
| One of the places I worked IPL'd twice a year - once during the summer, and once again around the holidays. It was a festive event, with pizza and soft drinks; and everyone was invited. The purpose was to refresh the IPL script. Tech support and operations folks were required to attend at least one of these each year. |
I prefer your site. My experience of the quarterly or twice yearly IPL had everyone invited (including tech support, ops, sys progs & an IBM engineer) but there was no pizza & soft drinks, just a lot of nervous pacing and chain smoked cigarettes.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Fri Jun 08, 2012 5:14 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
There is little/no reason to cold-start CICS either; except, perhaps, bad application design.
Way back when, developers would do fundamentally silly things, like load app data into CSA or TSQs at MVS/CICS startup - with no method of updating this data. This effectively forced IPLs and CICS cold-starts. The requirement for more availability forced rewrites of those apps.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Jun 08, 2012 8:35 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
[quote="mqjeff"]
| PeterPotkay wrote: |
| Quote: |
Peter - Option G is the right choice. The considerations for high availability do not seem meaningful to me. |
Jeff, this was surprising to read. Its not that much of an issue now that I have a z/OS QM to use as a Gateway into the cluster. But why wouldn't relying on a non H.A. QM in between an H.A. QM (QM3) and a multinode service (QM1 and QM2) not be an issue in your eyes? |
Mostly because of considerations like
| Quote: |
| But those IPLs are happening when this app is not up anyway, so not an issue. For now. |
But that second comment by me was made in a subsequent post. Before I posted that and you knew that, you had already dismissed the lack of H.A. concern. Anyhoo...not that big a deal now that we are dealing with a z/OS QM.
Bruce, its not every Sat nite, or even every week. But certainly more often than twice a year. I won't solve THAT issue in my project, I just gotta be aware of it and deal with it at the MQ level as best I can. And they are become less frequent. They did happen alomost every weekend. Not that they needed to from a z/OS technical perspective, I get that.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Jun 08, 2012 8:41 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| PeterPotkay wrote: |
| mqjeff wrote: |
| PeterPotkay wrote: |
| Quote: |
Peter - Option G is the right choice. The considerations for high availability do not seem meaningful to me. |
Jeff, this was surprising to read. Its not that much of an issue now that I have a z/OS QM to use as a Gateway into the cluster. But why wouldn't relying on a non H.A. QM in between an H.A. QM (QM3) and a multinode service (QM1 and QM2) not be an issue in your eyes? |
Mostly because of considerations like
| Quote: |
| But those IPLs are happening when this app is not up anyway, so not an issue. For now. |
|
But that second comment by me was made in a subsequent post. Before I posted that and you knew that, you had already dismissed the lack of H.A. concern. |
Yes. Because of
| Quote: |
| So, yes, in the abstract, one can clearly show that if one makes a single qmgr HA, one then should make all qmgrs that it talks to HA (and so on and so on and so on). But in practice, you're always going to make tradeoffs from this. So it's not a deal-breaker by any means. |
If one of your qmgrs is HA, and you believe that HA is a strong consideration for it's connections to other queue managers, then you are forced by good design to ensure that ALL queue managers are HA.
If you have already decided that it's okay that SOME queue managers are NOT HA, then the only thing you can do in this case is move the point of impact.
That is, you can choose to connect QM3 to QM2 and then make QM2 HA. Then you've got the reliable connectivity from QM3 that you want. Except that you're also sending messages to QM1, which is NOT HA. So all you've done is moved the impact of failure from messages stuck on QM3 transmission queue to messages stuck on QM2 transmission queue. You see? |
|
| Back to top |
|
|
|
|
|
|
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
