| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| WMQ General \ Discussion of a specific setup |
« View previous topic :: View next topic » |
| Author |
Message
|
| hopsala |
 Posted: Thu Oct 20, 2005 2:50 pm Post subject: WMQ General \ Discussion of a specific setup |
 |
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
This thread is a continuation of the debate with scottm in Discourse \ Why more than one QM per machine?, which, since it is a divergent subject, I chose to debate in a new thread. We continue from:
| scottm wrote: |
What I'm say is:
QMA on machine A happens to have 3 apps using it. There are 50 queues and 20 channels.
App1 decides it is going to get it's own box.
Now we have machine A and machine B.
To get app1 working, we will need to get it's MQ objects, and only it's MQ objects and re-create them over on machine B in a new QM QMB. Then we have to go through QMA and remove those objects. But, be careful and don't remove any that another app happened to share with app1 (transmission queues, channel).
But, if we had QMA and QMB both on machine A and app1 decided to get a new machine - all we'd have to do is move QMB over to machne B and we'd be done.
Of course, there are the common tasks of possibly changing channel names or transmissions queues, or channel connection strings but they are common tasks so I won't bother going through them.
And this type of QM movement seems to be more common than you'd think. Especially when it deals with WebServers. |
|
|
| Back to top |
|
 |
| hopsala |
| Posted: Thu Oct 20, 2005 2:53 pm Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
Let me elaborate on what Peter said:
| scottm wrote: |
| QMA on machine A happens to have 3 apps using it. There are 50 queues and 20 channels. |
Usually 3 apps = 3 application queues or 6 maybe (backouts), why 50?; and why 20 channels? are you connecting to 20 other qms? (this is a good example of the extra administration I was talking about when you have one qm per appl, but why here?) Onwards:
| scottm wrote: |
| Of course, there are the common tasks of possibly changing channel names or transmissions queues, or channel connection strings but they are common tasks so I won't bother going through them. |
Why change channel name or xmitq names? What IS your naming convention anyway? Plus CONNAME needn't be changed if you work with specific DNS name per QM, which is always a good idea. |
|
| Back to top |
|
|
| csmith28 |
| Posted: Thu Oct 20, 2005 3:34 pm Post subject: |
|
|
Grand Master
Joined: 15 Jul 2003
Posts: 1196
Location: Arizona
|
One thing that I notice about that discussion is that there were plenty of individuals who could come up with a variety of scenarios in which multiple MQManager on one Server were necessary or preferable due to circumstances both valid and whimsical but none of them could really say why it is better to have multiple MQManagers on one Server if it is not necessary.
One of my Production MQManagers supports over 30 client applications while interacting with 6 other MQManagers using 8 XMITQ's.
This MQManager has 180 QAliase that point to 136 (USAGE(NORMAL)QLocals, 36 QRemotes and at any given time there are 900+ SVRCONN Channels running down from 1200 due to recent tunning.
Can anyone honestly tell me that it would be better to have 30 MQManagers on one server to do the same job? Or 10 or five or even three? This MQManager moves over 4gigs of data per day.
_________________
Yes, I am an agent of Satan but my duties are largely ceremonial. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Oct 20, 2005 4:15 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
My Client Concentrator QMs have 400+ queues, 400+ SVRCONN channels running, and only ONE xmitq to any QM they need to talk to. Well one of the QMs has 2 XMITQs, a regular and a batch. Over 75 apps are happily using this.
1 app might have 1 or more request queues, 1 or more reply queues, and usually an exception queue.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| hopsala |
| Posted: Thu Oct 20, 2005 4:41 pm Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| csmith28 wrote: |
| This MQManager has 180 QAliase that point to 136 (USAGE(NORMAL)QLocals, 36 QRemotes and at any given time there are 900+ SVRCONN Channels running down from 1200 due to recent tunning. |
| PeterPotkay wrote: |
| My Client Concentrator QMs have 400+ queues, 400+ SVRCONN channels running. |
I stand corrected concerning more than one reply-queues, (which I assume are dynamic queues, yes?) - whether any client design should use WMQ clients at all, in this WebService day-and-age, is a good question indeed (though probably best raised later on); but why more than one request queues? (I prefer the terminology "service queues") And why so many SVRCONNs? Security considerations I presume?
Might I inquire what these client-oriented appls do exactly?
(Apologies for the proliferation of brackets, they are gregarious little devils.) |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Oct 20, 2005 4:54 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
No dynamic queues here. Get by Correl ID, or get any and all messages.
Client connects to QM1, and may have to send an insurance quote to any one of a multitude of backend systems, depending on the transactions. So it chooses which one to send to, and uses the appropriate request queue remote queue def.
Every app has its own SVRCONN channel, and most apps have more than one instance of their channel running. Not 400+ defined SVRCONNs, but rather 400+ total running instances.
Most of these apps are doing insurance related request replies.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| hopsala |
| Posted: Fri Oct 21, 2005 2:48 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| PeterPotkay wrote: |
| No dynamic queues here. Get by Correl ID, or get any and all messages. |
And how do they know which queue contains their replies? Parameter in INI file?
| PeterPotkay wrote: |
| Client connects to QM1, and may have to send an insurance quote to any one of a multitude of backend systems, depending on the transactions. So it chooses which one to send to, and uses the appropriate request queue remote queue def. |
So when you said 400+ queues, what did you mean exactly?
| PeterPotkay wrote: |
| Not 400+ defined SVRCONNs, but rather 400+ total running instances. |
Ok, that indeed explains it. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Oct 21, 2005 9:57 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| hopsala wrote: |
| PeterPotkay wrote: |
| No dynamic queues here. Get by Correl ID, or get any and all messages. |
And how do they know which queue contains their replies? Parameter in INI file?
|
Yes. Just like where they get the name of the queue to put the request, the hostname, the port #, the channel name, etc.
| hopsala wrote: |
| PeterPotkay wrote: |
| Client connects to QM1, and may have to send an insurance quote to any one of a multitude of backend systems, depending on the transactions. So it chooses which one to send to, and uses the appropriate request queue remote queue def. |
So when you said 400+ queues, what did you mean exactly? |
400 queues (LOCAL + REMOTE + ALIAS)
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| hopsala |
| Posted: Fri Oct 21, 2005 11:04 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| PeterPotkay wrote: |
| 400 queues (LOCAL + REMOTE + ALIAS) |
Come now, you can do better than that! 
I was asking "400 = x local + y remote + z alias" so how much is x, y, and z? And then, when you'd tell me "x=20" I'd ask - "so you have 20 different application logics running?" And if you'd say "nope just 4" I'd ask "why more than one queue per application logic?
Hope this improvised flow chart makes my question clearer  |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Oct 21, 2005 11:08 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
If one app can send to n back end systems, that one app will get n local queues for the replies, n remote queue defs for the requests, plus 1 or n exception queues.
Thus 1 app has more than 3 queues.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| hopsala |
| Posted: Fri Oct 21, 2005 11:24 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| PeterPotkay wrote: |
If one app can send to n back end systems, that one app will get n local queues for the replies, n remote queue defs for the requests, plus 1 or n exception queues.
Thus 1 app has more than 3 queues. |
First off, i'm only talking local queues - note x was local queues in my previous post - the others are quite obvious.
If app can send to n apps, it will have:
1. 1 or n queues for replies, why not use CorrlId? With your setting you have to define a local queue per application you send to, instead of just one for getting replies.
2. 1 or n exception queues - again, I personally would go with 1 using CorrlId, so that I wouldn't have to create an exception queue per application i'm sending to.
Also, I was asking about service applications - which I believe should have one ql for getting requests and possible another for backout, no more. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Oct 21, 2005 11:29 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| hopsala wrote: |
| 1. 1 or n queues for replies, why not use CorrlId? With your setting you have to define a local queue per application you send to, instead of just one for getting replies. |
This is the art to MQ Admin. With the Correl ID logic, we all only need 1 queue per Queue Manager. All apps can all share 1 single queue, and they can all just get by Corel ID, no?
The other extreme is each app gets a local reply queue for each of the 83 different transaction types, which is just as silly.
So what I have landed on is that if the reply is coming back from a different app on the other side, then we will give it a different reply queue. We then monitor depth and throughput on these queues. If one starts backing up with late orphaned replies, it is easy to see which component is having the problem.
And it makes it easy to tabulate total messages sent thru each reply queue, so you have a good idea of who is busy.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| hopsala |
| Posted: Fri Oct 21, 2005 11:47 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| PeterPotkay wrote: |
| This is the art to MQ Admin. With the Correl ID logic, we all only need 1 queue per Queue Manager. All apps can all share 1 single queue, and they can all just get by Corel ID, no? |
True It is the art in which we are the humble craftsman...
I have landed on a different plateau, slightly towards CorrlId, which you must look at from a different perspective - Main concept being that all applications are merely "receiving applications"; so in my little world, every appl has one and only one input queue, and all messages addressed to it wait on that queue. From this, CorrlId usage is understood I think.
I found this to be simplest conceptually, code and monitoring wise and gives the highest flexibility concerning WMQ definitions. Of course, it only applies if throughput allows it.
| PeterPotkay wrote: |
So what I have landed on is that if the reply is coming back from a different app on the other side, then we will give it a different reply queue. We then monitor depth and throughput on these queues. If one starts backing up with late orphaned replies, it is easy to see which component is having the problem.
And it makes it easy to tabulate total messages sent thru each reply queue, so you have a good idea of who is busy. |
But all this means you have one instance per reply-queue, instead of just one instance or a few on the same input queue - which means more monitoring, more hassle etc. And since most of the time CURDEPTH=0 anyway, I don't think you'll have a very good idea of traffic anyway, so I assume you monitor qstats?
True, a poisoned message in your scenario would seemingly do less damage, but in an aptly coded application, this argument loses validity - backout queue etc.
So to sum up, don't you think a much simpler network is more managable? (taking into account todays MQAdmin's declining technological skills...) My networks usually have only a few local queues and a few remote queues per QM, so things are kept rather at ease. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Oct 21, 2005 2:17 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Queues are very easy to make.
The more queues you have, the more options you have.
I have more flexibility with:
PUT Inhibiting
GET Inhibiting
Queue Statistics to let me know through put for a particular app
Alerting - if the queue starts backing up, the syatem pages the app who is failing to consume the messages. What if a common queue backs up. You can only automatically page 1 group - MQ Support! No thanks.
If one of the apps goes down on a common queue, and the q fills, all apps are effected by the one bad app.
Common queue, and some wise ass uses INPUT_EXCLUSIVE
etc
etc
etc
Queues are very easy to make.
They have a very small footprint on the server, especially at 5.3+
Looking at a large list of queues, all named according to strict standards, to me is very simple. Just because there are more queues does not make it more complicated. IMO, it makes it simpler.
Same thing with SVRCONNs. When I joined the team, all the apps used 1 SVRCONN. Worked fine. Till someone misbehaves, and starts taking to many connections. Then what? Now every app gets its own SVRCONN, and if I need to, I FORCE STOP their channel, leaving the shared environment safe. Well, unless you are on Linux MQ 6.0, where if you *try* to stop a SVRCONN channel, the Queue Manager locks up, and all connected clients, even MO71, hang as well. They don't even get a 2009 or a 2059! New apps tring to connect to get a 2059 at that point.
 
_________________
Peter Potkay
Keep Calm and MQ On
Last edited by PeterPotkay on Fri Oct 21, 2005 2:34 pm; edited 2 times in total |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Fri Oct 21, 2005 2:25 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
The way I look at it, queues should be used to distinguish different logical familys of messages.
A long-running server app, for example, could have two queues. A data queue and a command queue. The command queue could be used for the simple things - start, stop, reload config. The data queue would then be it's acutal business input.
But if I have, for example, a program that needs to audit several different types of transactions, then it might have a few different input queues - one for loans, one for deposits, one for...
Although if an app is reading many many different logical families of messages, one might wonder if it should be refactored into different apps - as the BigBallOfMud is an anti-pattern, not a pattern.
And if it's acting as a router process, then likely it is not doing enough logic on it's own to differentiate between different types of messages and relying on queues to perform that separation. Which is like using different directories to separate types of files... unecessarily confusing.
On the third hand, maybe you need to ensure that app a can only send transactions of type 1, and app b can only send transactions of type 2... then a separate queue for type 1 transactions from type 2 transactions makes some sense... (although alias queues handle this probably better).
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
