| Author |
Message
|
| mgm |
 Posted: Sun Apr 25, 2010 5:26 am Post subject: Consumer application & multi instance cluster queue |
 |
|
Apprentice
Joined: 14 Dec 2007
Posts: 48
Location: anywhere
|
Hi All,
We have 4 queue managers sharing one cluster with multiple instance queue. The sender application can connect to a specific queue manager and rely on load balancing algorithm to route the message across the 4 QMs.
We need to have similar approach of the consumer application, where the app connects only to one of the queue manages and get the messages from the other instances of the input queue. Any ideas ? |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Sun Apr 25, 2010 5:42 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
No. While MQPUTs may be to local or remote queues, all MQGETs are local. The consuming application muse be connected to the qmgr where the message resides (in a queue).
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sun Apr 25, 2010 6:04 am Post subject: Re: Consumer application & multi instance cluster queue |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| mgm wrote: |
Hi All,
We have 4 queue managers sharing one cluster with multiple instance queue. The sender application can connect to a specific queue manager and rely on load balancing algorithm to route the message across the 4 QMs.
We need to have similar approach of the consumer application, where the app connects only to one of the queue manages and get the messages from the other instances of the input queue. Any ideas ? |
You need to look at this with different eyes.
You have multiple choices:
- An instance of the consumer application paired to the qmgr / queue instance where it gets its messages from (i.e. 4 consumer app instances)
- At least 2 instances of the consumer app (no single point of failure) connecting in client mode to all 4 qmgrs hosting the queue instances feeding the app its messages
Think about it. Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mgm |
| Posted: Sun Apr 25, 2010 6:38 am Post subject: |
|
|
Apprentice
Joined: 14 Dec 2007
Posts: 48
Location: anywhere
|
Thanks for the quick reply.
I've heard that a gateway queue manager could be used but couldn't find any proper documentation for that. Appreciate if someone let me know if it could be another option or it's something else. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Sun Apr 25, 2010 7:12 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
The term gateway qmgr is usually used to indicate that a qmgr that is part of a cluster also has non-clustered point-to-point channels to a(nother) qmgr not part of the cluster.
Apps on the non-cluster qmgr wants/needs to get messages into a queue that is part of the cluster; and MQPUTs the messages to a QR definition that sends them into the cluster via the gateway qmgr.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| mgm |
| Posted: Sun Apr 25, 2010 7:21 am Post subject: |
|
|
Apprentice
Joined: 14 Dec 2007
Posts: 48
Location: anywhere
|
| bruce2359 wrote: |
The term gateway qmgr is usually used to indicate that a qmgr that is part of a cluster also has non-clustered point-to-point channels to a(nother) qmgr not part of the cluster.
Apps on the non-cluster qmgr wants/needs to get messages into a queue that is part of the cluster; and MQPUTs the messages to a QR definition that sends them into the cluster via the gateway qmgr. |
So how to configure a consumer app to get messages from a cluster QM through gateway QM |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Sun Apr 25, 2010 7:41 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
The usual request-reply model can have the requesting application on any qmgr inside our outside the cluster. If you are not familiar with the request-reply model, please refer to the APR and APG manuals.
The request-reply model presumes that the producer of the message creates a reply-to-queue (where the replying app will send its reply msg), puts the request message to the queue (local or remote) where the consuming app resides. The consumer app gets the request message, and sends a reply msg back to the producer.
Again, there is no such thing as a remote get - the consuming application must be mqconnected to the qmgr where the queue exists. If your app is an mq client, it is possible to mqconnect directly to the qmgr for both put and get, but this is not the common method.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| sedmq |
| Posted: Thu Jun 24, 2010 5:49 am Post subject: Re: Consumer application & multi instance cluster queue |
|
|
Newbie
Joined: 24 Jun 2010
Posts: 2
|
| fjb_saper wrote: |
At least 2 instances of the consumer app (no single point of failure) connecting in client mode to all 4 qmgrs hosting the queue instances feeding the app its messages
|
I don't see how you can load-balance consumption of messages from n queue managers without either:
1.) n instances of the consumer app connected to queue managers in a 1-to-1 fashion (your other suggestion)
2.) major application changes to each consumer app so that it will maintain multiple concurrent open connections, each to a different queue manager
Were you suggesting option 2? I still don't understand why MQ will round-robin a PUT but not a GET. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Jun 24, 2010 6:37 am Post subject: Re: Consumer application & multi instance cluster queue |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| sedmq wrote: |
| I still don't understand why MQ will round-robin a PUT but not a GET. |
Because it's not written to do that.
Because it's not necessary to do that. If you round-robin the PUT, then you have distributed work to multiple locations. Why would you possibly want to then UNDISTRIBUTE that work by reading it all from the same location? |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Jun 24, 2010 6:51 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| I still don't understand why MQ will round-robin a PUT but not a GET. |
Consuming applications MQCONNect to a qmgr, then MQOPEN a queue, then MQGET (with WAIT, usually) - whether there is a message to consume or not.
If you want to have the consumer app to come to life when a message arrives (and wherever it arrives, then you need to look at triggering the queue.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| sedmq |
| Posted: Thu Jun 24, 2010 7:44 am Post subject: Re: Consumer application & multi instance cluster queue |
|
|
Newbie
Joined: 24 Jun 2010
Posts: 2
|
| mqjeff wrote: |
| Why would you possibly want to then UNDISTRIBUTE that work by reading it all from the same location? |
I need persistence for my messages, and my producers and consumers are much faster at handling messages than a single queue manager.
In general, it would be nice to have an application stack (likely HA) which talks to many queue managers, the number of which depends solely on the performance I need from MQ. Does every developer who runs into a performance problem with persisted messages then deploy many instances of the application in addition to many queue managers?
Maybe I misunderstand the point of load balancing with MQ - is it really for spreading application load rather than messaging load? |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Jun 24, 2010 8:09 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Quote: |
| Does every developer who runs into a performance problem with persisted messages then deploy many instances of the application in addition to many queue managers? |
Cluster queues allow spreading of the workload to many consumers. So, yes, this is a method of dealing with performance bottlenecks.
| Quote: |
| Maybe I misunderstand the point of load balancing with MQ - is it really for spreading application load rather than messaging load? |
It does both at the same time.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Thu Jun 24, 2010 4:01 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
| Quote: |
| Does every developer who runs into a performance problem with persisted messages then deploy many instances of the application in addition to many queue managers? |
No, but multiple app instances and qmgrs is a valid technique for achieving scalability and improved availability.
MQ performance is not usually a bottleneck. I find that 99% of performance problems are due to poor application design and inappropriate usage of MQ. Maybe you need to take a closer look at MQ.
_________________
Glenn |
|
| Back to top |
|
|
|
|
