| Author |
Message
|
| koushikt |
 Posted: Mon Jan 26, 2015 5:02 pm Post subject: Multiple MQGETS on the same queue |
 |
|
Novice
Joined: 26 Jan 2015
Posts: 14
|
I have a cluster of machines listening to the same queue. I used queue instead of pubsub as the cluster is identical and I don't want the message to be received more than once.
What I noticed is, only the instance of MQGET which gets called the last gets all the message. And when that fails the last but one acts as its failover and so on.
Is there any setting I need to set to make it round robin/ randomize it.
And also, is there any way to make a particular instance the master/primary instance.
I used the amqsget sample to test this. |
|
| Back to top |
|
 |
| zpat |
| Posted: Mon Jan 26, 2015 10:50 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
It's less of an issue in a real application, because while one instance is busy processing the message that it has just got, the others have a chance to get the next one.
You can't force even distribution, I don't see that as necessary anyway. When you have enough workload to keep all the servers busy processing messages it will be fairly even. When they are not all busy, it won't matter anyway.
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
| PaulClarke |
| Posted: Mon Jan 26, 2015 11:30 pm Post subject: |
|
|
Grand Master
Joined: 17 Nov 2005
Posts: 1002
Location: New Zealand
|
This is a deliberate policy of MQ and has been discussed many times before. It is more efficient for MQ to keep one thread/process 'hot' than to maximise context switching and deliver equal numbers of messages across the threads.
Cheers,
Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com |
|
| Back to top |
|
|
| zpat |
| Posted: Tue Jan 27, 2015 12:51 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
In this case, I think he is referring to multiple servers (i.e. hosts) each running a copy of the MQGET'ing application, rather to threading issues within a single instance of the application.
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
| PaulClarke |
| Posted: Tue Jan 27, 2015 1:22 am Post subject: |
|
|
Grand Master
Joined: 17 Nov 2005
Posts: 1002
Location: New Zealand
|
Ultimately it is still the same. Even if the MQGETs are originating on different boxes it comes down to multiple threads/processes on the same machine fighting over the same queue. MQ will still try to keep one thread hot by default.
Of course if the clients are remote then you have other potential issues. For example, read-ahead. Read Ahead can offer significant performance advantages over a client link however, it also has the potential to deliver lots of messages to one server whereas other servers may be a better choice. It depends on your messaging profile but for some queues it may be worth switching read-ahead off.
Now, as for the 'hot' scheduling algorithm I have a vague recollection that there is indeed a tuning parameter of some sort, either a QM.INI value or possibly an environment variable, which allows you to ask for round robin rather than hot thread processing. However, I can't remember exactly what it is; perhaps someone with a better memory (or indeed access to the code) will enlighten us.
Cheers,
Paul.
_________________
Paul Clarke
MQGem Software
www.mqgem.com |
|
| Back to top |
|
|
| zpat |
| Posted: Tue Jan 27, 2015 2:59 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Google and you shall find....
http://www-01.ibm.com/support/docview.wss?uid=swg1IZ85974
| Quote: |
The FastTransferOrder tuning parameter allows the queue manager
to deliver messages in a more round-robin manner by servicing
waiting MQGETs in FIFO order rather than the default LIFO order.
|
FastTransferOrder doesn't seem to be documented in the knowledge center though.
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
| PaulClarke |
| Posted: Tue Jan 27, 2015 3:05 am Post subject: |
|
|
Grand Master
Joined: 17 Nov 2005
Posts: 1002
Location: New Zealand
|
Well found....I did try Googling...in fact I tried all sorts of combinations of keywords but I guess I just never hit on the right ones.
_________________
Paul Clarke
MQGem Software
www.mqgem.com |
|
| Back to top |
|
|
| zpat |
| Posted: Tue Jan 27, 2015 3:09 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
mqget round robin
was my first (and lucky) attempt!
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
| tczielke |
| Posted: Tue Jan 27, 2015 6:45 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
If you kind of have the rough name of the undocumented parameter in your head, and need a little help in finding the exact name, I have found the following helpful, too.
strings /opt/mqm/lib64/*.so | grep Fast | grep Order
FastTransferOrder |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Jan 27, 2015 7:12 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
How do you find the undocumented, but not in an .ini file?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| tczielke |
| Posted: Tue Jan 27, 2015 7:18 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
They come to me in my dreams. 
No seriously, this has more to do with an undocumented ini parameter that you have run across in the past, you are not curently using, and you are trying to get the name right in your head. The parameter is more than likely referenced in the MQ executables as a string, so you can potentially leverage the strings command to interrogate what strings are in the executables and get the exact name right for a more accurate "google" search. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Jan 27, 2015 7:19 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| bruce2359 wrote: |
| How do you find the undocumented, but not in an .ini file? |
He's extracting the string data from the lib files - .so or etc. |
|
| Back to top |
|
|
| koushikt |
| Posted: Tue Jan 27, 2015 7:27 am Post subject: |
|
|
Novice
Joined: 26 Jan 2015
Posts: 14
|
Thanks. That actually worked. I added a sleep in my code to verify the behavior.
I couldn't really understand the FastTransferOrder documentation. Is that a property that is set up on the Queue when it gets created or is it an option that the Api takes somewhere? I am on websphere 7.5.
And regarding make one of the machines/threads a primary (master) is there a way for it?
I want to have another setup where only the primary gets the queue as long as its alive and when it fails one of the other machine becomes the primary. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Jan 27, 2015 7:30 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| koushikt wrote: |
And regarding make one of the machines/threads a primary (master) is there a way for it?
I want to have another setup where only the primary gets the queue as long as its alive and when it fails one of the other machine becomes the primary. |
If you have both queue managers in a CCDT the application will connect to the first one in the list that responds (your primary). If the connection fails, the reconnection uses the same technique (connect to the first one in the list that responds). The primary will not respond (obviously) so it will try the next one and connect.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| zpat |
| Posted: Tue Jan 27, 2015 7:45 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Vitor - you are confusing QMs with client app servers.
He wants one client app server (aka primary) to get all the messages unless it fails.
There is one way to do that - use the queue open attribute of exclusive. This means that only one client app connection will succeed (the first one). If it fails, others can connect (assuming they are trying to connect at intervals all the time - don't reconnect in a tight loop!).
How do you make sure the primary app server opens the queue first? - that's a trickier one to solve.
Of course if the apps servers are in a HA cluster - then the app can simply failover (and therefore only run on one host at a time).
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
|
|
