| Author |
Message
|
| WingCommanderBadger |
 Posted: Fri Apr 07, 2006 2:33 am Post subject: Where does my message go?! |
 |
|
Apprentice
Joined: 06 Sep 2005
Posts: 32
Location: London, UK
|
I'm currently investigating whether or not to use MQ Clustering as part of a failover solution using MQ v6. However, before I started I thought I'd best run a very simple test, the scenario is as follows:
1. 2 QMs, 'A' and 'B' in the same cluster 'CLUSTER1'
2. QM B advertises a Queue 'TEST' to the cluster
3. I put a message using QM 'A' to the 'TEST' Queue and expect it to appear on QM B's 'TEST' Queue, which it does
4. However I now want to see what happens if the QM B temporarily shuts down, so I stop it.
5. I repeat step3, and expect the message to be placed on QM A's SYSTEM.CLUSTER.TRANSMIT.QUEUE, which it does
6. I restart QM B, and expect the message to now appear on QM B's 'TEST' Queue, but it doesn't arrive and is no longer on QM A's SYSTEM.CLUSTER.TRANSMIT.QUEUE
Has anyone any ideas? Or is this normal behaviour for MQ Clustering?! I can't believe it is! |
|
| Back to top |
|
 |
| Vitor |
| Posted: Fri Apr 07, 2006 2:44 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
First place to look for any lost message is the queue manager's dead letter queue. Secondly, if the message is not persistant it could just be gone - undeiverable within a variety of timeout periods (check the manuals regarding channels) it just gets discarded.
Thirdly, and this is a personal view, clusters are not ideal for fail-over. The new attributes in v6 go some way to addressing this but clusters are intended for workload distribution and simplified administration. Check through some of the threads in this forum concerning this, but let me illustrate.
In your example, QM B goes down. The cluster "fails over", but the message is already in the cluster xmitq heading for B. Until B comes back, that message will not be processed.
Simplistic and there are ways to mitigate this, but IMHO failover is best left to HA software built for the purpose.
I feel sure others will post varying and differing views! 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Fri Apr 07, 2006 3:30 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| Vitor wrote: |
IMHO failover is best left to HA software built for the purpose.
I feel sure others will post varying and differing views! |
I agree with Vitor.
Clusters are not a failover solution - they are a loadbalancing solution or a simplified administration solution.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| WingCommanderBadger |
| Posted: Fri Apr 07, 2006 4:20 am Post subject: |
|
|
Apprentice
Joined: 06 Sep 2005
Posts: 32
Location: London, UK
|
Yep, it was my test rig setting the message persistence explicitly to 'No' rather than to 'As Queue' as I expected. All works fine now.
However, you both claim that Clustering in MQv6 is not a failover solution even though that this functionality is now officially provided by IBM (I believe it was possible via a SupportPac for v5.3). Can you provide further explanation for your reasoning though? |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Fri Apr 07, 2006 4:27 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| WingCommanderBadger wrote: |
Yep, it was my test rig setting the message persistence explicitly to 'No' rather than to 'As Queue' as I expected. All works fine now.
However, you both claim that Clustering in MQv6 is not a failover solution even though that this functionality is now officially provided by IBM (I believe it was possible via a SupportPac for v5.3). Can you provide further explanation for your reasoning though? |
WebSphere MQ Clustering has been available since v5.1.
It has never been advertised or supplied as a failover solution for providing High Availability, as far as I know. If you have material from IBM that shows this has changed for v6 - please let me know where as that's very new!
WebSphere MQ has also had support for various forms of OS level High Availability like HACMP and Veritas and MSCS and etc. These are not thes same thing as WebSphere MQ Clustering, and are intended to provide High Availability. Support for these HA systems used to be shipped in different support packs for each technology, but has now been rolled into a single Support Pack MC91.
WebSphere MQ Clustering does not provide High Availability nor is it suitable for failover for exactly the reason already posted - it can strand messages either in a S.C.T.Q or on a downed queue manager. None of the HA implementations have this issue because of shared disk.
This has been talked about lots. I agree with Vitor - "Check through some of the threads in this forum concerning this".
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| mvic |
| Posted: Fri Apr 07, 2006 4:30 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| WingCommanderBadger wrote: |
| message persistence explicitly to 'No' rather than to 'As Queue' as I expected. All works fine now. |
Glad to hear you've fixed the immediate problem.
However, allow me to make one observation: if your app decides a message is persistent (I would dare to suggest this decision is usually made in an app rather than on a queue definition) then you should explicitly set the persistence of the message in the app. Delegating the persistence decision to the queue definition (and ultimately to the administrator) may not be the best approach - what happens if an administrator changes the default persistence to non-persistent? All of a sudden your app will be creating non-persistent messages, which may not be the right thing per the system design at all.
Just a thought... |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Apr 07, 2006 4:58 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| WingCommanderBadger wrote: |
| However, you both claim that Clustering in MQv6 is not a failover solution even though that this functionality is now officially provided by IBM (I believe it was possible via a SupportPac for v5.3). Can you provide further explanation for your reasoning though? |
Just done a quick blast through the IBM site & the MQv6 pages & can't see anything that says clustering is now a supported high availability solution - is it a redbook someplace? All I can find is the MC91 Pac which, as has been stated, is support for MQ running on HACMP et al. Only change in v6 I'm aware of for clustering is the new workload balancing parameters & they caught me out!
You turn your back for 1 version and they slip all sorts of functions in.... 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Apr 07, 2006 5:07 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Someone explain to me why a non persistent message, sitting safely in the S.C.T.Q on QMA, would be discarded once QMB came back up?
The channel on QMA to QMB should have been retrying while QMB was down, so the message was safe in the S.C.T.Q. When QMB came back up, the channel to QMB would eventually start up, and should safely transport the message to QMB. I see no reason why the message would be gone.
(And there is no built in hardware clustering support for MQ, other than the support for Microsoft's MSCS, which is explained in Chapter 13 of the S.A.G.)
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| WingCommanderBadger |
| Posted: Fri Apr 07, 2006 5:07 am Post subject: |
|
|
Apprentice
Joined: 06 Sep 2005
Posts: 32
Location: London, UK
|
I think I need to be more specific regarding my use of the word 'failover'. Yes, I know that clustering has been around in previous versions of MQ, but as far as I'm aware the ability to provide an 'active-standby' arrangement where all messages are routed to a single instance of a queue unless it becomes unavavailable was not provided until MQv6. This is the functionality I'm investigating at the moment.
The argument made about persistence is an interesting one - presumably at some point you have to make a decision about who is 'in charge' of message persistence. In this case I have a preference for allowing the systems administrator to remain in charge of this rather than the apps themselves - ultimately the systems administrator has the 'bigger picture' view of the MQ network and I believe is therefore ultimately responsible for making the final decision on message persistence. There's nothing to stop a developer inadvertently coding an app that writes non-persistent messages either! |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Apr 07, 2006 5:10 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| WingCommanderBadger wrote: |
| ultimately the systems administrator has the 'bigger picture' view of the MQ network and I believe is therefore ultimately responsible for making the final decision on message persistence. |
An MQ system usually supports dozens, or hundreds or thousands of apps. There is no way the MQ admin is gonna know the right decision for all of them.
| WingCommanderBadger wrote: |
| There's nothing to stop a developer inadvertently coding an app that writes non-persistent messages either! |
But in that case its their own fault. You can't save the world from itself.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Apr 07, 2006 5:17 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| WingCommanderBadger wrote: |
I think I need to be more specific regarding my use of the word 'failover'. Yes, I know that clustering has been around in previous versions of MQ, but as far as I'm aware the ability to provide an 'active-standby' arrangement where all messages are routed to a single instance of a queue unless it becomes unavavailable was not provided until MQv6. This is the functionality I'm investigating at the moment.
|
And in fairness, I've just found the following in the v6 Queue Manager Clusters pdf, page 157 - CLWLPRTY queue attribute
"Where there are two possible destinations you can use this attribute to allow one queue manager to act as a failover" (their italics, not mine).
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| WingCommanderBadger |
| Posted: Fri Apr 07, 2006 5:19 am Post subject: |
|
|
Apprentice
Joined: 06 Sep 2005
Posts: 32
Location: London, UK
|
| Quote: |
| An MQ system usually supports dozens, or hundreds or thousands of apps. There is no way the MQ admin is gonna know the right decision for all of them. |
I would say that it's very much dependent on the system being worked on though - in this case there are not hundreds or thousands of apps and it is anticipated that the MQ Admin will indeed be able to make the right decision for them all. Having said that, like a lot of things on IT projects, this 'anticipation' may turn out to be a requirement that changes in the future. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Apr 07, 2006 5:27 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Yeah, the whole persistence discussion goes back and forth, both here and on the list serve. Ultimatly it comes down to who gets blamed when an important message that is not persistent gets discarded. I say put the blame on the developers!  Some newbie MQ Admins go and make every q in the system default persistent = Yes. Eventually enough newbie developers come in and use the Default as Q Persistense code, and then everyone wonders why the MQ system is slow.
Anyway......
I still want to know why the NP message would dissappear as described in the first post of this thread.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Apr 07, 2006 5:30 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| Vitor wrote: |
And in fairness, I've just found the following in the v6 Queue Manager Clusters pdf, page 157 - CLWLPRTY queue attribute
"Where there are two possible destinations you can use this attribute to allow one queue manager to act as a failover" (their italics, not mine). |
Ugh, horrible choice of words, since we all know that marooned messages on the down QM will not get failed over in this scenario.
But, it does offer another option for the architecture. In some cases, I'm sure it will exactly what the doctor ordered.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| WingCommanderBadger |
| Posted: Fri Apr 07, 2006 5:33 am Post subject: |
|
|
Apprentice
Joined: 06 Sep 2005
Posts: 32
Location: London, UK
|
| Quote: |
And in fairness, I've just found the following in the v6 Queue Manager Clusters pdf, page 157 - CLWLPRTY queue attribute
"Where there are two possible destinations you can use this attribute to allow one queue manager to act as a failover" (their italics, not mine). |
That's the one. I understand you can use it to assign a priority to individual queues or the Cluster Receiver channel for the whole Queue Manager. The testing I have done on my clustering prototype indicates that it works fine. |
|
| Back to top |
|
|
|
|
