| Author |
Message
|
| BBM |
 Posted: Thu Feb 23, 2006 1:36 pm Post subject: Dilemma over queue manager rebuild |
 |
|
Master
Joined: 10 Nov 2005
Posts: 217
Location: London, UK
|
Hi,
I have a big dilemma. It has been decided that we are going to rebuild our W2K box which houses our MQ 5.3.7 queue manager.
The decision to rebuild this box is out of my hands and is to bring the machine in line with other boxes in my organisation, and to allow the application of some MS-DTC hotfixes.
The queue manager on the box is THE most important queue manager in the organisation. Suffice to say this box handles a huge amount of very very important messages per day, it is imperative that we do not lose a single message.
I know the traditional approach would be use saveqmgr.exe to save down the defs, edit the file to remove any problem defs and use amqoamd.exe in a similiar way.
With our QM we have proprietry security exits, a channel table and over 60 unique client connections as well as a fair number of ordinary channels.
The problem I have is if the rebuild takes longer than expected I am unsure what capacity for message buffering all our upstream apps and queue managers have. Our QM runs 24x7 and any downtime results in buildups of messages everywhere. So I'm looking for a quick solution to replicate the QM.
I know by copying the file system elsewhere, reinstalling Windows then MQ, creating a QM with the same name, then copying back the QMGRS and LOGS directory over the top I should be able to get the QM running without losing a message. Has anyone done this successfully? I know its the preferred method on UNIX for instance.
I do not have the luxury of stopping channels and waiting for messages to flush out unfortunately - as this could take ages due to some of the receiving apps being batch processors.
Oh and to further complicate things the box is clustered using MSCS.
So any advice? I know some of you will no doubt tell me to rebuild using scripts and amqoamd - but that is not really an option (although I'll certainly being backing using this method).
Thanks as always.
BBM |
|
| Back to top |
|
 |
| PeterPotkay |
| Posted: Thu Feb 23, 2006 2:44 pm Post subject: Re: Dilemma over queue manager rebuild |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| BBM wrote: |
| So any advice? I know some of you will no doubt tell me to rebuild using scripts and amqoamd - but that is not really an option |
I still don't understand why you would not want to do it this way. The right way.
| BBM wrote: |
| Our QM runs 24x7 and any downtime results in buildups of messages everywhere. |
Uh, yeah. That's why you have MQ.
| BBM wrote: |
| I know by copying the file system elsewhere, reinstalling Windows then MQ, creating a QM with the same name, then copying back the QMGRS and LOGS directory over the top I should be able to get the QM running without losing a message. Has anyone done this successfully? I know its the preferred method on UNIX for instance. |
Its a hack, I don't think its preferred over a nice and clean fresh install done properly front to back.
| BBM wrote: |
| The decision to rebuild this box is out of my hands |
Good, then tell whoever is charge this is how long it will take you to do things the right way, and let them fight it out with the customers.
I too have a QM running on a MSCS cluster dealing with 100s of thousands of messages everyday. I feel your pain.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Feb 23, 2006 3:35 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| BBM wrote: |
| Oh and to further complicate things the box is clustered using MSCS. |
In fact that should make things much more simple -- assuming that the queuemgr is MSCS clustered.
a) move the MSCS to the failover
b) decluster
c) replace the machine
d) recluster with new machine
e) move back from failover
f) run
Outage should then be minimal....
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mvic |
| Posted: Thu Feb 23, 2006 3:37 pm Post subject: Re: Dilemma over queue manager rebuild |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| BBM wrote: |
| I know the traditional approach would be use saveqmgr.exe to save down the defs, edit the file to remove any problem defs and use amqoamd.exe in a similiar way. |
I've not used these myself, so can't offer a personal recommendation. Just to probe a little, is there a reason you doubt the "traditional approach" (to use your own words)?
| PeterPotkay wrote: |
| BBM wrote: |
| I know by copying the file system elsewhere, reinstalling Windows then MQ, creating a QM with the same name, then copying back the QMGRS and LOGS directory over the top I should be able to get the QM running without losing a message. Has anyone done this successfully? I know its the preferred method on UNIX for instance. |
Its a hack, I don't think its preferred over a nice and clean fresh install done properly front to back. |
Theoretically, what BBM suggests should work, if done properly.
Here's my rationale. On *ix, a stopped queue manager is just files on disk. It's just data, waiting to be brought back to "life" the next time you run strmqm.
Similarly, on Windows, a stopped queue manager is just files on disk and config in the registry.
(NB. the security settings on the files - and in the Windows case, also the security settings on the registry data - are also important, and should be regarded as vital to the security of the data).
Reliably backing up the data in the registry is the weak point in this scheme. Maybe there is a way of doing this while preserving security information, but I'm not aware of it.
(Note : If one is going to take a copy of the MQ data directories, all MQ processes must be stopped first (and there be no chance they will restart automatically while files are being copied)).
(Note : disclaimer : I have only ever used this method successfully on *ix. I've never begun to try on Windows... sorry ) 
BBM, I hope you are given time to perform a dry run of the whole build-and-reinstall, whatever method you eventually choose. Doing it all live with no dry run leaves nowhere to hide if things go wrong.
Kind regards
EDIT: added disclaimer and doubts about backing up the registry |
|
| Back to top |
|
|
| BBM |
| Posted: Thu Feb 23, 2006 4:17 pm Post subject: |
|
|
Master
Joined: 10 Nov 2005
Posts: 217
Location: London, UK
|
Hi,
To answer a few of the questions:
I don't want to use the traditional approach because :
a.) I may have to wait a long time for the messages to clear down - probably too long in our case. But admittedly this needs to be investigated fully.
b.) With the 60 or so client applications connecting along with around 20 additional bindings I would have to have a lot of teams on standby globally - monitoring message buildup, resetting channels etc.
I cannot unfortunately rebuild the one cluster node whilst I fail over to the other (a great idea though thx) because the new operating system is going to be slightly but significantly different. Although I will investigate this with our Wintel team to see if it is any way possible. Seems a shame to not capitalise on the MQ data being on a shared disk.
Unfortunately the person in charge is more concerned with the other aspects of the build than with MQ.
Thanks as always for all the suggestions... I'm off to try the dry run on VMware.
Cheers
BBM |
|
| Back to top |
|
|
| mvic |
| Posted: Thu Feb 23, 2006 4:21 pm Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| BBM wrote: |
| I'm off to try the dry run on VMware. |
What approach are you going to dry-run? |
|
| Back to top |
|
|
| BBM |
| Posted: Mon Feb 27, 2006 2:56 pm Post subject: |
|
|
Master
Joined: 10 Nov 2005
Posts: 217
Location: London, UK
|
I'm dry running all approaches.
Interestingly, the regsitry entries seem to be kept in the config directory - I'm currently testing this to make sure it has all the settings. |
|
| Back to top |
|
|
| markt |
| Posted: Mon Feb 27, 2006 10:24 pm Post subject: |
|
|
Knight
Joined: 14 May 2002
Posts: 508
|
Reinstalling Windows will change the SIDs associated with any local userids. So things that were previously owned by "mqm" will not necessarily be accessible to a different "mqm". Any local userids defined to the OAM would also have the wrong permissions (as it's SID-based).
And this would not be a supported configuration: any problems you have further down the road might get bounced by IBM service as they wouldn't know whether it was caused by an incorrect setup. |
|
| Back to top |
|
|
| HubertKleinmanns |
| Posted: Tue Feb 28, 2006 11:54 am Post subject: |
|
|
Shaman
Joined: 24 Feb 2004
Posts: 732
Location: Germany
|
| BBM wrote: |
Hi,
To answer a few of the questions:
I don't want to use the traditional approach because :
a.) I may have to wait a long time for the messages to clear down - probably too long in our case. But admittedly this needs to be investigated fully.
b.) With the 60 or so client applications connecting along with around 20 additional bindings I would have to have a lot of teams on standby globally - monitoring message buildup, resetting channels etc.
I cannot unfortunately rebuild the one cluster node whilst I fail over to the other (a great idea though thx) because the new operating system is going to be slightly but significantly different. Although I will investigate this with our Wintel team to see if it is any way possible. Seems a shame to not capitalise on the MQ data being on a shared disk.
Unfortunately the person in charge is more concerned with the other aspects of the build than with MQ.
Thanks as always for all the suggestions... I'm off to try the dry run on VMware.
Cheers
BBM |
Did you think about MQ clustering? You could set up the new system - independent from the existing one. Then set up a MQ cluster which contains the old an new system. Put disable all queues on the old system and wait, until the queues are empty. Then you can stop the old system.
The only problem may be, that for a while both systems run in parallel - but you want 7x24 h  .
_________________
Regards
Hubert |
|
| Back to top |
|
|
| BBM |
| Posted: Tue Feb 28, 2006 12:02 pm Post subject: |
|
|
Master
Joined: 10 Nov 2005
Posts: 217
Location: London, UK
|
Great News!
thanks for all the helpful replies... I've been having over the last few days trying out different combinations of the suggestions.
For the record, it seems you can move the data around without any visible side effects (although there may well be some)..
Someone in my organistaion has seen sense and has decided that to do the CSD upgrade and hotfixes for MSDTC we do not have to carry out the rebuild! I am very happy with this decision. We can plan for the rebuild at a later date and do it the right way.
Now, I just need to go and figure out the correct way of applying a CSD to a MSCS cluster - I'm guessing take active node offline CSD it then the passive node, but hey that'll be in the manual.
Thanks again! |
|
| Back to top |
|
|
|
|
