| Author |
Message
|
| SAFraser |
 Posted: Mon Oct 29, 2007 8:30 am Post subject: Backout Plan for MQ 6.x Upgrade |
 |
|
Shaman
Joined: 22 Oct 2003
Posts: 742
Location: Austin, Texas, USA
|
Just want to do a reality check on my Backout Plan for an upcoming MQ 6.x upgrade.
Seems to me.... That there is no possibility of a "backout in place". By that I mean: I would need to not only uninstall MQ 6.x, but also remove all traces of queue managers and then recreate them after v5.3 is installed and patched. The queue managers would have been "migrated" to the new 6.x structure when first started and would presumably not work in a v5.3 environment.
Naturally, I hope this is simply an academic exercise. I have no reason to think I would have to execute a Backout Plan. But our change management process requires it, and it is good business practice in any case.
Thanks. |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Mon Oct 29, 2007 8:35 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Take a full backup of file system before migration. (this means with qmgrs shut down)
Restore to that point in time in case of backout.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| SAFraser |
| Posted: Mon Oct 29, 2007 9:50 am Post subject: |
|
|
Shaman
Joined: 22 Oct 2003
Posts: 742
Location: Austin, Texas, USA
|
I doubt that I can get enough maintenance window for a full system backup (complicated further by another app on this server). Also, I cannot unit test the system backup/restore as the unit test server is not maintained by our outsourced data center services vendor. (i.e., I cannot replicate their backup/restore procedure.)
But it is the most sensible approach, so I will pursue it with the vendor as a possibility.
My original question still stands. I hope to try it later this week on the unit test server but would like to hear any ideas or experiences from others. |
|
| Back to top |
|
|
| bbburson |
| Posted: Mon Oct 29, 2007 10:29 am Post subject: |
|
|
Partisan
Joined: 06 Jan 2004
Posts: 378
Location: Nowhere near a queue manager
|
Shirley,
The UNIX method we tested in our lab (and thankfully have not had to use in any of our real upgrades) is to save the files under /var/mqm with
| Code: |
| find /var/mqm -follow | cpio -o > /tmp/WMQsaves |
(use the root account).
If a backout is required, after reinstalling the 5.3 executables restore the /var/mqm files using
| Code: |
| cpio -iu < /tmp/WMQsaves |
(again run this from root).
Of course all queue managers have to be down at the time of backup/restore steps. Hope this helps.
(Edited to add "u" argument to cpio restore command; without it some files (such as mqs.ini) may not be correctly restored)
Last edited by bbburson on Mon Nov 05, 2007 7:07 am; edited 1 time in total |
|
| Back to top |
|
|
| SAFraser |
| Posted: Mon Oct 29, 2007 10:35 am Post subject: |
|
|
Shaman
Joined: 22 Oct 2003
Posts: 742
Location: Austin, Texas, USA
|
I am excited with this information! I would not have come up with anything like this.
I'm going to try it on my unit test box later this week. Thanks a million. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Oct 29, 2007 3:42 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
SAFraser,
We backed up the QM with MS03, then shut MQ down, then backed up /var/mqm, then proceeded with installing MQ 6.0. We never had to use it, but if the upgrade went bad we had 2 backup methods to fall back on. Our first choice would have been to restore /var/mqm after reinstalling 5.3 since that was the best chance of making the system look as a close as possible to the original, but we had the MS03 backup just in case it really hit the fan.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| SAFraser |
| Posted: Mon Oct 29, 2007 6:32 pm Post subject: |
|
|
Shaman
Joined: 22 Oct 2003
Posts: 742
Location: Austin, Texas, USA
|
Peter,
I'm planning to store fresh copies of these on another server:
MS03 queue manager backups
amqoamd output in script form
qm.ini files
channel exit access list files
(Can you think of anything I'm missing?)
But I hadn't thought of the /var/mqm backup that you and Bruce mention. (Is bbburson's name Bruce? Or did I just make that up?) I have a unit test box and (if time permits this week), I think I'll try it. No reason that is shouldn't work; as you say, I could still rebuild all of it if necessary.
I haven't any reason to believe that this upgrade will go bad, but I sure don't want to be caught unprepared.
Thanks so much for your advice.
Shirley |
|
| Back to top |
|
|
| Michael Dag |
| Posted: Mon Oct 29, 2007 10:37 pm Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)
|
| SAFraser wrote: |
I haven't any reason to believe that this upgrade will go bad, but I sure don't want to be caught unprepared.
|
I have no doubt your upgrade will go well, but what about the applications using MQ 6, were they properly tested?
a couple of months ago I was caught by a bad app on windows and had to backout...  on windows (not mentioned yet) you need a good backup of the registry aswell...
_________________
Michael
MQSystems Facebook page |
|
| Back to top |
|
|
| bbburson |
| Posted: Tue Oct 30, 2007 6:12 am Post subject: |
|
|
Partisan
Joined: 06 Jan 2004
Posts: 378
Location: Nowhere near a queue manager
|
Shirley,
A couple of things to keep in mind if you do have to restore /var/mqm using my method described above. 1) You'll be restoring the actual queue files so you may have old messages that had been read from the queue while you were up on version 6 and/or missing messages that got put under version 6 and are not in the restored files. 2) You may encounter sequence number mismatches if you had sender/receiver channels that ran while you were up on version 6.
I remembered these little gotchas last night and wanted to make the information as complete as possible.
 ...and yes my name is... |
|
| Back to top |
|
|
| SAFraser |
| Posted: Tue Oct 30, 2007 6:25 am Post subject: |
|
|
Shaman
Joined: 22 Oct 2003
Posts: 742
Location: Austin, Texas, USA
|
Michael,
I've built out a sandbox, first with v5.3. I had the applications test on it, to be sure it worked. Now I've upgraded to v6.x and they are testing again. This is a more extensive test and, frankly, it is not going well. I think it's because the database the app is using has antique data. So I am trying to sort out whether the errors are as a result of the upgrade or just stinky data (I suspect the latter). Thanks for the tip about the registry. I'm doing Solaris first, but will move to the Windows servers soon and I wouldn't have thought of that.
Bruce,
Good points. Channel sequence numbers should be easily resolvable. Sorting out data issues should not be too bad, as little data dwells on these queues. The really important stuff is tracked by its application and can be resent if needed. But, like you said, I'll have the MS03 backups as a second option.
Thanks so much,
Shirley
ps I don't know why I doubted that your name is Bruce, I've seen it a million times. It was late, I was tired...
Thanks, fellows. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Oct 30, 2007 9:42 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Shirley,
Lots of good stuff here:
Important MQ V6.0 migration information
You mentioned exits. Be aware that when you uninstall MQ 5.3 the /var/mqm/exits dir is cleaned out. And you may have to recompile exits to work with the 64bit QM, if your platform will be converted to MQ 64bit.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| SAFraser |
| Posted: Tue Oct 30, 2007 11:48 am Post subject: |
|
|
Shaman
Joined: 22 Oct 2003
Posts: 742
Location: Austin, Texas, USA
|
I have that Migration manual printed in a three-ring binder, highlighted and flagged all over the place! Same with the Quick Beginnings manual.
The exits are SCARY to me, being truly an admin and not a developer; the word "recompile" strikes terror in my heart. I've noticed that the upgrade did not update the qm.ini file with a new path to exits. I thought that was odd, as I understood that the exits go to the exits64 folder.
First big server to be upgraded is this coming Sunday, but it doesn't use exits so I am avoiding the exits issue till next week. Then I will re-read all that I have collected and start trying to make them work in our Solaris platform.
If you think of any more 'gotchas', keep them coming. This is a most useful thread for me. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Oct 30, 2007 12:00 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Luckily the 2 exits we use (Capitalware's MQAUSX and HP's Transaction Vision) are very well supported by the vendors and they gave me versions that worked with the Solaris 64 bit QMs once I moved the exits into the exits64 dir. I didn't have to go down that scary path alone! 
On Windows not a problem because the QM is still 32 bit, but be aware they exits dir will be wiped clean so save it.
If you haven't gone through every link in that above link I posted please do. There is tons of good stuff and there is a link specific to exits as well.
So far no fallout from any of my MQ upgrades, although I did uncover 3 bugs in 6.0.2.1 that I need to get hot fixes for. Fortunatly none were show stoppers the night of the upgrade.
In addition, I have these steps at the end of my Windows plan. Some apply to UNIX too:
43. In MQ 6.0, the environment variable MQMAXERRORLOGSIZE only applies to the System MQ error logs. The QM error logs are now controlled in MQ Explorer on Windows QMs and in qm.ini on UNIX. Open up MQ 6.0 Explorer on the server you just upgraded. For each QM on that server, right click on it and select Properties. Under the “Extended” section, change the QM’s Error Log Size to 1000000 and select Apply. (see pic below)
44. Right Click on the QM and select Properties…General to get to the below screen.
In MQExplorer, you also want to make sure the Command Server Control and Channel Init Control is set to “Queue Manager”. The one right above them titled “Startup” refers to the QM itself. If this QM is under the control of Microsoft Hardware Clustering, this must be set to Manual, otherwise set it to Automatic.
45. In MQExplorer, you want to verify the Listener is set to “Queue Manager” for Control.
48. The MQ 5.3 to 6.0 upgrade has a minor bug where some local queues get their CLWLUSE attribute converted into an integer instead of a valid value (default being “LOCAL”). Restart MO71 so it realizes this is a 6.0 QM now and then, list all local queues, scroll over and sort by the Use Q attribute (Cluster Use Q (CLWLUSEQ)), and change any that have a number to “LOCAL”.
49. The upgrade process sometimes causes MQ channels to come up in a STOPPED status. Do a Channel Status for all channels on this QM and verify none are STOPPED. If there are any, right click START.
If you have any RCVR channels running with MCAUSERs coded make sure those IDs have +ctrlx and +dsp for the channels. Now that channels are objects in MQ like queues the user ID that the channel runs under needs these 2 rights to be able to RESET and RESOLVE the channel.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Oct 30, 2007 12:34 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Exits go to exits and exits64 - as determined by the DefaultExitPath parameters in mqs.ini, unless overridden in qm.ini.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| SAFraser |
| Posted: Wed Oct 31, 2007 9:18 am Post subject: |
|
|
Shaman
Joined: 22 Oct 2003
Posts: 742
Location: Austin, Texas, USA
|
Peter,
You are very generous to share steps from your upgrade plan. Several of these were new to me, and will be added to my own plan.
Jeff,
As I mentioned, I haven't starting working in earnest on the 64-bit exits. But the upgrade did not add exits64 as the default exits path to either the mqs.ini file or the qm.ini files. Just puzzled me, but I'm sure I'll find a reference to the path issue when I start studying it.
Thanks mucho,
Shirley |
|
| Back to top |
|
|
|
|
