| Author |
Message
|
| meekings |
 Posted: Tue Mar 04, 2003 4:18 pm Post subject: Recovery with circular logging |
 |
|
Voyager
Joined: 28 Jun 2001
Posts: 86
Location: UK, South West
|
I know the linear vs circular logging issue has been covered extensively in this forum, but I have a question that doesn't seem to have been answered yet.
If I'm using circular logging and a queue containing persistent messages becomes "damaged", can I recover by deleting the queue and re-creating it with the same name and characteristics - if I stop and restart the qm, will the messages be recovered.
If so, I can't normally delete a queue that contains messages, so how would I do that? |
|
| Back to top |
|
 |
| dgolding |
| Posted: Tue Mar 04, 2003 11:24 pm Post subject: |
|
|
Yatiri
Joined: 16 May 2001
Posts: 668
Location: Switzerland
|
| As you said, quite extensively covered. But no, you can't recreate the queue that way - at least, the messages won't come back. I'm not too sure what you're trying to do - recreate the queue, or recover the messages? |
|
| Back to top |
|
|
| pgorak |
| Posted: Tue Mar 04, 2003 11:37 pm Post subject: |
|
|
Disciple
Joined: 15 Jul 2002
Posts: 158
Location: Cracow, Poland
|
Unfortunately, the scenario that you have described will not work. Unless you have a backup of queue manager files, you can't recover messages because media recovery is supported only for linear logging. Of course, even if you have a backup, it may be out of date.
Actually I did a test on HP-UX:
1. create a queue manager with circular logging
2. define queue QUEUE1
3. put 1000 persistent messages on QUEUE1
4. deliberately damage the queue file
5. try to access messages on QUEUE1 - results in MQRC_OBJECT_DAMAGED (2101) and AMQERR01.LOG says that you must delete QUEUE1
Piotr |
|
| Back to top |
|
|
| emiranda |
| Posted: Wed Mar 05, 2003 5:05 am Post subject: |
|
|
Disciple
Joined: 21 Nov 2002
Posts: 196
Location: Dublin, Ireland
|
Hi,
Actually, when you delete the queue, all information about that queue is gone, so even you create another one with the same name and characteristics, it's only a new queue, no relation with the old one.
_________________
Warm Regards,
EM |
|
| Back to top |
|
|
| meekings |
| Posted: Wed Mar 05, 2003 5:09 am Post subject: |
|
|
Voyager
Joined: 28 Jun 2001
Posts: 86
Location: UK, South West
|
I'm trying to make sure that messages can be recovered in the event of a system malfunction during which queue(s) are damaged, but logs are not, but don't want to use linear logging if possible - I agree with the arguments that I've seen saying disk mirroring etc with circular logging are a better solution.
So ... assuming the logs are intact, and assuming that I can exactly rebuild the queue manager (say from a script generated from saveqmgr), can I copy out the log files, delete the damaged queue manager, recreate it, copy the log files back in, restart the queue manager, and have the messages back in the queue(s)? |
|
| Back to top |
|
|
| dgolding |
| Posted: Wed Mar 05, 2003 5:14 am Post subject: |
|
|
Yatiri
Joined: 16 May 2001
Posts: 668
Location: Switzerland
|
| Nope, 'fraid not. The circular logs do not contain queue message entries, only those that are "in-flight" at the time....so when you delete the queue, you delete all the messages in the queue at the same time.... |
|
| Back to top |
|
|
| emiranda |
| Posted: Wed Mar 05, 2003 5:23 am Post subject: |
|
|
Disciple
Joined: 21 Nov 2002
Posts: 196
Location: Dublin, Ireland
|
You have to use the rcdmqimg command to write an image of an object to the log for use in media recovery, and the associated command rcrmqobj to recreate the object from this image. Such commands are only supported with linear logs.
_________________
Warm Regards,
EM |
|
| Back to top |
|
|
| dgolding |
| Posted: Wed Mar 05, 2003 5:29 am Post subject: |
|
|
Yatiri
Joined: 16 May 2001
Posts: 668
Location: Switzerland
|
| Being pedantic, strictly speaking, you only need rcdmqimg to "compress" trhe number of log files you need to keep for media recovery. If you haven't done one, you can still recover - you'll just need more (older) log files that's all.... rcdmqimg takes a snapshot of your current queue setup, so can delete older log files... |
|
| Back to top |
|
|
| meekings |
| Posted: Wed Mar 05, 2003 5:43 am Post subject: |
|
|
Voyager
Joined: 28 Jun 2001
Posts: 86
Location: UK, South West
|
I tried the scenario I described above, and it worked:
1. Create and start qm TESTQM
2. Create q TEST
3. Use saveqmgr to generate snapshot of qm config
3. Put a persistent message to q TEST
4. Stop qm
5. Copy log directory for TESTQM elsewhere
6. Delete qm
7. Create and start qm TESTQM
8. Re-configure from snaphot using runmqsc
9. Stop qm
10. Delete log directory
11. Copy in old log directory
12. Restart qm
and the message was there. So is this a valid technique, or not a recommended method of recovery? |
|
| Back to top |
|
|
| dgolding |
| Posted: Wed Mar 05, 2003 7:28 am Post subject: |
|
|
Yatiri
Joined: 16 May 2001
Posts: 668
Location: Switzerland
|
Well, it worked, so it must be valid 
IMHO you were lucky, with an "not-busy" queue manager with only one transacaction going on. In a "real" situation with other transactions and stuff happening, I think you would not be so lucky.
The official word from IBM is that circular logging will NOT allow media recovery. However, IBM also officially say that you cannot backup a running queue manager, and then expect to restore using that backup with no problems. The truth is, if nothing is "going on" transaction wise, it IS possible to restore it - I've seen it happen on production systems two or three times.... |
|
| Back to top |
|
|
| meekings |
| Posted: Fri Mar 07, 2003 7:17 am Post subject: |
|
|
Voyager
Joined: 28 Jun 2001
Posts: 86
Location: UK, South West
|
At the risk of flogging this issue to death ... I'm still trying to come up with definitive reasons for using circular or linear logging.
With circular logging, the maximum configurable log size (5.3) is 4Gb (63 log files of 64Mb). If I'm sending 10Mb persistent messages (I know this sounds a bit pathological, but my org is planning on messages much larger than this), that means that if I get a build up of more than 400 messages in total on my queues, I blow the log, right? Then my only option is linear logs? |
|
| Back to top |
|
|
| dgolding |
| Posted: Fri Mar 07, 2003 7:24 am Post subject: |
|
|
Yatiri
Joined: 16 May 2001
Posts: 668
Location: Switzerland
|
You can only have 63 log files active at any time - using circular or linear.
IMHO linear logging is not worth the administration headache it brings. It was designed for a time when disks were expensive and prone to failure. Cheap reliable disks, using mirroring, is a much surer form of media recovery.
Abut the only EXTRA protection you get from LL is when MQ internally screws up the queues. Then, you can delete the queue from the file system and use "rcrmqobj".
...and it's not the number of messages in the queue, but the number of UNCOMMITED messages that uses circular logs...
....it's up to you |
|
| Back to top |
|
|
| Troilus |
| Posted: Fri Mar 07, 2003 9:02 am Post subject: |
|
|
Apprentice
Joined: 12 Jul 2002
Posts: 28
Location: Belgium
|
| If you are serious about data integrity, linear logging is a must since it is the only way MQ garuantees media recovery. Also consider the danger of long running transactions. If one application "forgets" to commit, and you have pumped your 400 messages through the system, all applications will hang with a MQ_RC_RESOURCE_PROBLEM return code. You don't want that to happen do you. The risk is more real than you might think. Some interactive tools work under syncpoint control, so the uow stays open when the user goes to lunch, or worse, on holiday. |
|
| Back to top |
|
|
| meekings |
| Posted: Fri Mar 07, 2003 10:06 am Post subject: |
|
|
Voyager
Joined: 28 Jun 2001
Posts: 86
Location: UK, South West
|
So ... if I use linear logging I can only have 63 log files active - max 4Gb. If I have a queue with 401 x 10Mb persistent messages (ie >4Gb) on it, and the queue gets corrupted, there's no way to recover the messages?
From what I know, I'm definitely on the circular logging/disk mirroring side, but I'm trying to put together irrefutable arguments to support that viewpoint - apologies if this discussion is using too much bandwidth. The sysadmin manual devotes a lot of space to logging, but leaves me with more questions than answers. |
|
| Back to top |
|
|
| jhalstead |
| Posted: Mon Mar 10, 2003 1:22 am Post subject: |
|
|
Master
Joined: 16 Aug 2001
Posts: 258
Location: London
|
This last statement is I think a bit confused.
1)you can't put over 2Gb on a a single queue. (As of version 5.2.) This is extended for v5.3...
2) The 63 active log files referes to exactly that, the log files that are required to re-start the queue manager (related to last synpoint), if a message has been commited it can exist in any of the previous logs. You could have hundereds of linear log files floating around - all with valid data in them.
The benefit Linear Logging brings is that it allows object to be recreated from the log files (rcrmqobj) if they get damaged:
i) by a corrupt queue manager
ii)by a fastpath application
iii) by a maliceous or clumsy user.
Cheers
Jamie |
|
| Back to top |
|
|
|
|
