| Author |
Message
|
| MQMB&WAS |
 Posted: Sun Dec 08, 2019 10:26 pm Post subject: Reasons for damaged objects? |
 |
|
Centurion
Joined: 12 Jun 2016
Posts: 130
|
Hello experts,
I had a damaged mq object and I was able to recover it with rcrmqobj.
But, how do I find the cause for the damage? How to check for the exact reason? And what are the possible causes?
Appreciate your time.
Thanks |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Mon Dec 09, 2019 5:35 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Two of my favorite causes:
- the queue-holding disk recordable media is damaged
- someone in the mqm group or with superuser rights navigated down the file system path under qmgrs/yourqmgr/queues, and deleted the queue pointer at the bottom
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Dec 09, 2019 6:05 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
There also used to be some tech note that said in extreme circumstances while using syncpoint there was a slight chance of corrupting an object (queue).
An other example would be about starting a multi-instance qmgr in a replicated data center when the primary is running and the file locks are not getting replicated...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Mon Dec 09, 2019 6:56 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| fjb_saper wrote: |
There also used to be some tech note that said in extreme circumstances while using syncpoint there was a slight chance of corrupting an object (queue).
An other example would be about starting a multi-instance qmgr in a replicated data center when the primary is running and the file locks are not getting replicated... |
MQMB&WAS, to better help you, please provide us with detailed information on your MQ configuration.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Andyh |
| Posted: Mon Dec 16, 2019 4:12 am Post subject: |
|
|
Master
Joined: 29 Jul 2010
Posts: 239
|
"There also used to be some tech note that said in extreme circumstances while using syncpoint there was a slight chance of corrupting an object (queue). "
It's possible that such a tech note existed, but it would only go to show that you shouldn't believe everything you read !!!
A damaged object reflects some inconsistency in either the memory image of a queue, or the disk image of a queue. It might be caused by something external to MQ (typically applies to an inconsistency in the disk image), or internal (an APARable bug in MQ code).
In the early days of MQ damaged objects were much more common, but nowadays reports of damaged objects are much less frequent, and more often caused by operational errors (e.g. truncating/deleting queue files).
Manual changes to any MQ files should be avoided wherever possible, for example one of the big advantages of automatic log management in MQ V9 is that it removes one past reason that customers might have had for deleting MQ files.
If you ever start to run out of file space in an MQ related file system you should be VERY WARY of modifying anything directly at the file level, for example a CLEAR QL(xxx) has a much more predictable outcome than deleting a queue file. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Mon Dec 16, 2019 2:39 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
Other possible reasons
- MQ queue file system becomes full
- OS crashes
- Critical MQ processes crash, usually due to:
- Severe resource issues (eg. running out of shared memory or semaphores, long lock waits on semaphores)
_________________
Glenn |
|
| Back to top |
|
|
| Andyh |
| Posted: Tue Dec 17, 2019 3:23 am Post subject: |
|
|
Master
Joined: 29 Jul 2010
Posts: 239
|
Sorry to disagree, but from my perspective NONE of those are acceptable reasons for a damaged MQ object. If MQ were to report a damaged object under those conditions (assuming the underlying filesystem is intact (it should be!)) I'd consider it APARable.
Regards
Andy. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Dec 17, 2019 5:52 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| Andyh wrote: |
| ... (assuming the underlying filesystem is intact ... |
File system Read/Write/Execute permission bits are set appropriately during installation, and should not need to be changed unless directed to do so by IBM product support.
Quite a while ago, one of my clients responded to a developer request to change permission bits in order to resolve an application 2035 R/C. Other clients have mistakenly added developers and other non-administrators to the mqm (or equivalent) groups.
MQ application users/groups are granted access to MQ objects by access control lists (ACLs), and NOT by RWX permission bits. Membership in the mqm (or equivalent) admin groups are for MQ administrators. mqm membership should NOT be granted to end-users, developers, and the like.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Tue Dec 17, 2019 3:13 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
| Andyh wrote: |
Sorry to disagree, but from my perspective NONE of those are acceptable reasons for a damaged MQ object. If MQ were to report a damaged object under those conditions (assuming the underlying filesystem is intact (it should be!)) I'd consider it APARable.
Regards Andy. |
Hi Andy. These reasons will not generally cause damaged MQ objects, but they might under certain circumstances. We have had several incidents like these on large production systems and there was no damage. It demonstrates that MQ is very resilient to external failures.
_________________
Glenn |
|
| Back to top |
|
|
| HubertKleinmanns |
| Posted: Wed Dec 18, 2019 2:03 am Post subject: |
|
|
Shaman
Joined: 24 Feb 2004
Posts: 732
Location: Germany
|
| gbaddeley wrote: |
| Andyh wrote: |
Sorry to disagree, but from my perspective NONE of those are acceptable reasons for a damaged MQ object. If MQ were to report a damaged object under those conditions (assuming the underlying filesystem is intact (it should be!)) I'd consider it APARable.
Regards Andy. |
Hi Andy. These reasons will not generally cause damaged MQ objects, but they might under certain circumstances. We have had several incidents like these on large production systems and there was no damage. It demonstrates that MQ is very resilient to external failures. |
I#m working with MQ for more than 20 years with more than 1000 QMgrs, and I found damaged objects in real live only three or four times.
In MQ trainings damaged objects are forced by removing the physical queue file (and recreated by "rcrmqobj"  ).
_________________
Regards
Hubert |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Dec 18, 2019 5:01 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| HubertKleinmanns wrote: |
| In MQ training, damaged objects are forced by removing the physical queue file (and recreated by "rcrmqobj" ). |
Yes, IBM's WM15n System Administration hands-on training class includes a lab exercise that demonstrates that someone with mqm (or equivalent) o/s-level authority can damage the file system, resulting in a 'damaged object' error.
| Andyh wrote: |
| ... (assuming the underlying filesystem is intact ... |
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
|
|
