| Author |
Message
|
| tczielke |
 Posted: Sat Jul 30, 2016 5:14 am Post subject: 8.0.0.4 amqrrmfa generating large FDC in /var/mqm/errors |
 |
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
I got paged last night at 2:00 am (yeah!  ) about both of our full repositories having the /var/mqm/ filesystem at 100%. When I looked at the FDC file, the amqrrmfa had written an FDC in /var/mqm/errors that had completely filled that filesystem. I am going to open a PMR about it on Monday, but just curious if anyone else has run across something like this at 8.0.0.4. This was on Solaris 10 SPARC.
_________________
Working with MQ since 2010. |
|
| Back to top |
|
 |
| hughson |
| Posted: Sat Jul 30, 2016 1:48 pm Post subject: |
|
|
Padawan
Joined: 09 May 2013
Posts: 1959
Location: Bay of Plenty, New Zealand
|
Might be worth including a few details about the FDC apart from that it was "large".
Cheers
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Sat Jul 30, 2016 2:16 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
The first 50 or so lines of the FDC will have the probe point identified, at least.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| tczielke |
| Posted: Mon Aug 01, 2016 6:58 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
|
| Back to top |
|
|
| tczielke |
| Posted: Wed Aug 03, 2016 4:59 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
To close the loop on this, the full respositories were reporting AMQ9469 errors for 5 partial repositories in the cluster. It had to do with not receiving an update from those PRs. There was no connectivity issues between the PRs and FRs, so not sure why that was happening. I chose to run a REFRESH CLUSTER (which is a disruptive action!) on the PRs and that made the AMQ9469 and the FDC errors go away on the FRs.
_________________
Working with MQ since 2010.
Last edited by tczielke on Sun Aug 07, 2016 4:19 am; edited 1 time in total |
|
| Back to top |
|
|
| tczielke |
| Posted: Fri Aug 05, 2016 5:08 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
I would also recommend running a validation step like the following when you do have to run a REFRESH CLUSTER on a partial repository.
1) Get a list of all the clustered queues on the PR.
2) Run the REFRESH CLUSTER. Wait until phase 2 is complete (you do this by validating that the queue depth on the SYSTEM.CLUSTER.COMMAND.QUEUE is consistently zero).
3) Run a program (you will probably have to write this program) that will then connect to your PR queue manager and loops with logic that just does an MQOPEN on the list of clustered queues and reports any non-zero reason codes for an MQOPEN. Have this check run for a little bit to ensure that the PR is functioning properly with the FR in recognizing existing queues in the cluster.
_________________
Working with MQ since 2010.
Last edited by tczielke on Sun Aug 07, 2016 4:19 am; edited 1 time in total |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sat Aug 06, 2016 10:50 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| tczielke wrote: |
| To close the loop on this, the full respositories were reporting AMQ9469 errors for 5 partial repositories in the cluster. It had to do with not receiving an update from those PRs. There was no connectivity issues between the PRs and FRs, so not sure why that was happening. I chose to run a REFRESH SECURITY (which is a disruptive action!) on the PRs and that made the AMQ9469 and the FDC errors go away on the FRs. |
AHA: so there was a potential communication issue between the PR and the FR that was solved by running the REFRESH SECURITY command...
Did membership of the mcauser in any of the groups change? Was that change relevant to the security set up of the queue manager? 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| tczielke |
| Posted: Sun Aug 07, 2016 4:21 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
I apologize for the confusion. I went back and corrected the posts so the say REFRESH CLUSTER, instead of REFRESH SECURITY. What was actually performed and what I meant to type was REFRESH CLUSTER!
_________________
Working with MQ since 2010. |
|
| Back to top |
|
|
|
|
