| Author |
Message
|
| Korn |
 Posted: Wed Jul 02, 2008 7:59 am Post subject: WMQ 5.3 crashed : Need help! |
 |
|
Newbie
Joined: 26 Nov 2007
Posts: 9
|
I get the following FDC repeatedly on a production AIX 5.3 server. I have MQ v 5.3 CSD 11 installed on the server. All MQ file systems are mounted on a JFS2 file system allowing large file support. There is plenty of free disk space available for MQ ( >48 GB). The file size parameter in ulimit for the MQ admin id is also set to unlimited. This should allow me to have a queue of any size as long as I have disk left. What I do not understand is why MQ is complaining that "A file cannot be larger than the value set by ulimit". Lower down in the FDC I see that it mentions the queue name.
This error caused MQ to hang, was not getting any response to MQSC commands and had to bounce MQ to recover. Can someone please suggest what could have gone wrong here?
+-----------------------------------------------------------------------------+
| |
| WebSphere MQ First Failure Symptom Report |
| ========================================= |
| |
| Date/Time :- Tuesday July 01 01:56:56 EDT 2008 |
| Host Name :- XXXXXX (AIX 5.3) |
| PIDS :- 5724B4101 |
| LVLS :- 530.11 CSD11 |
| Product Long Name :- WebSphere MQ for AIX |
| Vendor :- IBM |
| Probe Id :- AD020000 |
| Application Name :- MQM |
| Component :- adiSetFSize |
| Build Date :- Aug 2 2005 |
| CMVC level :- p530-11-L050802 |
| Build Type :- IKAP - (Production) |
| UserID :- 00009316 (mqm) |
| Program Name :- amqzlaa0_nd |
| Process :- 00234056 |
| Thread :- 00052693 |
| QueueManager :- XXXXXXXXXXXX |
| Major Errorcode :- xecF_E_UNEXPECTED_SYSTEM_RC |
| Minor Errorcode :- OK |
| Probe Type :- MSGAMQ6119 |
| Probe Severity :- 2 |
| Probe Description :- AMQ6119: An internal WebSphere MQ error has occurred |
| (Rc=27 from ftruncate) |
| FDCSequenceNumber :- 0 |
| Comment1 :- Rc=27 from ftruncate |
| Comment2 :- A file cannot be larger than the value set by ulimit. |
| |
+-----------------------------------------------------------------------------+ |
|
| Back to top |
|
 |
| sami.stormrage |
| Posted: Wed Jul 02, 2008 9:29 am Post subject: re |
|
|
Disciple
Joined: 25 Jun 2008
Posts: 186
Location: Bangalore/Singapore
|
Then how about opening a PMR to IBM? 
_________________
*forgetting everything  *
Last edited by sami.stormrage on Wed Jul 02, 2008 10:26 am; edited 1 time in total |
|
| Back to top |
|
|
| Korn |
| Posted: Wed Jul 02, 2008 10:10 am Post subject: |
|
|
Newbie
Joined: 26 Nov 2007
Posts: 9
|
It does not make any sense for the following reasons
I do not have any damaged object.
I am at a higher CSD level (CSD 11)
I have enabled large file system support for MQ (ulimit, JFS2) as detailed in my first post. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Wed Jul 02, 2008 11:02 am Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
errno 27 = File too large.
What is the current size of the queue file ?
_________________
Glenn |
|
| Back to top |
|
|
| Korn |
| Posted: Wed Jul 02, 2008 11:32 am Post subject: |
|
|
Newbie
Joined: 26 Nov 2007
Posts: 9
|
| The current file size(size of queue on disk) is 800 MB but I have other queues larger than 2 GB. |
|
| Back to top |
|
|
| jhidalgo |
| Posted: Wed Jul 02, 2008 12:34 pm Post subject: |
|
|
Disciple
Joined: 26 Mar 2008
Posts: 161
|
| set your ulimits to unlimited, also check your file system, ran out of inodes ? |
|
| Back to top |
|
|
| Korn |
| Posted: Wed Jul 02, 2008 1:10 pm Post subject: |
|
|
Newbie
Joined: 26 Nov 2007
Posts: 9
|
ulimits were already set to unlimited as shown below and I check for inode usage when the system crashed, it was very normal
$ ulimit -a
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) 131072
stack(kbytes) 32768
memory(kbytes) 32768
coredump(blocks) 2097151
nofiles(descriptors) 2000 |
|
| Back to top |
|
|
| Korn |
| Posted: Thu Jul 03, 2008 6:36 am Post subject: |
|
|
Newbie
Joined: 26 Nov 2007
Posts: 9
|
| Can the grand master's in the forum please suggest me on how to go about preventing this from happening again and find out what caused this issue? I cannot open a PMR because we do not have extended support contract from IBM for MQ v 5.3 |
|
| Back to top |
|
|
| SAFraser |
| Posted: Thu Jul 03, 2008 10:14 am Post subject: |
|
|
Shaman
Joined: 22 Oct 2003
Posts: 742
Location: Austin, Texas, USA
|
Hmm. Asking for the grand masters. I'm not one, but I'll give you my two cents anyway.
I would wait for the queue to be empty (and if the queue is "never empty", I would work with the application team to make a special case where all the messages get consumed), and then I would delete the queue and recreate it. In fact, when I recreated it, I would name it something different and make an alias queue to point to it, so that the queue file itself would (presumably) go to a different spot on the disk.
There is absolutely no reason why this should help, but if I were "grasping at straws", that would be one crazy thing I would try.
I assume there is some good reason why your site has not upgraded to v6.0. Any chance you could at least patch to current CSD14?
Good luck. |
|
| Back to top |
|
|
|
|
