| Author |
Message
|
| blovell |
 Posted: Fri Mar 25, 2011 11:39 am Post subject: Refresh Pack 7.0.1.3 |
 |
|
Acolyte
Joined: 08 Feb 2006
Posts: 63
Location: Alpharetta, GA
|
We have been running MQ 7.0.0.1 on Red Hat ES server i386 for the past year. At some point we identified some possible Shared Memory leaks and decided to upgrade to 7.0.1.3. We have been running the 1.3 refresh pack in QA for the past few months w/o issue. Once the new refresh pack was installed many of our SDR channels are now alarming AMQ9206 events. (5-6 an hour). all connections are to various networks and were not experiencing issues prior to upgrade.
MQ event:
-------------------------------------------------------------------------------
03/25/2011 10:06:50 AM - Process(12807.1) User(mqm) Program(runmqchl)
Host(MQPROD)
AMQ9206: Error sending data to host X.X.X.X(1414).
EXPLANATION:
An error occurred sending data over TCP/IP to X.X.X.X(1414). This may be
due to a communications failure.
ACTION:
The return code from the TCP/IP(write) call was 110 X('6E'). Record these
values and tell your systems administrator.
----- amqccita.c : 2617 -------------------------------------------------------
03/25/2011 10:06:50 AM - Process(12807.1) User(mqm) Program(runmqchl)
Host(MQPROD)
AMQ9999: Channel program ended abnormally.
EXPLANATION:
Channel program 'LOCALQMGR.REMOTEQMGR' ended abnormally.
ACTION:
Look at previous error messages for channel program 'LOCALQMGR.REMOTEQMGR' in the
error files to determine the cause of the failure.
----- amqrccca.c : 921 --------------------------------------------------------
Based on the output event (110) it points to a TCP timeout during the WRITE. I would feel it was network related, however a rollback to 7.0.0.1 clears the issue.
At a complete lost. It is not creating and FDC files, below is a copy of my QM.INI.
$ cat qm.ini
#*******************************************************************#
#* Module Name: qm.ini *#
#* Type : WebSphere MQ queue manager configuration file *#
# Function : Define the configuration of a single queue manager *#
#* *#
#*******************************************************************#
#* Notes : *#
#* 1) This file defines the configuration of the queue manager *#
#* *#
#*******************************************************************#
ExitPath:
ExitsDefaultPath=/var/mqm/exits/
ExitsDefaultPath64=/var/mqm/exits64/
#* *#
#* *#
Log:
LogPrimaryFiles=50
LogSecondaryFiles=20
LogFilePages=4096
LogType=CIRCULAR
LogBufferPages=0
LogPath=/var/mqm/log/QMGR/
LogWriteIntegrity=TripleWrite
Channels:
MaxChannels=250
TCP:
KeepAlive=yes
Service:
Name=AuthorizationService
EntryPoints=13
ServiceComponent:
Service=AuthorizationService
Name=MQSeries.UNIX.auth.service
Module=/opt/mqm/lib/amqzfu
ComponentDataSize=0
SSL:
OCSPAuthentication=OPTIONAL
Any thoughts?
_________________
Bradley M. Lovell |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Fri Mar 25, 2011 12:11 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
It probably is network related, in that there is some configuration in the network that exposes a vulnerability in 7.0.1.3.
You should open a PMR. You should, before you open the PMR, take steps to recreate the issue, and document those steps. Then take an MQ trace of the issue when it fails on 7.0.1.3 and from the same system at 7.0.0.1.
You should also confirm that you didn't need to update your RedHat before moving to 7.0.1.3. You might, for example, have a lower level of the C libraries or TCP/IP stack than are supported at 7.0.1.3 (because of things like the issue you experience). |
|
| Back to top |
|
|
| blovell |
| Posted: Mon Mar 28, 2011 6:38 am Post subject: Cannot seem to find Pre-Reqs |
|
|
Acolyte
Joined: 08 Feb 2006
Posts: 63
Location: Alpharetta, GA
|
I have been looking for the kernel specs on the TCP/IP stack. I cannot seem to find specific specs for the IPV4 specs for 7.0.1.3.
I found the link to correct some of our shared memory issues, but i cannot seem to find kernel specs on the IP stack.
_________________
Bradley M. Lovell |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Mar 28, 2011 6:53 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
http://www-01.ibm.com/support/docview.wss?rs=171&uid=swg27011929
| Quote: |
# For SSL/TLS support on Red Hat Enterprise Linux V4, patch compat-libstdc++-33-3.2.3-47.3 or later is required.
# For SSL/TLS support on Red Hat Enterprise Linux V5, patch compat-libstdc++-33-3.2.3-61 or later is required.
# For SSL/TLS support on Red Hat Enterprise Linux V6, patch compat-libstdc++-33-3.2.3-69 or later is required. |
This may not specifically apply, but again check your overall RedHat level. |
|
| Back to top |
|
|
| blovell |
| Posted: Fri May 27, 2011 12:01 pm Post subject: Red Hat Kernel Params |
|
|
Acolyte
Joined: 08 Feb 2006
Posts: 63
Location: Alpharetta, GA
|
All of the Kernel Params regarding the TCPIP stack appear to be valid. I am capturing some before and after details and hoping I can get an answer.
_________________
Bradley M. Lovell |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri May 27, 2011 1:57 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
You should also look into the fixes for 7.0.1.4 AND 7.0.1.5 as both are out and then decide whether you want them. 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| blovell |
| Posted: Mon Jun 20, 2011 9:50 am Post subject: 7.0.1.5 |
|
|
Acolyte
Joined: 08 Feb 2006
Posts: 63
Location: Alpharetta, GA
|
I am going to install 7.0.1.5 in prepro with the hopes it will correct. Results to be posted shortly.
_________________
Bradley M. Lovell |
|
| Back to top |
|
|
| blovell |
| Posted: Mon Aug 27, 2012 11:35 am Post subject: |
|
|
Acolyte
Joined: 08 Feb 2006
Posts: 63
Location: Alpharetta, GA
|
By shortly i mean a couple of months later. 
The upgrade to 7.0.1.5 has done nothing for the 9206 event.
_________________
Bradley M. Lovell |
|
| Back to top |
|
|
| mvic |
| Posted: Fri Sep 07, 2012 5:11 pm Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| blovell wrote: |
| The upgrade to 7.0.1.5 has done nothing for the 9206 event. |
That's because the error is coming from the network, so is unrelated to the software running on the machine where this error is logged.
"The return code from the TCP/IP(write) call was 110 X('6E')."
I think 110 (x6E) is ETIMEDOUT on Linux (ask your Linux OS support person). So maybe you have some connectivity problem, the remote server is down for a reboot, etc. |
|
| Back to top |
|
|
| blovell |
| Posted: Mon Oct 29, 2012 11:39 am Post subject: MQ Channels complaining 9206 too all of my remote QMGR's. |
|
|
Acolyte
Joined: 08 Feb 2006
Posts: 63
Location: Alpharetta, GA
|
My 7.0.1.3 QMGR talks to multiple remote QMGR's. Since the upgrade i receive the below events to all remote QMGR's. These reside on different networks, accessed through multiple ingress connection points to my internal infrastructure.
I agree it has to be network related, but so far looking for needle in haystack. Odd I don't see similar behavior prior to refresh pack.
_________________
Bradley M. Lovell |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Oct 29, 2012 3:12 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
and a telnet to the host + mqport works fine?
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| blovell |
| Posted: Tue Nov 13, 2012 10:20 am Post subject: |
|
|
Acolyte
Joined: 08 Feb 2006
Posts: 63
Location: Alpharetta, GA
|
That is correct. Telnet to the complaining IP/Port results in the ^escape character response..
My suspicion is that their is a timer for packet return that is manipulated by the refresh pack (to a smaller value) that is exposing this.
While the underlying root cause is most likely network latency, this value was adjusted with refresh pack, and my hope would be to adjust it to ignore. (It obviously isn't causing that much impact.) I just want the alarm to go away.
_________________
Bradley M. Lovell |
|
| Back to top |
|
|
|
|
