| Author |
Message
|
| mmmustafa |
 Posted: Wed May 18, 2011 12:50 am Post subject: Websphere MQ v7 on windows - Sender channel Issue AMQ 9209 |
 |
|
Newbie
Joined: 16 May 2011
Posts: 6
|
I have installed MQ v7 six months back on windows server 2k8 64bit.
Previously it was working fine .
But from last few days it is showing some problems.
The sender channel automatically goes on retrying -> Binding->running mode multiple times in an hour.
In the error logs it is showing AMQ 9209 error.
Iahve checked for network issue , but ping and telnet commands are working properly.
Please help |
|
| Back to top |
|
 |
| exerk |
| Posted: Wed May 18, 2011 1:09 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
What do the logs at each state? What, if anything, has changed at each queue manager? What network changes, if any, have been made recently?
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed May 18, 2011 1:25 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
And what firewall changes if any have been made recently?
Are your channels triggered or are they always running?
Have you recently added SSL into the mix? What is the key renegotiation interval?
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mmmustafa |
| Posted: Wed May 18, 2011 1:38 am Post subject: |
|
|
Newbie
Joined: 16 May 2011
Posts: 6
|
| exerk wrote: |
| What do the logs at each state? What, if anything, has changed at each queue manager? What network changes, if any, have been made recently? |
---------------------
No changes in the queue manager has been made. And there has not been any changes made in the network.
---log file------
5/13/2011 11:09:42 - Process(5696.1) User(MUSR_MQADMIN) Program(runmqchl.exe)
Host(ADCRB)
AMQ9209: Connection to host '10.30.237.11(1843)' closed.
EXPLANATION:
An error occurred receiving data from '10.30.237.11(1843)' over TCP/IP. The
connection to the remote host has unexpectedly terminated.
ACTION:
Tell the systems administrator.
----- amqccita.c : 3473 -------------------------------------------------------
5/13/2011 11:09:42 - Process(5696.1) User(MUSR_MQADMIN) Program(runmqchl.exe)
Host(ADCRB)
AMQ9999: Channel program ended abnormally.
EXPLANATION:
Channel program '50119_MP1CH01M' ended abnormally.
ACTION:
Look at previous error messages for channel program '50119_MP1CH01M' in the
error files to determine the cause of the failure.
----fdc file------------------
+-----------------------------------------------------------------------------+
| |
| WebSphere MQ First Failure Symptom Report |
| ========================================= |
| |
| Date/Time :- Wed May 11 2011 18:27:13 India Standard Time |
| UTC Time :- 1305118633.611000 |
| UTC Time Offset :- 330 ((UNKNWOWN) |
| Host Name :- ADCRB |
| Operating System :- Windows Server 2008 Server Enterprise x64 Edition, |
| Build 6002: SP2 |
| PIDS :- 5724H7220 |
| LVLS :- 7.0.1.3 |
| Product Long Name :- WebSphere MQ for Windows |
| Vendor :- IBM |
| Probe Id :- PS017077 |
| Application Name :- MQM |
| Component :- psiProcessProxySubs |
| SCCS Info :- base/src/cmqxzmuc.c, 1.168.2.3 |
| Line Number :- 6051 |
| Build Date :- Aug 18 2010 |
| CMVC level :- p701-103-100818 |
| Build Type :- IKAP - (Production) |
| UserID :- MUSR_MQADMIN |
| Process Name :- C:\Program Files (x86)\IBM\WebSphere MQ\bin\amqzmu |
| r0.exe |
| Addressing mode :- 32-bit |
| Process :- 00004672 |
| Thread :- 00000011 |
| QueueManager :- 50119_MP1QM01G |
| UserApp :- FALSE |
| ConnId(1) IPCC :- 52 |
| ConnId(2) QM :- 55 |
| Last HQC :- 4.0.0-22064 |
| Last HSHMEMB :- 0.0.0-0 |
| Major Errorcode :- MQRC_CONNECTION_BROKEN |
| Minor Errorcode :- OK |
| Probe Type :- INCORROUT |
| Probe Severity :- 4 |
| Probe Description :- AMQ6125: An internal WebSphere MQ error has occurred. |
| FDCSequenceNumber :- 0 |
| Arith1 :- 2009 7d9 |
| Comment1 :- |
| |
| Comment2 :- |
| |
| Comment3 :- 50119_MP1QM01G |
| |
| |
+-----------------------------------------------------------------------------+ |
|
| Back to top |
|
|
| mmmustafa |
| Posted: Wed May 18, 2011 1:47 am Post subject: |
|
|
Newbie
Joined: 16 May 2011
Posts: 6
|
| fjb_saper wrote: |
And what firewall changes if any have been made recently?
Are your channels triggered or are they always running?
Have you recently added SSL into the mix? What is the key renegotiation interval?
Have fun |
A new firewall has been installed recently.
And channels are in always running state.
No we havent added any new ssl thing into mix. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed May 18, 2011 6:17 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Firewalls have been know to look at a running MQ channel and if there have not been any application messages moving over it for x minutes, consider it a stale connection and kill it. Even if you are using Heartbeats, those or only 12 bytes I think and at least here were not enough to fool the firewall.
The firewall guys were not willing to drop the time they would consider this connection "dead" so all we did was set the DISCINT to a value one minute less than their setting. If the channel was moving app messages, the firewall left it alone. When things quieted down, the DISCINT kicked in and gracefully ended the channel before the firewall got involved.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| mmmustafa |
| Posted: Wed May 18, 2011 8:19 pm Post subject: |
|
|
Newbie
Joined: 16 May 2011
Posts: 6
|
| PeterPotkay wrote: |
Firewalls have been know to look at a running MQ channel and if there have not been any application messages moving over it for x minutes, consider it a stale connection and kill it. Even if you are using Heartbeats, those or only 12 bytes I think and at least here were not enough to fool the firewall.
The firewall guys were not willing to drop the time they would consider this connection "dead" so all we did was set the DISCINT to a value one minute less than their setting. If the channel was moving app messages, the firewall left it alone. When things quieted down, the DISCINT kicked in and gracefully ended the channel before the firewall got involved. |
Where will i find the parameter DISCINT. And how to get the setting value of firewall? |
|
| Back to top |
|
|
| garyprmr |
| Posted: Thu May 19, 2011 2:18 am Post subject: |
|
|
Acolyte
Joined: 03 Sep 2005
Posts: 74
|
Dis channel(ChannelName) you can get the DISCNT interval parameter.
alter command you can change it .
Your Firewall and network folks should be able to provide the other part of information.
Thanks |
|
| Back to top |
|
|
| mmmustafa |
| Posted: Mon May 30, 2011 12:28 am Post subject: |
|
|
Newbie
Joined: 16 May 2011
Posts: 6
|
| garyprmr wrote: |
Dis channel(ChannelName) you can get the DISCNT interval parameter.
alter command you can change it .
Your Firewall and network folks should be able to provide the other part of information.
Thanks |
We had applied a fix pack 7.0.1.5 and the problem was resloved but after 5-6 days the same problem is coming again.
Please help. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon May 30, 2011 3:48 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Something changed in your environment when the problem came back.
Did somebody make a change to the firewall again? Or accidentally reverse some change there?
You might find that switching to triggered channels may be more forgiving of problems than having your channels in an "always running" state...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| mmmustafa |
| Posted: Mon May 30, 2011 8:01 pm Post subject: |
|
|
Newbie
Joined: 16 May 2011
Posts: 6
|
| fjb_saper wrote: |
Something changed in your environment when the problem came back.
Did somebody make a change to the firewall again? Or accidentally reverse some change there?
You might find that switching to triggered channels may be more forgiving of problems than having your channels in an "always running" state... |
I have investigated a pattern in the error. I think it might help .
When there is a lot of transaction going on, the error doesn't come but when there is some idle time between transactions , the problem comes frequently.
I have gone through the settings of firewall but everything seems okay.
Can anyone please tell
What is the corresponding parameter in the firewall(syntesia firewall) for DISCINT parameter of Webspher MQ. |
|
| Back to top |
|
|
| exerk |
| Posted: Tue May 31, 2011 1:47 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| mmmustafa wrote: |
| ...Can anyone please tell what is the corresponding parameter in the firewall(syntesia firewall) for DISCINT parameter of Webspher MQ. |
Ask your firewall people. They'll know the time-out parameter for connections and can up it accordingly for the IPs for your queue managers. Obviously when you have traffic the connection is not viewed as idle but they may have set too low a value for the time-out.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
|
|
