| Author |
Message
|
| guest468 |
 Posted: Fri May 31, 2013 11:14 am Post subject: MQ client seeing AMQ9248 error , Tcp/ip return code 10048 |
 |
|
Centurion
Joined: 30 May 2006
Posts: 146
Location: NY
|
Hi,
MQ client running in Windows server 2003 R2. It is a working(apparently healthy) program for as long as the developers can remember. But suddenly a few days ago they started seeing issues connecting to the QMGR. No deployments or changes made to MQ or application in a long while. Client side error logs show the following...
MQ Version V7.0.1
AMQ9248: The program could not bind to a TCP/IP socket.
EXPLANATION:
The attempt to bind to socket '0' failed with return code 10048. The failing
TCP/IP call was 'bind'. The most likely cause of this problem is incorrect
configuration of the TCP/IP local address or incorrect start and end port
parameters.
netstat showed around 5000 socket connections in TIME_WAIT state (except 5 established). At the server(MQ) side, there were only around 10 connections (time_wait + established). No errors or FDCs at MQ server side.
Now given that Tcp/ip return code 10048 means "Address already in use" I was interested to know why this happens. Is it MQCONN somehow trying to reuse a closed socket connection?.
Does ClietIdle would help? I thought this would be a risky move unless it is set to a large interval (1 or 2 hours may be).
Does SHARECNV gonna help in this particular scenario?
As a temporary fix we increased the number of tcp ports available from default 5k to 32k.
Thanks |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Fri May 31, 2013 12:58 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9470
Location: US: west coast, almost. Otherwise, enroute.
|
TCP/IP working as designed - connections are not closed immediately, as packets may arrive out of order or be retransmitted after the connection has been closed.
TIME_WAIT indicates that this side has closed the connection. The connection is being kept around so that any delayed packets can be matched to the connection and handled appropriately. The connections will be removed when they time out within four minutes - the default.
Set this value to a smaller value. On Windows 2003, reduce the TIME_WAIT by setting the TcpTimedWaitDelay TCP/IP parameter to 30 seconds on the windows registry key HKLM\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters, as a DWORD value.
IMS, 5000 is the default Windows value for maximum number of ports to allocate.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| guest468 |
| Posted: Fri May 31, 2013 1:04 pm Post subject: |
|
|
Centurion
Joined: 30 May 2006
Posts: 146
Location: NY
|
Thanks Bruce,
And yes windows admin did change the TcpTimedWaitDelay parameter to 30 seconds along with the number of ports change. We haven't seen the issue ever since.
But I was wondering if at MQ level we can handle this using SHARECNV and ClientIdle? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Fri May 31, 2013 10:03 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Well ... I would try and investigate why you have the need client side of 5,000 + connections / sockets in such a short interval. Seems there that may be some programming optimization could greatly reduce the number of connections needed? 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| guest468 |
| Posted: Tue Jun 04, 2013 12:34 pm Post subject: |
|
|
Centurion
Joined: 30 May 2006
Posts: 146
Location: NY
|
yes saper,
This a small application running standalone.
What I hear is there was some big change in the size of the message they send out (due to xsd change). The size is about 2MB (used to be only 5kb).
Would that result in multiple socket connections. I was thinking may be default tcp buffer(TcpWindowSize) may be behind this in some way? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Jun 05, 2013 6:15 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| guest468 wrote: |
yes saper,
This a small application running standalone.
What I hear is there was some big change in the size of the message they send out (due to xsd change). The size is about 2MB (used to be only 5kb).
Would that result in multiple socket connections. I was thinking may be default tcp buffer(TcpWindowSize) may be behind this in some way? |
Message size should not matter. What will matter is connect and disconnects in the program...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| guest468 |
| Posted: Wed Jun 05, 2013 10:34 am Post subject: |
|
|
Centurion
Joined: 30 May 2006
Posts: 146
Location: NY
|
yes you are right.
App team found the the root cause. It seems there was some changes made to the code; the wait time from the MQGET was removed 
Thanks, |
|
| Back to top |
|
|
|
|
