| Author |
Message
|
| sm138929 |
 Posted: Thu Nov 15, 2007 5:27 pm Post subject: Channel drop issue between Q Mgrs + Load balancer |
 |
|
Apprentice
Joined: 29 Aug 2007
Posts: 25
|
Hi ,
Our Sender and receiver channels from our Qmgr to our Businesspartner q mgr have been dropping sicne we moved to a new data centre and drops are fairly regular and has a interval of 8hrs for both sender and reciever ...Our Bpartner is in same infrastructure but but we have a new set of hardware and have a new load balancer from Cisco called CSS 11503 ..We have 3 MQ server s 5.3 CSD 4 in cluster ,2 MQIPT servers ver 1.3.2 connected to the CSS load balancer .All areon Solaris .We have been getting the AMQ 9208,AMQ9999,AMQ9209,AMQ9213,AMQ9558,AMQ9514 errors .Sometime both channels drop together ,some time they drop at different time ...this is going for last 2 months .We have a ticket open with IBM and tech support requested t ohave look in our VPN ,network devices ,Load balancer and our network guys are always saying everything is perfect in the network ..Our Business partner has the same setup but the are having mqipt 1.3 ...
So is there anyway we can track the issue...
Thanks ,
sm |
|
| Back to top |
|
 |
| Michael Dag |
| Posted: Thu Nov 15, 2007 11:21 pm Post subject: |
|
|
Jedi Knight
Joined: 13 Jun 2002
Posts: 2607
Location: The Netherlands (Amsterdam)
|
|
| Back to top |
|
|
| sm138929 |
| Posted: Fri Nov 16, 2007 9:01 am Post subject: Channel drop issue |
|
|
Apprentice
Joined: 29 Aug 2007
Posts: 25
|
Hi Mike,
This query is a seque to the thread you mentioned .However I couldnot follow the second thread ....which could have been related to our load balancer . |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Fri Nov 16, 2007 9:08 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
You can't use a network level load balancer between two queue managers.
You can only use it between a queue manager and a client.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| sm138929 |
| Posted: Fri Nov 16, 2007 11:05 am Post subject: Regarding load balancer |
|
|
Apprentice
Joined: 29 Aug 2007
Posts: 25
|
Hi Jeff,
Thanks for the information.
However in our earlier data center we used F5 Big Ip load balancer between our MQ servers and our partner MQ servers over a Frame Relay circuit instead of VPN which we have currently in our new Datacenter.Anyway I am not a network person so I maynot be correct ..
At that time we never ran into this regular channel drop issue .. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Fri Nov 16, 2007 11:11 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
You can, potentially, use a network load balancer between two MQIPT installations.
That might be what you had done.
It sounds like they've mangled your network somehow in the changeover. Or the MQIPT configuration is bad.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| sm138929 |
| Posted: Fri Nov 16, 2007 11:22 am Post subject: MQIpt configuration ??Or Network |
|
|
Apprentice
Joined: 29 Aug 2007
Posts: 25
|
Hi ,,
We have the two MQIPT servers connected in the Cisco CSS load balancer. We have a ticket open with IBM support and they had the MQIPT configuration checked and told us it was ok ..If required I can always share the same here ...However we are still looking in the network settings and network guys always says it is perfect as always  ...
Thanks ,
SM |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Fri Nov 16, 2007 11:34 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Well. If IBM says it's likely a network problem... and we say it's likely a network problem...
You should be able to give the network team a very specific time when the channel drops. They should be able to examine their logs in detail for that very specific time, and see what's going on.
It might be that they have failed to allow connections to stay open 'indefinitely' and are closing them after a certain period of time.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| sm138929 |
| Posted: Fri Nov 16, 2007 12:09 pm Post subject: Channel drop issue |
|
|
Apprentice
Joined: 29 Aug 2007
Posts: 25
|
Thanks for the comment ..Actually we are doing a test in our preprod environment which is having a similar drop of channels and have the same laod balancer serving it ....Do we need to have a look at the Cisco CSS load balancer timeouts ....One question regarding MQIPT configuration...
Here are the global parameters for our MQIPT v 1.3.2 :
[global]
CommandPort=1881
RemoteShutDown=true
MinConnectionThreads=5
MaxConnectionThreads=100
IdleTimeout=20
ClientAccess=true
QMgrAccess=true
HTTP=false
HTTPChunking=false
Trace=5
ConnectionLog=true
MaxLogFileSize=50
AccessPW=XXXXX
_____________________________________________________________
Also our Sender channel DISCINT is set to 0 as it was in our earlier datacenter environment .However our Businesspartner has set the same to 400 in their MQ Sender channels ...
My question is there any relation to the idletimeout value in mqipt (set to 20 mins) and the Cisco CSS load balancer idle timeout settings.
Thanks,
SM |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Nov 16, 2007 1:01 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
An Inactive channel can't get into trouble. With DISCINT set to 0 your channel is always running even if it hasn't send a message in hours. So basically its running looking for a problem.
I set DISCINT waaaay down when dealing with firewalls and/or flaky network connections, because, you know, an Inactive channel can't get into trouble. Let XMITQ triggering start up the channel when you need it.
Do you have heartbeats turned on? What is that value on both the SNDR and RCVR? HBs will pulse traffic across the channel, but its only something like 12 bytes so that might not be enough to make whatever is stopping your connection think the connection is still alive.
Do you have channel problems while you are activily shipping messages?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| sm138929 |
| Posted: Fri Nov 16, 2007 1:32 pm Post subject: Channel problems |
|
|
Apprentice
Joined: 29 Aug 2007
Posts: 25
|
Hi Peter,
Yes we have heartbeat set to 300 seconds for both SDR and RCVR channels.
We do have channel problem when there are active messages .Also we have triggering set for the channels.
Regarding the DISCINT set to 0 it was like that in our old environment..Anyways its always worth looking into this setting ....
Thanks,
SM |
|
| Back to top |
|
|
| sm138929 |
| Posted: Mon Nov 19, 2007 1:40 pm Post subject: Regarding channels |
|
|
Apprentice
Joined: 29 Aug 2007
Posts: 25
|
Hi ,
I have one question..Do we need to look at heartbeat interval while setting disconnection interval for the channels ....Is there any relation to the timeout values in MQIPT ,Load balancer and other network devices .I am just trying to find the possible cause behind the channel drops ..
thanks,
SM |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Nov 19, 2007 4:14 pm Post subject: Re: Regarding channels |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| sm138929 wrote: |
I am just trying to find the possible cause behind the channel drops ..
|
You shouldn't be, your network folks should be. MQ channels don't drop. The underlying network connections can for various reasons. Let them tell you why the connection goes. Once you know that we can see if tweaking an MQ setting or two might be a work around.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| sm138929 |
| Posted: Wed Nov 28, 2007 10:02 am Post subject: question with the Cisco CSS load balancer with MQ |
|
|
Apprentice
Joined: 29 Aug 2007
Posts: 25
|
Hi,
We have been checking our MQ channels by changing the idle flow timeout value in Cisco CSS Laod balancer from 16 secs to 400 secs ,had the Channel disconnection int changed from 0 to 200 secs and the MQIPT idletimeout set to 300 secs or 5 mins fro mexisting 20 mins ..Also I changed the HBT int in SDR channel from 300 to 150 secs ...This prevented the abnomal drops but channels went to inactive state when no message arrived ...
However we need to to know how we can have Cisco CSS idletimeout /idleflowtimeout value to be set because this is creating a lot of confusion with network guys ...who always believe that the network devices are perfect and MQ is creating the problem
Thanks,
SM |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Nov 28, 2007 12:19 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Nothing should interrupt an MQ channel network connection except an MCA.
Doing anything else will cause the channel to fail, and may cause message loss.
This means that CISCO CSS should have NO timeout set for the channel connections.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
