| Author |
Message
|
| morrisea |
 Posted: Tue Feb 16, 2010 1:01 am Post subject: Max number of channels reached |
 |
|
Novice
Joined: 09 Apr 2006
Posts: 21
Location: South Africa
|
Good Day,
We have recently started experiencing the following problem :
----- amqrmrsa.c : 468 --------------------------------------------------------
02/16/2010 07:12:59 AM - Process(19503.269409) User(mqm) Program(amqrmppa)
AMQ9513: Maximum number of channels reached.
EXPLANATION:
The maximum number of channels that can be in use simultaneously has been
reached. The number of permitted channels is a configurable parameter in the
queue manager configuration file.
ACTION:
Wait for some of the operating channels to close. Retry the operation when some
channels are available.
----- amqrcsia.c : 1080 -------------------------------------------------------
02/16/2010 07:12:59 AM - Process(19503.269409) User(mqm) Program(amqrmppa)
AMQ9999: Channel program ended abnormally.
EXPLANATION:
Channel program 'SYSTEM.DEF.SVRCONN' ended abnormally.
ACTION:
Look at previous error messages for channel program 'SYSTEM.DEF.SVRCONN' in the
error files to determine the cause of the failure.
I when we check the status of 'SYSTEM.DEF.SVRCONN' we see 600 connections. I googled the error and the result was to check the qm.ini file for the channels settings there. This is what is defined :
#*******************************************************************#
#* Module Name: qm.ini *#
#* Type : WebSphere MQ queue manager configuration file *#
# Function : Define the configuration of a single queue manager *#
#* *#
#*******************************************************************#
#* Notes : *#
#* 1) This file defines the configuration of the queue manager *#
#* *#
#*******************************************************************#
ExitPath:
ExitsDefaultPath=/var/mqm/exits/
ExitsDefaultPath64=/var/mqm/exits64/
#* *#
#* *#
Log:
LogPrimaryFiles=128
LogSecondaryFiles=64
LogFilePages=65535
LogType=CIRCULAR
LogBufferPages=4096
LogPath=/var/mqm/log/QM_ESB_1/
LogWriteIntegrity=SingleWrite
Service:
Name=AuthorizationService
EntryPoints=13
ServiceComponent:
Service=AuthorizationService
Name=MQSeries.UNIX.auth.service
Module=/opt/mqm/lib/amqzfu
ComponentDataSize=0
CHANNELS:
MaxActiveChannels=3000
MaxChannels=3001
Whay are we getting max channels reached with 600 connections but qm.ini is set to 3001?
Further information about the environment :
mqm@ptabrmq01:/var/mqm/qmgrs/QM_ESB_1 10:56:36 $ uname -a
Linux ptabrmq01 2.6.16.46-0.12-bigsmp #1 SMP Thu May 17 14:00:09 UTC 2007 i686 i686 i386 GNU/Linux
mqm@ptabrmq01:/var/mqm/qmgrs/QM_ESB_1 10:58:25 $ dspmqver
Name: WebSphere MQ
Version: 6.0.2.2
CMVC level: p600-202-070801
BuildType: IKAP - (Production)
mqm@ptabrmq01:/var/mqm/qmgrs/QM_ESB_1 10:58:33 $
No FDC's are being generated, the only info I have is from the qmgr log pasted above. As a result of the error we get transaction failures.
Please can someone advise as to what is going wrong. Thanks. |
|
| Back to top |
|
 |
| zpat |
| Posted: Tue Feb 16, 2010 2:20 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
I would set
TCP KeepAlive=Yes
This helps to remove channels where the other end has gone away.
I would apply later maintenance as well.
6.0.2.8 is the latest.
Then see if the problem still happens.
Are there any other channels in use apart from SYSTEM.DEF.SVRCONN? |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Feb 16, 2010 3:05 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
I would set an illegal MCAUSER on SYSTEM.DEF.SVRCONN, force stop all instances, and see which app team comes yelling into the room.
And then I would give them the trout for using SYSTEM.DEF.SVRCONN, and also for failing to properly close their connections. |
|
| Back to top |
|
|
| shashivarungupta |
| Posted: Tue Feb 16, 2010 3:23 am Post subject: |
|
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
| Quote: |
CHANNELS:
MaxActiveChannels=3000
MaxChannels=3001
Whay are we getting max channels reached with 600 connections but qm.ini is set to 3001? |
Friend, MaxActiveChannels and MaxChannels property is on QMgr not on that particular Channel with which you are facing problems.
*You can increase the max no of connections on that channel !!
Firstof - you should define channels for each application and get them allocated with the respective field values ( make sure that application agrees on those values as max no. of connections on that channel).
*As mqjeff correctly said, do force stop and then set the MCAUSER for that applications.
*To avoid the abnormal shut down better you should have a script to check the no of connections instances on a particular channel, if it crosses the limit, bounce it after consulting with application team.
*Most of the times, application teams do not close the connection.. ask them to correct their code.
_________________
*Life will beat you down, you need to decide to fight back or leave it.
Last edited by shashivarungupta on Thu Feb 25, 2010 8:20 am; edited 1 time in total |
|
| Back to top |
|
|
| zpat |
| Posted: Tue Feb 16, 2010 3:40 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
WMQ V7 can enforce connection limits by channel and/or IP address.
You can also use BlockIP2 (free exit) to limit concurrent connections by channel. |
|
| Back to top |
|
|
| RogerLacroix |
| Posted: Wed Feb 17, 2010 12:50 pm Post subject: |
|
|
Jedi Knight
Joined: 15 May 2001
Posts: 3264
Location: London, ON Canada
|
And there are a couple of supported commercial solutions (MQAUSX and MQSSX) that include limiting the number of connections per channel as a feature of the solution.
Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter |
|
| Back to top |
|
|
| shashivarungupta |
| Posted: Fri Feb 19, 2010 12:47 am Post subject: |
|
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
... I was about to include the comment that Roger gave but I was thinking that to keep the track of no.of threads on the channel mca is good enough as 'mqjeff' said... and as per Roger we would have to install the mqausx/mqssx to curb the no. of connections per channel.... that would need a little involvement from the application team aswell on their settings while connection to mq. ( as security exit on application respective channel).
Better I would say, application team should make the changes in their code to close the connections ! If thats done, they can anytime go for mqausx/mqssx as per the requirements in their system.
_________________
*Life will beat you down, you need to decide to fight back or leave it. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Feb 19, 2010 1:17 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| shashivarungupta wrote: |
| ... I was about to include the comment that Roger gave but I was thinking that to keep the track of no.of threads on the channel mca is good enough as 'mqjeff' said |
Huh?
| shashivarungupta wrote: |
.and as per Roger we would have to install the mqausx/mqssx to curb the no. of connections per channel.... that would need a little involvement from the application team aswell on their settings while connection to mq. ( as security exit on application respective channel).
|
The applications would not need to change a thing for the MQ Admin to install MQAUSX or MQSSX on the SVRCONN channel for the sole purpose of limiting instances of SVRCONN channels.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| shashivarungupta |
| Posted: Mon Feb 22, 2010 12:38 am Post subject: |
|
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
| PeterPotkay wrote: |
| shashivarungupta wrote: |
.and as per Roger we would have to install the mqausx/mqssx to curb the no. of connections per channel.... that would need a little involvement from the application team aswell on their settings while connection to mq. ( as security exit on application respective channel).
|
The applications would not need to change a thing for the MQ Admin to install MQAUSX or MQSSX on the SVRCONN channel for the sole purpose of limiting instances of SVRCONN channels. |
I do know that.
For all the settings at the svrconn channel.. we got to do the settings at the server side of mq... as settings in mqausx.ini file at the server side to set the IPs(if required) and/or the no. of connections on the respective svrconn. channel(s) using which application is trying to connect to the mq queue manager.
I was saying.. for example the application is java jndi based and so then they got to make the settings at jvm and custom properties (QCFs). And they got to include the MQAUSX.Jar file at their library. If they are using capitalware security exit then they got to have a copy of .enc or .ini file of the user id and password in their application server (at some location).
_________________
*Life will beat you down, you need to decide to fight back or leave it. |
|
| Back to top |
|
|
| shashivarungupta |
| Posted: Mon Feb 22, 2010 12:40 am Post subject: |
|
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
| PeterPotkay wrote: |
| shashivarungupta wrote: |
| ... I was about to include the comment that Roger gave but I was thinking that to keep the track of no.of threads on the channel mca is good enough as 'mqjeff' said |
Huh?
|
_________________
*Life will beat you down, you need to decide to fight back or leave it. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Feb 22, 2010 4:50 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| shashivarungupta wrote: |
| PeterPotkay wrote: |
| shashivarungupta wrote: |
| ... I was about to include the comment that Roger gave but I was thinking that to keep the track of no.of threads on the channel mca is good enough as 'mqjeff' said |
Huh?
|
|
I didn't say anything about keeping track of the number of threads.
I said that original poster should block the channel entirely such that the number of threads will always be 0 because nobody at all is authorized to use it. |
|
| Back to top |
|
|
| shashivarungupta |
| Posted: Mon Feb 22, 2010 4:57 am Post subject: |
|
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
| mqjeff wrote: |
| shashivarungupta wrote: |
| PeterPotkay wrote: |
| shashivarungupta wrote: |
| ... I was about to include the comment that Roger gave but I was thinking that to keep the track of no.of threads on the channel mca is good enough as 'mqjeff' said |
Huh?
|
|
I didn't say anything about keeping track of the number of threads.
|
That's what I said, that you didn't say about no. of threads but about the MCA and there is nothing wrong in that.
_________________
*Life will beat you down, you need to decide to fight back or leave it. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Feb 22, 2010 5:39 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| shashivarungupta wrote: |
| PeterPotkay wrote: |
| shashivarungupta wrote: |
.and as per Roger we would have to install the mqausx/mqssx to curb the no. of connections per channel.... that would need a little involvement from the application team aswell on their settings while connection to mq. ( as security exit on application respective channel).
|
The applications would not need to change a thing for the MQ Admin to install MQAUSX or MQSSX on the SVRCONN channel for the sole purpose of limiting instances of SVRCONN channels. |
I do know that.
For all the settings at the svrconn channel.. we got to do the settings at the server side of mq... as settings in mqausx.ini file at the server side to set the IPs(if required) and/or the no. of connections on the respective svrconn. channel(s) using which application is trying to connect to the mq queue manager.
I was saying.. for example the application is java jndi based and so then they got to make the settings at jvm and custom properties (QCFs). And they got to include the MQAUSX.Jar file at their library. If they are using capitalware security exit then they got to have a copy of .enc or .ini file of the user id and password in their application server (at some location).
|
The applications would not need to change a thing for the MQ Admin to install MQAUSX or MQSSX on the SVRCONN channel for the sole purpose of limiting instances of SVRCONN channels. They do not need to install anything on the client side. Nothing.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| shashivarungupta |
| Posted: Mon Feb 22, 2010 6:14 am Post subject: |
|
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
| PeterPotkay wrote: |
| shashivarungupta wrote: |
| PeterPotkay wrote: |
| shashivarungupta wrote: |
.and as per Roger we would have to install the mqausx/mqssx to curb the no. of connections per channel.... that would need a little involvement from the application team aswell on their settings while connection to mq. ( as security exit on application respective channel).
|
The applications would not need to change a thing for the MQ Admin to install MQAUSX or MQSSX on the SVRCONN channel for the sole purpose of limiting instances of SVRCONN channels. |
I do know that.
For all the settings at the svrconn channel.. we got to do the settings at the server side of mq... as settings in mqausx.ini file at the server side to set the IPs(if required) and/or the no. of connections on the respective svrconn. channel(s) using which application is trying to connect to the mq queue manager.
I was saying.. for example the application is java jndi based and so then they got to make the settings at jvm and custom properties (QCFs). And they got to include the MQAUSX.Jar file at their library. If they are using capitalware security exit then they got to have a copy of .enc or .ini file of the user id and password in their application server (at some location).
|
The applications would not need to change a thing for the MQ Admin to install MQAUSX or MQSSX on the SVRCONN channel for the sole purpose of limiting instances of SVRCONN channels. They do not need to install anything on the client side. Nothing. |
For no. of connections , to make a check, that has to be done on the server side of mq... no doubt on that. ( and if appli. side doesn't follow the steps it won't be able to make the connection(s) over the queue manager using the SVRCONN channel on which security exit is implemented and for which parameters are being set by mq admin on mqausx.ini file at mq server end.)
_________________
*Life will beat you down, you need to decide to fight back or leave it. |
|
| Back to top |
|
|
| morrisea |
| Posted: Wed Feb 24, 2010 9:56 pm Post subject: Max connections exceeded |
|
|
Novice
Joined: 09 Apr 2006
Posts: 21
Location: South Africa
|
Hello,
Many thanks for the replies and discussions. I agree that there are a number of ways forward which we are working on. However, none of the comments adress the matter that "when we check the status of 'SYSTEM.DEF.SVRCONN' we see 600 connections" yet the settings are set to 3000 (MaxChannels), and we are getting max.channel connections reached at 600. |
|
| Back to top |
|
|
|
|
