| Author |
Message
|
| WingCommanderBadger |
 Posted: Mon Jul 05, 2010 6:42 am Post subject: Server Connection Channels and the Channel Status Table |
 |
|
Apprentice
Joined: 06 Sep 2005
Posts: 32
Location: London, UK
|
My team is in the process of migrating a number of 'listener' style Java/JMS services from a WebSphere Application Server 6.1 platform using Message Driven Beans to a Tomcat 6.0.26 platform using Spring DefaultMessageListenerContainers. In both cases, WebSphere MQ v7.0.1.2 is used and the listeners connect to the Queue Manager in Client Mode.
One of the issues we have encountered is that we now seem to occasionally max out the active channels (which are still defaulted to 100) in the Channel Status Table. What I would like to know is how does MQ decide when to put a new entry into this table? I'm aware of the obvious case where separate, distinct remote applications will be allocated at least a single entry in this table, however it's not clear to me how MQ behaves when there is only one remote application sharing what I believe to be a single JMS Connection amongst multiple listeners.
I have created a very simple test environment outside of either WAS or Tomcat where I define a number of Spring DefaultMessageListenerContainers and connect them to MQ so I can observe the number of entries in the Channel Status Table for the Server Connection Channel specified. My most recent result uses 12 listeners, each connected to the same Queue Manager but listening on different Queues. In this scenario, there are 3 entries in the Channel Status Table but I haven't got a clue why! 
Can someone please shed some light on this issue at all?
Additionally, does anyone have any insight into why the behaviour using Tomcat and Spring is so different to that of using WAS and MDBs?
Thanks in advance. |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Mon Jul 05, 2010 6:51 am Post subject: Re: Server Connection Channels and the Channel Status Table |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| WingCommanderBadger wrote: |
My team is in the process of migrating a number of 'listener' style Java/JMS services from a WebSphere Application Server 6.1 platform using Message Driven Beans to a Tomcat 6.0.26 platform using Spring DefaultMessageListenerContainers. In both cases, WebSphere MQ v7.0.1.2 is used and the listeners connect to the Queue Manager in Client Mode.
One of the issues we have encountered is that we now seem to occasionally max out the active channels (which are still defaulted to 100) in the Channel Status Table. What I would like to know is how does MQ decide when to put a new entry into this table? I'm aware of the obvious case where separate, distinct remote applications will be allocated at least a single entry in this table, however it's not clear to me how MQ behaves when there is only one remote application sharing what I believe to be a single JMS Connection amongst multiple listeners. |
Here is your first bad assumption. If you have not explicitly set connection sharing (and I would not recommend it in this case), each instance of the MDB type connection would use 1 channel. In fact MDBs use #of instances +1 channels.
| WingCommanderBadger wrote: |
I have created a very simple test environment outside of either WAS or Tomcat where I define a number of Spring DefaultMessageListenerContainers and connect them to MQ so I can observe the number of entries in the Channel Status Table for the Server Connection Channel specified. My most recent result uses 12 listeners, each connected to the same Queue Manager but listening on different Queues. In this scenario, there are 3 entries in the Channel Status Table but I haven't got a clue why!
Can someone please shed some light on this issue at all?
Additionally, does anyone have any insight into why the behaviour using Tomcat and Spring is so different to that of using WAS and MDBs?
Thanks in advance. |
Again with the use of MDB's I would not suggest sharing connections. Remember that the messages are being synchronized through a connection. Connection sharing there may affect performance.
As a rule of thumb you need #instances of MDB + 1 connections for any single MDB. Needing more than 100 connections is not uncommon in production environments. You need to set your limit according to your needs and double it to give it some room to grow...
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| WingCommanderBadger |
| Posted: Mon Jul 05, 2010 7:14 am Post subject: |
|
|
Apprentice
Joined: 06 Sep 2005
Posts: 32
Location: London, UK
|
Thanks for the response fjb_saper, however your reply seems to be talking a lot about MDBs which we're not using any more. Are you suggesting the rules for using MDBs and Spring DefaultMessageListenerContainers are the same or similar?
With regard to Application Connections, each listener does indeed seem to have 2 connections each, in fact the same happens for bog-standard JMSMessageListeners so I imagine this will be the case regardless of whether it's wrapped in an MDB or a DefaultMessageListenerContainer.
Each of the 12 ListenerContainers defined in my test environment is set to cache the JMS Connection, Session and Consumer objects once they've been created, however the JMS Connections themselves are originally sourced via the same QueueConnectionFactory object.
My issue is to do with entries in in the MQ Channel Status table however - the entries in this table do not seem to share a 1:1 relationship with the Application Connections. As I said, why do I only have 3 entries when I have 12 listeners, each with 2 application connections? |
|
| Back to top |
|
|
| WingCommanderBadger |
| Posted: Mon Jul 05, 2010 7:38 am Post subject: Re: Server Connection Channels and the Channel Status Table |
|
|
Apprentice
Joined: 06 Sep 2005
Posts: 32
Location: London, UK
|
Just realised, it's a new MQv7 feature: Sharing Conversations. When I set this value to 1 for the Server Connection Channel in question I see 24 entries in the table. Now I'm making some progress....  |
|
| Back to top |
|
|
| jeevan |
| Posted: Mon Jul 05, 2010 10:27 am Post subject: Re: Server Connection Channels and the Channel Status Table |
|
|
Grand Master
Joined: 12 Nov 2005
Posts: 1432
|
| WingCommanderBadger wrote: |
| Just realised, it's a new MQv7 feature: Sharing Conversations. When I set this value to 1 for the Server Connection Channel in question I see 24 entries in the table. Now I'm making some progress.... |
Also consider the orphan CONNECTION. Have you set tcp_keepalive in your server side? that helps to kill the orphane connection which used up the chanels in channel table. |
|
| Back to top |
|
|
| WingCommanderBadger |
| Posted: Mon Jul 05, 2010 11:47 pm Post subject: Re: Server Connection Channels and the Channel Status Table |
|
|
Apprentice
Joined: 06 Sep 2005
Posts: 32
Location: London, UK
|
| Jeevan - yes I'm already aware of this potential issue from my MQv6 days. We have KeepAlive=YES in all our Queue Manager's qm.ini files. I'm now wondering whether MQv7 offers a way of achieving this for Server Connection Channels via MQSC though. Any thoughts? |
|
| Back to top |
|
|
| WingCommanderBadger |
| Posted: Tue Jul 06, 2010 2:03 am Post subject: Re: Server Connection Channels and the Channel Status Table |
|
|
Apprentice
Joined: 06 Sep 2005
Posts: 32
Location: London, UK
|
The behaviour I'm now observing is that MQ doesn't seem to allocate Conversations per Channel in the status table in a consistent way. The default max number of Conversations per Channel is 10. So, for 10 Listeners I would expect to see two entries in the Channel Status table, both with their maximum number of conversations. This is not always the case unfortunately. I've seen MQ place anywhere between 2 and 7 entries in the Channel Status table with the scenario defined above. I guess only IBM know how it decides to allocate conversations per channel, unless someone can enlighten me?
Anyhow, I've decided to use the following formula to decide what the contributing value should be for MaxChannels on a per application, per DefaultMessageListener basis:
MaxConcurrentConsumers + 1
Therefore, for an application with 11 DefaultMessageListeners, each with a MaxConcurrentConsumers value of 10, the value would be 11 x 11 = 121.
I realise that there are other processes that may add entries to the Channel Status Table, so I guess an arbitrary amount needs adding to this figure. I've read that IBM recommends multiplying by 3, but this seems like overkill to me. I thought just maybe adding 100 to this figure would suffice. Does anyone have any further advice with regard to this? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Jul 06, 2010 2:56 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Think of each MDB as playing the server process on a request/reply model.
So your MDB potentially needs (#of instances *2) +1 channel instances...
At a minimum you will need a different session to send the response...
Add to it the number of concurrent raw sends happening in the appserver...
So multiplying the number as given in the previous responses calculation (#mdb instances +1) by 3 is not a bad guesstimate.
Ultimately monitoring of the channels (status) will tell you what your approximate max usage is. Double that to give room to grow and assess regularly when new applications or new functionality(application) comes on board.
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Jul 06, 2010 3:04 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Set Max Active Channels to a really big number way bigger than you need. Use 1,000, or 2,500, or more. The default of 100 is ridiculously small in 2010 and is leftover from when MQ first came out. Make that value big now because all you're gonna do when you hit it (because you tried to make it exactly what you need + 1) is bump it up and take another Queue Manager outage to pick it up.
Then, since you are at MQ 7, set the maximum number of instances at the individual channel level to prevent any one channel from sucking up all your Max Active Channels at the QM level.
You still have to use KeepAlive at the QM level, and all that says is use the O/S KeepAlive value, if its set there, so verify that. Just because the QM's ini file says use Keep Alive doesn't mean its available to be used. The typical O/S default of 2 hours for Keep Alive should also be reviewed and probably bumped way down (we use 15 minutes). Only on the mainframe can you set keep alive on the individual channels and have it used by the QM, although at version 7 of MQ Heartbeats on SVRCONN channels on all platforms work just as well as Keep Alive (so I've read).
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| WingCommanderBadger |
| Posted: Fri Jul 09, 2010 3:50 am Post subject: Re: Server Connection Channels and the Channel Status Table |
|
|
Apprentice
Joined: 06 Sep 2005
Posts: 32
Location: London, UK
|
Yes I seem to recall that about setting KeepAlive on MQ Channels on z/OS.
Anyhow, thanks very much to all for your advice. |
|
| Back to top |
|
|
|
|
