ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexIBM MQ Java / JMSIs it safe to load balance JMS sessions at the TCP level?

Post new topicReply to topic
Is it safe to load balance JMS sessions at the TCP level? View previous topic :: View next topic
Author Message
paulau
PostPosted: Wed Jul 24, 2019 4:40 pm Post subject: Is it safe to load balance JMS sessions at the TCP level? Reply with quote

Novice

Joined: 06 Feb 2017
Posts: 14

Hi,
I have been asked to look at the MQ integration approach for a Camel/Spring/JMS over MQ application thats running on OpenShift containers.

One of the requirements is to support connections to more than one queue manager from each container. We tried the CCDT Scenario 6 'queue manager groups' approach documented here https://www-01.ibm.com/support/docview.wss?uid=swg27020848

Unfortunately it doesn't work because the Spring layer binds to the first connection factory and NEVER allows another new connection factory call though to JMS. This means that there is only ever one JMS connection factory channel instances to MQ and all of the subsequent sessions (which create their own svrconn instances) are associated with the one queue manager.

When we route the connections & sessions through MQIPT they get load balanced appropriately using the SampleRoutingExit documented here https://www.ibm.com/support/knowledgecenter/SSFKSJ_9.1.0/com.ibm.mq.con.doc/ipt4191_.htm

The application is event based so doesn't have any issues with requests and responses going to different queue managers.

I have seen some posts where its claimed that JMS traffic shouldn't be routed through a load balancer e.g. "Load Balancer the load balancer could be provided by the container orchestration engine, or be a standard network load balancer such as an F5. Regardless it can provide TCP load balancing in front of the IBM MQ Queue Managers. This approach removes the application from making the connection selection, and instead will forward onto the load balancer IP and port. Often load balancers have the capability for more feature rich workload management strategies, that may make it more appealing than CCDT. Although there are certain capabilities such as JMS that are not recommended with an external load balancer, so these need to be considered when selecting an effective strategy."


Does anybody have any details around JMS load balancing issues as it seems to work OK.

Any help would be appreciated.
Paul
Back to top
View user's profile Send private message
hughson
PostPosted: Thu Jul 25, 2019 7:36 pm Post subject: Reply with quote

Grand Master

Joined: 09 May 2013
Posts: 1185
Location: Bay of Plenty, New Zealand

Load balancing of client connections is generally OK. Obviously you need to take care of ensuring that wherever the client connection ends up, the queues it expects to use are available to it.

The reason load balancing of client connections is generally OK is because they are stateless, unlike QMgr-QMgr channels where you very definitely shouldn't load balance between them.

There is one situation where clients are not stateless and that is when they are using XA transactions. If you are using XA transactions, then please don't load balance or state about prepared but not yet completed transactions may not be found.

Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software
Back to top
View user's profile Send private message Visit poster's website
paulau
PostPosted: Thu Jul 25, 2019 10:38 pm Post subject: Reply with quote

Novice

Joined: 06 Feb 2017
Posts: 14

Hi Morag, Thanks for the response.

They currently use XA to save state between steps in the payments process so thats bad news. Understand that if XA thinks there are things in the LUW that the queue manager doesn't know about because its on a different connection it would cause issues during recovery.

I guess we will have to keep looking for a way to get the ConnectionFactory to load balance. I am assuming that XA is happy if you keep all of the session connections on the same queue manager that got the ConnectionFactory.

Paul
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Fri Jul 26, 2019 4:36 pm Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20072
Location: LI,NY

It is also a very bad idea to try and balance what is in essence a JMS MessageDrivenBean. (Think about load balancing an activation specification... nightmare!!).
When dealing with MDBs you should set the max number of instances for a specific queue and then you should setup one instance of the MDB for each queue manager hosting the queue. This way you will get the message wherever it lands...

Have fun
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
paulau
PostPosted: Tue Jul 30, 2019 3:46 pm Post subject: Reply with quote

Novice

Joined: 06 Feb 2017
Posts: 14

Well after checking with the development team it turns out that they are NOT using XA for transaction management. I think they may have started out down the XA path a couple of years ago and then removed it during the performance test phase of the project.

So what they have now is the ability to specify concurrentConsumers in Camel, then some weird behaviour in Spring where it uses 1 ConnectionFactory and 2* concurrentConsumers to generally create a cached pool of 9 MQ connections for the 4 application threads in camel. There are only 4 open handles on the queue. It doesn't seem to matter what you do with maxConnections and maxSessionsPerConnection as the MQAT trace shows a new connection for every session.

There are 150+ different Microservices. some of which have very low volumes. The development team would like more options for managing the connections to MQ. With the Camel/Spring setup that means changing the Spring code to support independent ConnectionFactories, or changing the infrastructure to route the JSM sessions to different queue managers. We have also asked the MQ admins to deploy AMQSCLM so they don't have to have an application instance for each of the 4 queue managers.

Personally I think changing the code to have a connection factory for each queue manager is the more consistent way of managing MQ connections between the OpenShift containers and the queue managers in the cluster however understand that there is a benefit to not changing all of the legacy application code.

Part of the solution is increased visibility of what's happening with the connections so they are in the process of adding MQ statistics for channels and queue handles into the existing Prometheus monitoring framework.

@fjb_saper I understand where you are coming from with the nightmare comment and agree that a connection factory per queue manager is the preferred way to listen to multiple queue mangers. The current setup is a bit of a nightmare to understand because the Spring framework does some weird stuff and it was also being masked by the conversations setting on the svrconn channel. Now that they have some more visibility of the MQ connections and open handles and a better understanding of the framework behaviour they can start to tune settings and make some more informed decisions.
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Wed Jul 31, 2019 9:19 pm Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20072
Location: LI,NY

Using MDBs I would also set shared conversations to 1 on the svrconn channel, or to 0 if some old compatibility behavior is required.
Remember that you will need a minimum of MDB number of sessions + 1 connections to read from the queue. (The =1 counts for the activation specification).
Have fun
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
paulau
PostPosted: Mon Aug 12, 2019 4:25 pm Post subject: Reply with quote

Novice

Joined: 06 Feb 2017
Posts: 14

The queue managers are V8 on Solaris and they currently have shared conversations set to 10. Agree the target should be 1 as its proven to be compatible and is more efficient than 10.

The downside of the camel/spring boot/jms/mq stack is that we start with 4 threads in camel and the spring boot cache seems to manage a connection pool thats over 30. So our micro services end up using a lot of MQ connections for a small number of threads. The MQ team want us to keep under 600 channel instances in production and the actual number of MQ conversations is likely to be a couple of thousand.

So no option of moving to 1 conversation in the short term until we understand how to measure and manage the connection pool.
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Page 1 of 1

MQSeries.net Forum IndexIBM MQ Java / JMSIs it safe to load balance JMS sessions at the TCP level?
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.