ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » IBM MQ Performance Monitoring » Performance degraded from local MQ to Remote MQ Connection

Post new topic  Reply to topic
 Performance degraded from local MQ to Remote MQ Connection « View previous topic :: View next topic » 
Author Message
pandeg
PostPosted: Thu Apr 14, 2016 6:33 am    Post subject: Performance degraded from local MQ to Remote MQ Connection Reply with quote

Disciple

Joined: 21 Oct 2014
Posts: 195

Hi, We have lots of JMS applications which were using local MQ and then we migrated them to Remote Central MQ inorder to save the licensing cost and ease of support and maintenance ,but we found that the transaction time of the messages increased to three folds reducing the performance.
For example - previously it was taking 10 minutes to send 500 messages , after centralization it takes 30 minutes to send 500 messages. Every things remains the same , network, bandwidth, JMS connection configuration, MQ objects.
I can understand the performance cannot be same when application connecting locally vs connecting centrally. But how much performance difference can be expected. Since , its asynchronous communication in our case , its not creating any issue at this time but we want to know if we going below the expected performance level.
Back to top
View user's profile Send private message
Vitor
PostPosted: Thu Apr 14, 2016 6:52 am    Post subject: Re: Performance degraded from local MQ to Remote MQ Connecti Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

pandeg wrote:
Every things remains the same , network, bandwidth, JMS connection configuration, MQ objects.


That can't be true. You must be traversing network routes and network components that you're not traversing when you connect locally.

(I'm assuming that when you say "using a local MQ" you mean a client connection to a local queue manager not a binding connection. If you've gone from binding to client I've found your problem.)

What do the channel & network statistics say is happening over these 30 minutes?
_________________
Honesty is the best policy.
Insanity is the best defence.


Last edited by Vitor on Thu Apr 14, 2016 6:53 am; edited 1 time in total
Back to top
View user's profile Send private message
mqjeff
PostPosted: Thu Apr 14, 2016 6:53 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

10 minutes to send 500 messages is an extreme performance problem.

That suggests either significant issues with the config of your app server (or queue manager) or significant issues with your application.

The fact that it goes up by a factor of 3 when you move to network location makes it more likely that it's an application issue. Although your queue manager could be undersized for the transactions you're doing.

Even if you're sending asynchronous messages, those can still sit in uncommitted transactions until your application does the right thing.

Start by reviewing how the application handles transactions. Follow up by how the app server is configured for transactions. Then work with your MQ admins to review their config and setup for transaction size and handling.
_________________
chmod -R ugo-wx /
Back to top
View user's profile Send private message
bruce2359
PostPosted: Thu Apr 14, 2016 7:29 am    Post subject: Re: Performance degraded from local MQ to Remote MQ Connecti Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

pandeg wrote:
Every things remains the same , network, bandwidth, JMS connection configuration, MQ objects.

This statement contradicts what surrounds it. Many things have changed.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
pandeg
PostPosted: Thu Apr 14, 2016 7:34 am    Post subject: Re: Performance degraded from local MQ to Remote MQ Connecti Reply with quote

Disciple

Joined: 21 Oct 2014
Posts: 195

Quote:

That can't be true. You must be traversing network routes and network components that you're not traversing when you connect locally.


I meant its same in the sense from the remote Queue Manager end. When the application were connecting locally, its still the same network configuration between these local and remote queue manager.
Quote:

(I'm assuming that when you say "using a local MQ" you mean a client connection to a local queue manager not a binding connection. If you've gone from binding to client I've found your problem.)


yeah, application is connecting in client mode both in case of local MQ and remote Central MQ.

Quote:
What do the channel & network statistics say is happening over these 30 minutes?[
we haven't captured any channel and network statistics. Can you please let me know what channel and network information should i look for ?
Back to top
View user's profile Send private message
exerk
PostPosted: Thu Apr 14, 2016 7:38 am    Post subject: Re: Performance degraded from local MQ to Remote MQ Connecti Reply with quote

Jedi Council

Joined: 02 Nov 2006
Posts: 6339

pandeg wrote:
yeah, application is connecting in client mode both in case of local MQ and remote Central MQ

So, localhost as far as the network was concerned - and maybe never would have gone across the network?
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys.
Back to top
View user's profile Send private message
Vitor
PostPosted: Thu Apr 14, 2016 7:59 am    Post subject: Re: Performance degraded from local MQ to Remote MQ Connecti Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

exerk wrote:
pandeg wrote:
yeah, application is connecting in client mode both in case of local MQ and remote Central MQ

So, localhost as far as the network was concerned - and maybe never would have gone across the network?




The network people should be able to produce traffic statistics including (but not limited to):

- traffic pattern (are the 500 messages being transmitted continuously but very slowly, or is there a significant pause between each message)
- route taken (are there a significant number of hops degrading performance)
- packet loss and retry (is the connection unreliable)

The channel will tell you how quickly messages are being committed from the other end and other statistics.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
mqjeff
PostPosted: Thu Apr 14, 2016 8:25 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

The change in network routes only altered the performance by a factor of 3.

This is huge, but the performance was already incredibly bad to begin with.

In this rare case, it's better to start with troubleshooting the application, move to troubleshooting the queue manager, and *then* blame it all on the network team - instead of the opposite order.
_________________
chmod -R ugo-wx /
Back to top
View user's profile Send private message
bruce2359
PostPosted: Thu Apr 14, 2016 8:53 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

mqjeff wrote:
The change in network routes only altered the performance by a factor of 3.

This is huge, but the performance was already incredibly bad to begin with.

In this rare case, it's better to start with troubleshooting the application, move to troubleshooting the queue manager, and *then* blame it all on the network team - instead of the opposite order.


If the only thing that changed is "the network," then the cause is most likely "the network."

So, no line or lines of code changed? No o/s changes? No MQ changes? No hardware changes?

Really? What exactly do you mean by "migrated?" When you migrated to a different center, did you physically move the server hardware? Or did you copy the server file system, and restore it on a new server hardware?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
mqjeff
PostPosted: Thu Apr 14, 2016 9:20 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

bruce2359 wrote:
mqjeff wrote:
The change in network routes only altered the performance by a factor of 3.

This is huge, but the performance was already incredibly bad to begin with.

In this rare case, it's better to start with troubleshooting the application, move to troubleshooting the queue manager, and *then* blame it all on the network team - instead of the opposite order.


If the only thing that changed is "the network," then the cause is most likely "the network."


Sure. That explains why the performance degraded by a factor or 3.

It doesn't explain why the performance was so horrible in the first place.
_________________
chmod -R ugo-wx /
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » IBM MQ Performance Monitoring » Performance degraded from local MQ to Remote MQ Connection
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.