ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » IBM MQ Installation/Configuration Support » Improve throughput

Post new topic  Reply to topic Goto page 1, 2  Next
 Improve throughput « View previous topic :: View next topic » 
Author Message
vincent
PostPosted: Fri Apr 21, 2006 1:45 pm    Post subject: Improve throughput Reply with quote

Voyager

Joined: 21 Apr 2006
Posts: 81

Loacal Application is putting messages on Local Queue Manager on Solaris-8 but this is being retrieved by Windows client through JMS program. There is no complex setup. We are expecting 1500 per second throughput but we are getting only 400/500 as an output to the client program. Can anyone help me to improve this throughput?
-vincent
Back to top
View user's profile Send private message
mvic
PostPosted: Fri Apr 21, 2006 1:54 pm    Post subject: Re: Improve throughput Reply with quote

Jedi

Joined: 09 Mar 2004
Posts: 2080

vincent wrote:
Can anyone help me to improve this throughput?
-vincent

I'm sure we'd love to help. Please take a little time to say what analysis and troubleshooting you've already done. Also, why expect 1500 - what is the reason to pick that number?

My approach would be to turn on MQ trace on the client and server for 10 seconds or so, while you are running your high throughput testcase. Then read the trace and look for where the time is being spent. Is it mainly being spent within MQ, within the network, or within the app? Or is the time evenly distributed? Could it be that you are running as fast as possible already? At first sight, 500 msg/sec over a network seems pretty good.

Are you using large messages, persistent messages, messages within syncpoint...? All of these are relevant factors.

Hopefully this helps
Back to top
View user's profile Send private message
vincent
PostPosted: Fri Apr 21, 2006 2:23 pm    Post subject: Reply with quote

Voyager

Joined: 21 Apr 2006
Posts: 81

No. The message size does not exceed 2K and yes the messages are persisted and no syncpoint . Do you think that 1500 msg/sec is achievable?

This the trading business. so the business expects that it can touch upto 1500/sec in the peak stage.

Before turning on the trace. Would the following fine turning help to improve much?
1. The default message lenght is being setup, ie 4MB, can it be reduced to 2KB in bothe Manager and Q - level
2. By seperating the volume of data and log would help
3. Writting log parameter can be changed into either SingleWrite/DoubleWrite. By defaul, TripleWrite

-Vincent
Back to top
View user's profile Send private message
mvic
PostPosted: Fri Apr 21, 2006 2:39 pm    Post subject: Reply with quote

Jedi

Joined: 09 Mar 2004
Posts: 2080

vincent wrote:
No. The message size does not exceed 2K and yes the messages are persisted and no syncpoint . Do you think that 1500 msg/sec is achievable?

As always with such questions, It Depends. It depends on how fast your disks are, how fast your network is, how fast your putting app goes, and how fast your getting app goes. Further, if you are using persistent messages, then you have to be aware that the disk is going to be very busy.

Quote:
This the trading business. so the business expects that it can touch upto 1500/sec in the peak stage.

What has to happen 1500 times a second? Is it
* client app puts request, waits for reply
* server picks up request, produces reply
* client app gets reply
or something else?

Quote:
Before turning on the trace. Would the following fine turning help to improve much?
1. The default message lenght is being setup, ie 4MB, can it be reduced to 2KB in bothe Manager and Q - level
2. By seperating the volume of data and log would help
3. Writting log parameter can be changed into either SingleWrite/DoubleWrite. By defaul, TripleWrite

Again it depends. I would expect 1 to have no effect. I'd expect 2 and 3 to have an effect, but the size of the effect is dependent upon latencies of disk and network. You really do have to play with a test system and produce some hard numbers.
Back to top
View user's profile Send private message
ashoon
PostPosted: Fri Apr 21, 2006 3:00 pm    Post subject: servers vs. clients? Reply with quote

Master

Joined: 26 Oct 2004
Posts: 235

might be a good time to look at MQ servers on the windows systems vs. clients and MQ clustering to provide scalability and continous availability of your system.



vincent wrote:
No. The message size does not exceed 2K and yes the messages are persisted and no syncpoint . Do you think that 1500 msg/sec is achievable?

This the trading business. so the business expects that it can touch upto 1500/sec in the peak stage.

Before turning on the trace. Would the following fine turning help to improve much?
1. The default message lenght is being setup, ie 4MB, can it be reduced to 2KB in bothe Manager and Q - level
2. By seperating the volume of data and log would help
3. Writting log parameter can be changed into either SingleWrite/DoubleWrite. By defaul, TripleWrite

-Vincent
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Fri Apr 21, 2006 5:06 pm    Post subject: Re: servers vs. clients? Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7722

vincent wrote:

being retrieved by Windows client through JMS program.

JMS is the slowest MQ API.

Are the JMS clients doing selective MQ gets? If yes, how, as the way you do selective gets has huge swings in performance.

We've done some performance testing with JMS clients getting messages as fast as they can. If you are getting 500 a second, pat yourself on the back. I have to assume this is multiple MQClients pulling from the queue concurrently, right?

mvic said it; u need a trace to show where the time is being spent.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
JLRowe
PostPosted: Sat Apr 22, 2006 12:48 am    Post subject: Reply with quote

Yatiri

Joined: 25 May 2002
Posts: 664
Location: South East London

I think with a single client, you are doing very well to get 500 messages/second. You have to remember that each message is a round trip on the network, so latency is always going to be the biggest factor rather than the actual transmission time of the message over the wire.

If you want more client throughput than this, then you are going to need multiple consumers. Otherwise, switch to using a server.
Back to top
View user's profile Send private message Send e-mail
vincent
PostPosted: Sat Apr 22, 2006 7:08 am    Post subject: Reply with quote

Voyager

Joined: 21 Apr 2006
Posts: 81

The total throughput should be 1500/sec. clinet app is able to put around 4000/sec on the server but the clinet sitting on windows is able to process around 500/s. Need to improve it to 1500/s .

If additional client is added then would it improve this througput?
-vincent.
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Sat Apr 22, 2006 4:58 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY

Have you tested it?

Everything else being equal I found out that the progression is not fully linear. Yes adding multiple clients will improve throughput but at the same time it may increase the time any individual client uses for processing the message.

You really need to test to come up with a base line that will satisfy or outperform your requirement.
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
jefflowrey
PostPosted: Sat Apr 22, 2006 5:55 pm    Post subject: Reply with quote

Grand Poobah

Joined: 16 Oct 2002
Posts: 19981

Is your JMS client inside a J2EE container, or outside?

This can make a big difference, both in speed and in your ability to tune.
_________________
I am *not* the model of the modern major general.
Back to top
View user's profile Send private message
vincent
PostPosted: Mon Apr 24, 2006 5:04 am    Post subject: Reply with quote

Voyager

Joined: 21 Apr 2006
Posts: 81

JMS is outside J2EE container
-vincent
Back to top
View user's profile Send private message
vincent
PostPosted: Mon Apr 24, 2006 5:32 am    Post subject: Reply with quote

Voyager

Joined: 21 Apr 2006
Posts: 81

Why JMS client would yield less message from Q? is that because of JVM?
-vincent
Back to top
View user's profile Send private message
vincent
PostPosted: Mon Apr 24, 2006 5:34 am    Post subject: Reply with quote

Voyager

Joined: 21 Apr 2006
Posts: 81

By having seperate Q Manager in Windows and Solair and clustering these two would give more throughput?
-vincent.
Back to top
View user's profile Send private message
zpat
PostPosted: Mon Apr 24, 2006 5:35 am    Post subject: Reply with quote

Jedi Council

Joined: 19 May 2001
Posts: 5866
Location: UK

Increase the number of instances/threads of the client application (reading the same queue) until max throughput is reached.

Persistent messages are generally disk speed I/O limited. Separate the MQ logs and queues onto separate physical disks. Use the highest performing disks you can get (also fast-write caching will help enormously). Make sure the physical disks are dedicated to MQ.

If using a SAN, check how the underlying physical disks are being assigned as you may think you have dedicated disks when you actually have not.
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Mon Apr 24, 2006 5:41 am    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7722

vincent wrote:
Why JMS client would yield less message from Q? is that because of JVM?
-vincent

JMS is the slowest MQ API. There are supporting numbers in the Performance Support Pacs and at sessions at the MQ Tech Conferances.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum Index » IBM MQ Installation/Configuration Support » Improve throughput
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.