| Author |
Message
|
| vincent |
 Posted: Fri Apr 21, 2006 1:45 pm Post subject: Improve throughput |
 |
|
Voyager
Joined: 21 Apr 2006
Posts: 81
|
Loacal Application is putting messages on Local Queue Manager on Solaris-8 but this is being retrieved by Windows client through JMS program. There is no complex setup. We are expecting 1500 per second throughput but we are getting only 400/500 as an output to the client program. Can anyone help me to improve this throughput?
-vincent |
|
| Back to top |
|
 |
| mvic |
| Posted: Fri Apr 21, 2006 1:54 pm Post subject: Re: Improve throughput |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| vincent wrote: |
Can anyone help me to improve this throughput?
-vincent |
I'm sure we'd love to help. Please take a little time to say what analysis and troubleshooting you've already done. Also, why expect 1500 - what is the reason to pick that number?
My approach would be to turn on MQ trace on the client and server for 10 seconds or so, while you are running your high throughput testcase. Then read the trace and look for where the time is being spent. Is it mainly being spent within MQ, within the network, or within the app? Or is the time evenly distributed? Could it be that you are running as fast as possible already? At first sight, 500 msg/sec over a network seems pretty good.
Are you using large messages, persistent messages, messages within syncpoint...? All of these are relevant factors.
Hopefully this helps |
|
| Back to top |
|
|
| vincent |
| Posted: Fri Apr 21, 2006 2:23 pm Post subject: |
|
|
Voyager
Joined: 21 Apr 2006
Posts: 81
|
No. The message size does not exceed 2K and yes the messages are persisted and no syncpoint . Do you think that 1500 msg/sec is achievable?
This the trading business. so the business expects that it can touch upto 1500/sec in the peak stage.
Before turning on the trace. Would the following fine turning help to improve much?
1. The default message lenght is being setup, ie 4MB, can it be reduced to 2KB in bothe Manager and Q - level
2. By seperating the volume of data and log would help
3. Writting log parameter can be changed into either SingleWrite/DoubleWrite. By defaul, TripleWrite
-Vincent |
|
| Back to top |
|
|
| mvic |
| Posted: Fri Apr 21, 2006 2:39 pm Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| vincent wrote: |
| No. The message size does not exceed 2K and yes the messages are persisted and no syncpoint . Do you think that 1500 msg/sec is achievable? |
As always with such questions, It Depends. It depends on how fast your disks are, how fast your network is, how fast your putting app goes, and how fast your getting app goes. Further, if you are using persistent messages, then you have to be aware that the disk is going to be very busy.
| Quote: |
| This the trading business. so the business expects that it can touch upto 1500/sec in the peak stage. |
What has to happen 1500 times a second? Is it
* client app puts request, waits for reply
* server picks up request, produces reply
* client app gets reply
or something else?
| Quote: |
Before turning on the trace. Would the following fine turning help to improve much?
1. The default message lenght is being setup, ie 4MB, can it be reduced to 2KB in bothe Manager and Q - level
2. By seperating the volume of data and log would help
3. Writting log parameter can be changed into either SingleWrite/DoubleWrite. By defaul, TripleWrite |
Again it depends. I would expect 1 to have no effect. I'd expect 2 and 3 to have an effect, but the size of the effect is dependent upon latencies of disk and network. You really do have to play with a test system and produce some hard numbers. |
|
| Back to top |
|
|
| ashoon |
| Posted: Fri Apr 21, 2006 3:00 pm Post subject: servers vs. clients? |
|
|
Master
Joined: 26 Oct 2004
Posts: 235
|
might be a good time to look at MQ servers on the windows systems vs. clients and MQ clustering to provide scalability and continous availability of your system.
| vincent wrote: |
No. The message size does not exceed 2K and yes the messages are persisted and no syncpoint . Do you think that 1500 msg/sec is achievable?
This the trading business. so the business expects that it can touch upto 1500/sec in the peak stage.
Before turning on the trace. Would the following fine turning help to improve much?
1. The default message lenght is being setup, ie 4MB, can it be reduced to 2KB in bothe Manager and Q - level
2. By seperating the volume of data and log would help
3. Writting log parameter can be changed into either SingleWrite/DoubleWrite. By defaul, TripleWrite
-Vincent |
|
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Apr 21, 2006 5:06 pm Post subject: Re: servers vs. clients? |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| vincent wrote: |
being retrieved by Windows client through JMS program.
|
JMS is the slowest MQ API.
Are the JMS clients doing selective MQ gets? If yes, how, as the way you do selective gets has huge swings in performance.
We've done some performance testing with JMS clients getting messages as fast as they can. If you are getting 500 a second, pat yourself on the back. I have to assume this is multiple MQClients pulling from the queue concurrently, right?
mvic said it; u need a trace to show where the time is being spent.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| JLRowe |
| Posted: Sat Apr 22, 2006 12:48 am Post subject: |
|
|
Yatiri
Joined: 25 May 2002
Posts: 664
Location: South East London
|
I think with a single client, you are doing very well to get 500 messages/second. You have to remember that each message is a round trip on the network, so latency is always going to be the biggest factor rather than the actual transmission time of the message over the wire.
If you want more client throughput than this, then you are going to need multiple consumers. Otherwise, switch to using a server. |
|
| Back to top |
|
|
| vincent |
| Posted: Sat Apr 22, 2006 7:08 am Post subject: |
|
|
Voyager
Joined: 21 Apr 2006
Posts: 81
|
The total throughput should be 1500/sec. clinet app is able to put around 4000/sec on the server but the clinet sitting on windows is able to process around 500/s. Need to improve it to 1500/s .
If additional client is added then would it improve this througput?
-vincent. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sat Apr 22, 2006 4:58 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Have you tested it?
Everything else being equal I found out that the progression is not fully linear. Yes adding multiple clients will improve throughput but at the same time it may increase the time any individual client uses for processing the message.
You really need to test to come up with a base line that will satisfy or outperform your requirement.
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Sat Apr 22, 2006 5:55 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Is your JMS client inside a J2EE container, or outside?
This can make a big difference, both in speed and in your ability to tune.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| vincent |
| Posted: Mon Apr 24, 2006 5:04 am Post subject: |
|
|
Voyager
Joined: 21 Apr 2006
Posts: 81
|
JMS is outside J2EE container
-vincent |
|
| Back to top |
|
|
| vincent |
| Posted: Mon Apr 24, 2006 5:32 am Post subject: |
|
|
Voyager
Joined: 21 Apr 2006
Posts: 81
|
Why JMS client would yield less message from Q? is that because of JVM?
-vincent |
|
| Back to top |
|
|
| vincent |
| Posted: Mon Apr 24, 2006 5:34 am Post subject: |
|
|
Voyager
Joined: 21 Apr 2006
Posts: 81
|
By having seperate Q Manager in Windows and Solair and clustering these two would give more throughput?
-vincent. |
|
| Back to top |
|
|
| zpat |
| Posted: Mon Apr 24, 2006 5:35 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Increase the number of instances/threads of the client application (reading the same queue) until max throughput is reached.
Persistent messages are generally disk speed I/O limited. Separate the MQ logs and queues onto separate physical disks. Use the highest performing disks you can get (also fast-write caching will help enormously). Make sure the physical disks are dedicated to MQ.
If using a SAN, check how the underlying physical disks are being assigned as you may think you have dedicated disks when you actually have not. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Mon Apr 24, 2006 5:41 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| vincent wrote: |
Why JMS client would yield less message from Q? is that because of JVM?
-vincent |
JMS is the slowest MQ API. There are supporting numbers in the Performance Support Pacs and at sessions at the MQ Tech Conferances.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
|
|
