| Author |
Message
|
| xmw200 |
 Posted: Wed Sep 13, 2006 9:36 am Post subject: MQ get message slow performance on AIX |
 |
|
Newbie
Joined: 13 Sep 2006
Posts: 6
|
Hi, experts:
I met one performance problem when getting message from MQ queue of AIX machine(mq server is at this machine), every q.get(msg, mgo) method will consue about 6 micro seconds. Although it seems that it's very small, but we need every second can handle 300-500 messages, so 6 micro seconds is very long. In this case, we don't need to consider transaction, we just get it and make some transfomration, and remove it from message.
I also tested MQ get performance, the speed is very fast, almost 0.
Can any person give some suggestions for it? such as parameter tuning.
Any suggestions would be appricated! |
|
| Back to top |
|
 |
| vennela |
| Posted: Wed Sep 13, 2006 10:52 am Post subject: |
|
|
Jedi Knight
Joined: 11 Aug 2002
Posts: 4055
Location: Hyderabad, India
|
| Is it possible to run multiple copies of your application? |
|
| Back to top |
|
|
| xmw200 |
| Posted: Wed Sep 13, 2006 12:45 pm Post subject: |
|
|
Newbie
Joined: 13 Sep 2006
Posts: 6
|
Thanks!
In my post, i missed one word. On my Windows, MQ get is very fast, almost always 0.
I am strange why there is so much difference. |
|
| Back to top |
|
|
| mvic |
| Posted: Wed Sep 13, 2006 1:22 pm Post subject: Re: MQ get message slow performance on AIX |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| xmw200 wrote: |
| every q.get(msg, mgo) method will consue about 6 micro seconds. |
If a Get takes 6 microseconds (0.000006 seconds) then one thread will be able to Get 1 / 0.000006 = approx. 166000 messages per second. So it looks like your performance requirement of 500 messages per second will be OK. |
|
| Back to top |
|
|
| xmw200 |
| Posted: Wed Sep 13, 2006 8:36 pm Post subject: |
|
|
Newbie
Joined: 13 Sep 2006
Posts: 6
|
sorry, i made mistakes again. What i mean is 1/1000 second, not micro seconds.
In this case, one function flow is: MQ get message -> XSLT parse this message -> sequential file.
I found every flow will take 13 * 1/1000 second. So one second it can only handle about 70-80 transactions. I also found within 13, 6/13 is for MQ get , 7/13 is for XSLT.
Any ideas or suggestions? |
|
| Back to top |
|
|
| mvic |
| Posted: Thu Sep 14, 2006 1:10 am Post subject: |
|
|
Jedi
Joined: 09 Mar 2004
Posts: 2080
|
| xmw200 wrote: |
| sorry, i made mistakes again. What i mean is 1/1000 second, not micro seconds. |
FYI 1/1000 second = 0.001 second = 1 millisecond (ms)
If a Get is taking 1 ms, then possibly (a) the message is very large (b) your disks are very slow.
Is your message persistent? If so, use MQGMO_SYNCPOINT to get the message.
Is your message small and non-persistent? Then there is no way it should take 1 ms.
Is your machine heavily loaded? Are your disks busy, or mounted over a network for example?
... just guesses I'm afraid.
| Quote: |
| I found every flow will take 13 * 1/1000 second. |
Have you analysed the other work taking place in the flow? In other words, what makes you sure that MQ is going slow? |
|
| Back to top |
|
|
| xmw200 |
| Posted: Thu Sep 14, 2006 1:52 am Post subject: |
|
|
Newbie
Joined: 13 Sep 2006
Posts: 6
|
Thank you very much.
The messages number is very big. And this disk has many I/Os, addition to MQ. So i think you are right. I will try to move other program which consume IOs to other disks.
BTW, i am new to MQ. Could you tell me how to identify whether one queue(or qmgr) is persitant? In our case, we just need to read message, then move. Don't need consider other features.
Thanks for your help! |
|
| Back to top |
|
|
| wschutz |
| Posted: Thu Sep 14, 2006 2:02 am Post subject: |
|
|
Jedi Knight
Joined: 02 Jun 2005
Posts: 3316
Location: IBM (retired)
|
Neither queue managers nor queues are persistent. Messages are persistent. Message persistence is determine at the time messages a PUT onto the queue by the putting application (there is a default associated with the queue if the application doesn't specify persistence).....
_________________
-wayne |
|
| Back to top |
|
|
| moe |
| Posted: Fri Sep 15, 2006 12:16 am Post subject: |
|
|
Apprentice
Joined: 05 Sep 2006
Posts: 33
Location: Sydney, Australia
|
| Remember to keep it simple, check the basics like mvic suggested. In reality things like busy cpu's, hdd and networks are more likely to have a performance impact then parameters. |
|
| Back to top |
|
|
| xmw200 |
| Posted: Mon Sep 18, 2006 4:58 am Post subject: |
|
|
Newbie
Joined: 13 Sep 2006
Posts: 6
|
Thanks all of your for your expertise advice.
Now we have found the mainly issues is for slow Disk I/O. The AIX machine is very old, about bought 2000 year. When i using 2 java thread to read MQ message, it found every thread will take 11 mili-seconds to get message. If using single thread, it will take about 506 milisseconds.
I want to know whether we can create multiple Qmgrs into different physical disks. And using many same java program(just qmgr, queues are different) to reach parallel processing.
I found another strange thing is when i get one message using java mqget API, i found there will be some write DISK I/O. i think it just read message, can any expert explain it ?
Thank you very much! |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Sep 18, 2006 5:05 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Messages are stored in files, behind the scenes. Just like database records are stored in files behind the scenes.
When you *remove* a message from a queue, the file behind that queue must be updated to indicate that the message is no longer there.
When you get a message from a queue in syncpoint, the file behind the queue must be updated to indicate that the message is unavailable.
You can put multiple queue managers on different physical disks. It is not going to improve your parallelism in the way you want.
The best practices for qmgr performance are to put /var/mqm/qmgrs on a different physical volume than /var/mqm/logs.
The best practice for increasing parallelism is to a) increase the threads in your application, b) increase the number of copies of your application that are running simultaneously AGAINST THE SAME QUEUE, c) scale the *same* queue (same name, really) across multiple queue managers, with many copies of the application reading from that queue on each queue manager.
In order to ensure that you get good performance under multi-threading, you must make sure that no two threads are trying to use the same connection at the same time. You can share connections between threads, but don't use the same one in two threads at once.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| xmw200 |
| Posted: Mon Sep 18, 2006 7:41 am Post subject: |
|
|
Newbie
Joined: 13 Sep 2006
Posts: 6
|
Thanks for your detail explaination!
In this case, MQ message(XML message) will put to DW after they are parsed, we don't need to consider persistance and syncpoint.
For your option a) increase the threads in your application, b) increase the number of copies of your application that are running simultaneously AGAINST THE SAME QUEUE, which has better performance do you think? I have one basic test, i found b) will take more memory, and performance is not good, maybe it related to my Thinkpad perfromance. For a), multiple-thread, i need to keep thread-safe. it is also cost.
Currently, our parser is very fast, the bottleneck is MQ get speed. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Sep 18, 2006 7:52 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
I guess I meant those three to be done in order. Increase the threads, until it's not any more efficient. Then increase the processes. Then scale the queue across multiple queue managers.
If you are talking to a database, in all cases you have to be concerned about database contention as well.
If you can reasonably demonstrate that your application is performance-bound on I/O, then you have a good business justification for increasing the I/O performance. If you can show that the gain to your application will save more money than the costs of a new I/O system or a new server - then you have solved your performance problem.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
