| Author |
Message
|
| KeeferG |
 Posted: Thu Nov 25, 2004 3:18 am Post subject: Performance question with correlid |
 |
|
Master
Joined: 15 Oct 2004
Posts: 215
Location: Basingstoke, UK
|
Just a theoretical quicky.
Each QM on own hardware
8 application QMs
1 monitor QM
Each app QM has 96 threads sending logging info to the monitor QM.
Which would be quickest and anyone know by how much roughly.
8 queues on monitor qm and get by 96 correlids.(fast)
96 queues on monitor qm and get by 8 correlids.(faster)
768 queues on monitor QM and no correlids. (fastest)
_________________
Keith Guttridge
-----------------
Using MQ since 1995 |
|
| Back to top |
|
 |
| PeterPotkay |
| Posted: Thu Nov 25, 2004 7:17 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
How many messages?
How big are the messages?
How fast will the messages arrive?
Do you think the queues will back up, even if you only had only 8 queues?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| KeeferG |
| Posted: Thu Nov 25, 2004 7:30 am Post subject: |
|
|
Master
Joined: 15 Oct 2004
Posts: 215
Location: Basingstoke, UK
|
Each app QM about 100 messages per second
maximum message size of 100 bytes
so messages arriving at 800 per second.
There will be a one to one correlation between putting and getting threads. We are going for the 8 queues with 96 threads getting by correlid but i was wondering what performance differences were.
Couldn't find anything in the performance docs.
_________________
Keith Guttridge
-----------------
Using MQ since 1995 |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Nov 25, 2004 7:54 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Huh. Well, I don't know that there is anything that will tell you exactly what the numbers will be ahead of time. Its dependent on so many different things.
When faced with three options, and no clear cut reason as to which one will perform the best, choose the one that is the simplest and easiest to implement, and test it hard. If it performs adequatly, who cares if another more complicated way is faster (if the simple way is adequate)? The K.I.S.S. theory I guess.
If the monitorl QM is on z/OS, you can idex the queue by correl ID, which improves performance greatly (we did it here for one busy queue).
One thing I will say about many thread getting by CorrelIds on a busy queue...If it can't keep up, it will get worse and worse over time if the volume of messages is constatnt. As the queue gets deeper, the threads have to work that much harder and slower on their gets by correlID (more messages to sift thru), so the queue backs more, so the threads get slower, etc.....
You have to do gets by correlID?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| KeeferG |
| Posted: Thu Nov 25, 2004 8:03 am Post subject: |
|
|
Master
Joined: 15 Oct 2004
Posts: 215
Location: Basingstoke, UK
|
The messages are trace messages from threads inside the applications. The monitoring system is writing a file for each thread on each app QM so its either threads getting by correlid or on thread getting all messages then farming out to the file writing threads. The app guys would prefer MQ to do the filtering for them.
We are running large queue buffers in memory and hope to start some tests soon. I believe it will cope ok. Its more to satisfy my own personal understanding really. All the QM's on solaris so no Indexing available.
Will go probe some of the Hursley guys for some info when i see them next
_________________
Keith Guttridge
-----------------
Using MQ since 1995 |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Nov 25, 2004 8:32 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Having the app do the filtering is a much better idea, really. Why try to optimize your automobile when you can take a shorter route?
It's a small amount of code, and it's going to run faster - and in a constant time regardless of the depth on the queue.
And it keeps the two applications from being coupled in any way other than the data in the message, which is one of the big bonueses of MQ.
The logging application should run as many threads as it needs to give it the performance it needs. This should likely have nothing to do with the number of threads applications on the other side have.
But we've discussed this already.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
