| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| Performance: Selectors vs. Many Queues |
« View previous topic :: View next topic » |
| Author |
Message
|
| dtauzell |
 Posted: Thu Jul 11, 2002 10:17 am Post subject: Performance: Selectors vs. Many Queues |
 |
|
Apprentice
Joined: 23 May 2002
Posts: 37
|
Is it better to have a bunch of threads ( 100+ ) listening on one queue with selectors (to route messages to the appropriate thread) or to have each thread listening on its own queue ( 100+ queues )?
Or, is the right answer somewhere in-between? |
|
| Back to top |
|
 |
| bduncan |
| Posted: Thu Jul 11, 2002 10:51 am Post subject: |
|
|
Padawan
Joined: 11 Apr 2001
Posts: 1554
Location: Silicon Valley
|
The main factor to answering your question is knowing the queue depth over time. When applications listen to queues using selectors, they are basically doing linear searches, looking for a match. If I am matching against Correlation Id, and the message I want happens to be at the back of a queue with depth 100, my application ends up doing 100 checks before finding the right message. That's a worst case scenario, but now imagine the queue depth is at 30,000! Of course, the problem here is that as the queue depth increases, the search time increases, which results in a slower processing rate, which means the depth will increase at an ever greater rate!
At one of my previous companies, we had a situation where our mainframe went down for 8 hours. While this was happening, a bunch of messages were piling up on various transmission queues. When the mainframe came back up, its queues were flooded with these messages. We had 300,000 messages on each of several queues. The problem was, because there were so many messages, and our applications were matching against CorrelId, the search times were much longer compared to normal, when the depth might only reach a couple thousand. Because of this, our processing rate went down so much, that the normal flow of messages to the mainframe was faster than our processing rate, so even though we were only getting a couple messages per second to the mainframe, our processing rate went to less than one per second, so the queue depths continued to increase. Finally, the solution was to shut down the channels, and wait for the mainframe to process what was already on its queues, which took several days. In the meantime, tons of messages piled up on the transmission queues, and to avoid a repeat of what caused this whole mess, we only started one channel at a time, and waited to process all those messages before starting another. In the end, it took over 1 week to correct this problem.
What's this all mean? Well in my opinion, avoid listening with selectors at all costs! Many times you can't avoid it, and that's fine. But if your architecture permits you the choice, I'd always go with more queues which allows you to write your applications in such a way that they are simply grabbing the first message on the queue (constant search time) instead of selecting a particular message (linear search time).
_________________
Brandon Duncan
IBM Certified MQSeries Specialist
MQSeries.net forum moderator |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
