ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexWebSphere DataPowerworkload management question

Post new topicReply to topic Goto page 1, 2  Next
workload management question View previous topic :: View next topic
Author Message
anveshita
PostPosted: Fri Apr 22, 2011 9:20 am Post subject: workload management question Reply with quote

Master

Joined: 27 Sep 2004
Posts: 246
Location: Jambudweepam

My knowledge of Datapower is limited, but I would like to know how DP can be used in this particular situation.


I have a queue Q where I get messages which are a mix of small and large. The small messages (Msg_Small) constitutes say 80% . Eash Msg_Small gets processed in 1 sec and large messages (Msg_Large) constitues 20%. Eash Msg_Large gets processed in 5-6 sec.
For the time being let us assume I could identify the type of the message(MSG_large vs MSg_small)before feeding to the queue Q.
The messages from Q are sent to queues Q1,Q2,Q3,Q4 in a round robin fashion.
We have consumers P1,P2,P3 and P4 that consume the messages from queues Q1,Q2,Q3,Q4.


Now the issue is since I have a mix of Msg_Large and Msg_Small messages with different processing times.
By having a round-robin logic in feeding the queues Q1,Q2,Q3 and Q4, I am finding most of the times Msg_Smalls are getting stuck behing Msg_Larges and resulting in timeouts.


Now instead of using a round-robin routing logic, I would like to some type of intelligent router say (IR).
I want to allocate Q1 and Q2 for processing Msg_Small and allocate Q3 and Q4 for processing msg_Large. Msg_small can be routed to Q3 and Q4 if the queues are empty
IR will send messages to Q1,Q2, Q3 and Q4 based on the queue depth instead of round robin. In case of
Msg_small, since these are of high volume, we will route the messages to Q3 and Q4 if the queue depth is zero, so that the process P3 and P4 can process these small messages. But Msg_Large will always go to queues Q3 and Q4.
Now my questions:
1. Is this a right approach for work load balancing or are there any established patterns for such sort of situations?
2. Can the IR router I indicated above can be done on Datapower? Can the configuration changes can be changed on the fly. Think of as property file changes for a running java program. For example I may wish to revert to round-robin instead of IR logic.
3. If #2 is yes, then can you indicate the steps in general one has to follow to set up the router? ( I can go read the manuals but need the steps so I know what to read.)

Please let me know your comments
Back to top
View user's profile Send private message
anveshita
PostPosted: Tue Apr 26, 2011 7:52 am Post subject: Reply with quote

Master

Joined: 27 Sep 2004
Posts: 246
Location: Jambudweepam

Any help?
Back to top
View user's profile Send private message
WMBDEV1
PostPosted: Tue Apr 26, 2011 7:59 am Post subject: Reply with quote

Sentinel

Joined: 05 Mar 2009
Posts: 888
Location: UK

Have you considered using message priorities?

It may give you a simpler interface.

How quickly do the smaller messages need to be processed?
Back to top
View user's profile Send private message
anveshita
PostPosted: Tue Apr 26, 2011 9:39 am Post subject: Reply with quote

Master

Joined: 27 Sep 2004
Posts: 246
Location: Jambudweepam

Thanks.
Quote:
Have you considered using message priorities?



Could you please explain. I was told assigning priorities and polling the messages based on priority would slow down the process and is not a best practice. ( Not sure where I read).
The smaller ones need to be processed in 0.6 sec
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Tue Apr 26, 2011 2:15 pm Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7557

You don't poll. MQ will put the messages with the higher priorities into the queues in front of the lower priorities. Your consumers just process the messages as they are on the queue. Small messages with higher priorities will be always in front of large messages with lower priorities. But if a consumer is already working on a large message, small messages will wait on that queue until the consumer is done with the large. Either have multiple instances of the consuming app watching each queue, so there is always an available thread waiting for a new small message, or split the messages into 2 separate sets of queues - 1 set for small and 1 set for big, and use something like WMB to route them to the correct queues.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Tue Apr 26, 2011 8:06 pm Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20106
Location: LI,NY

PeterPotkay wrote:
You don't poll. MQ will put the messages with the higher priorities into the queues in front of the lower priorities. Your consumers just process the messages as they are on the queue. Small messages with higher priorities will be always in front of large messages with lower priorities. But if a consumer is already working on a large message, small messages will wait on that queue until the consumer is done with the large. Either have multiple instances of the consuming app watching each queue, so there is always an available thread waiting for a new small message, or split the messages into 2 separate sets of queues - 1 set for small and 1 set for big, and use something like WMB to route them to the correct queues.


The assumption here is that your message delivery method for the queue is the default (priority) and not set to FIFO.

Also having a consumer with a selector on high priority can assure you that only messages with high priority will be processed by this consumer. You can then have an additional consumer (either without selector or with a selector on low priority) will allow you to consume the other messages.

Have fun
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
anveshita
PostPosted: Wed Apr 27, 2011 11:49 am Post subject: Reply with quote

Master

Joined: 27 Sep 2004
Posts: 246
Location: Jambudweepam

PeterPotkay:
Message priorties may help, but I am leaning towards seperating the messages to different queues. But the downside will be what to do if I need to classify messages further down the line, for ex: small,large and batch/online. Having separate queues for each classification may not scale well in such situations.


fjb_saper:
Quote:
Also having a consumer with a selector on high priority can assure you that only messages with high priority will be processed by this consumer. You can then have an additional consumer (either without selector or with a selector on low priority) will allow you to consume the other messages.


The consumer is a packaged application I do not have much control. It is a .NET app. I believe selectors are applicable to JMS clients in MQV6.0 which we are using. If I understand it is provided on MQ server side with MQOPEN calls going forward with MQ V 7.0. Hence this may not work for me.

As a follow up, indicated in OP, is it possible to implement "least queue depth" pattern efficiently. Means the PUT application would post to the queue with least number of messages contrary to the regular "round-robin pattern" we use.
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Wed Apr 27, 2011 4:28 pm Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7557

anveshita wrote:

As a follow up, indicated in OP, is it possible to implement "least queue depth" pattern efficiently. Means the PUT application would post to the queue with least number of messages contrary to the regular "round-robin pattern" we use.


So you will be happy sending your next small message to a queue with a Q depth of 3 instead of a q depth of 4? What if the q with a Q depth of 3 has 3 large messages, and the 4 q depth is 4 small messages. (Don't propose to then browse each queue to see how big the messages are!)

Not only can you not rely on the q depth to tell you exactly how many available messages are really there, it will not perform well if you have to constantly poll each q depth to see what the # is before each and every put.

Polling for queue depth to make decisions on where to put messages is a bad design. If it was a good idea, IBM would have built that type of option into basic MQ clustering.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
anveshita
PostPosted: Thu Apr 28, 2011 4:57 am Post subject: Reply with quote

Master

Joined: 27 Sep 2004
Posts: 246
Location: Jambudweepam

Quote:
So you will be happy sending your next small message to a queue with a Q depth of 3 instead of a q depth of 4? What if the q with a Q depth of 3 has 3 large messages, and the 4 q depth is 4 small messages. (Don't propose to then browse each queue to see how big the messages are!)


Not necessarily. I still need to have queues separated out for small and large. In some of the queuing theory simulations, I have seen an options to route messages to a queue with least depth. Instead of round-robin way. In my case I will have small messages sent to separate queues. With in the queues meant for small messages I would like to follow "least queue depth".

You are right IBM could have done this easily. But did not, I would like to know a reason if there is any. This is my idea, you are welcome to comment on it. Set up PCF events to be generated on a QM for queue depth and route the messages to a pre-defined queue say pcfQ. A trigger on PCF queue will invoke a program that reads the message and updates a table or file with the latest message depth. The router reads the table to know the queue with least depth and sends the message to it. I know I am missing lot of details, but that is just an idea.
Back to top
View user's profile Send private message
WMBDEV1
PostPosted: Thu Apr 28, 2011 5:17 am Post subject: Reply with quote

Sentinel

Joined: 05 Mar 2009
Posts: 888
Location: UK

anveshita wrote:
This is my idea, you are welcome to comment on it.


Ok, I will..... I think you're now trying to implement a solution for a problem that doesnt really exist and wasnt the original problem you presented.

In the process of doing this you are creating a support nightmare.
Back to top
View user's profile Send private message
Vitor
PostPosted: Thu Apr 28, 2011 5:29 am Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 25823
Location: Ohio, USA

anveshita wrote:
You are right IBM could have done this easily. But did not, I would like to know a reason if there is any.


Because it's not a good idea and not as easy as it sounds.

anveshita wrote:
This is my idea, you are welcome to comment on it. Set up PCF events to be generated on a QM for queue depth and route the messages to a pre-defined queue say pcfQ. A trigger on PCF queue will invoke a program that reads the message and updates a table or file with the latest message depth. The router reads the table to know the queue with least depth and sends the message to it. I know I am missing lot of details, but that is just an idea.


Let's see now:

- so you make a PCF call for each inbound message.
- once this call has executed and the message interpreted a table's updated
- this table is read and the message sent

All this takes how long? For each message?
What happens when you get high message volume and the table updates either single thread or deadlock?
What happens when the database is unavailable? At best you need to pay for sufficient hardware/software to make it HA, and in the event of a ham-fisted DBA it'll still go down.

As my associate says, this is a nightmare. It could probably be made to work, but it'll be a rod for your own back.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
anveshita
PostPosted: Thu Apr 28, 2011 5:30 am Post subject: Reply with quote

Master

Joined: 27 Sep 2004
Posts: 246
Location: Jambudweepam

WMBDEV1 wrote:
anveshita wrote:
This is my idea, you are welcome to comment on it.


Ok, I will..... I think you're now trying to implement a solution for a problem that doesnt really exist and wasnt the original problem you presented.

In the process of doing this you are creating a support nightmare.

Thanks.
But that I stated in my OP.
Quote:
IR will send messages to Q1,Q2, Q3 and Q4 based on the queue depth instead of round robin. In case of
Msg_small, since these are of high volume, we will route the messages to Q3 and Q4 if the queue depth is zero, so that the process P3 and P4 can process these small messages
Back to top
View user's profile Send private message
WMBDEV1
PostPosted: Thu Apr 28, 2011 6:14 am Post subject: Reply with quote

Sentinel

Joined: 05 Mar 2009
Posts: 888
Location: UK

Ok. It had been a while since I had read your OP but to me the real problem here is:
Quote:
I am finding most of the times Msg_Smalls are getting stuck behing Msg_Larges and resulting in timeouts.


We are really debating your solution to this issue now and trying to help you make a solution that is supportable going forward.
Back to top
View user's profile Send private message
anveshita
PostPosted: Thu Apr 28, 2011 8:31 am Post subject: Reply with quote

Master

Joined: 27 Sep 2004
Posts: 246
Location: Jambudweepam

WMBDEV1 wrote:
We are really debating your solution to this issue now and trying to help you make a solution that is supportable going forward.


Thank you. I appreciate your input.

Taking that "least queue depth" is not a choice I am sticking to the round robin pattern.

Now referring back to my OP: The solution so far I have is:
--------------
Group Q1 and Q2 to handle small messages and Q3 and Q4 to handle large messages. Process messages from Q1 with consumer P1, Q2 with P2, Q3->P3,Q4->p4
-----------
I see a problem with the above. I can not configure P3 to process messages from other queues if there are no messages in Q3 as it is a packaged application. Now if Q3 is empty then I will not be using P3 at all. Ideally I would like small messages to go to Q3 and Q4 in case the queues are empty. Is there a better way to achieve this? or you have different approach to this problem. Please share
Back to top
View user's profile Send private message
Vitor
PostPosted: Thu Apr 28, 2011 9:07 am Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 25823
Location: Ohio, USA

anveshita wrote:
Is there a better way to achieve this? or you have different approach to this problem.


What proportion of small messages to large do you have? What is the difference in SLA?

Following from this:

- How did you arrive at 4 queues? Is that just for illustrative purposes in this thread?

- Why are there the same number of queues for large and small messages? As you talked about timeouts. I'm assuming here the small messages have a short SLA? Why do you not have 3 queues (using this example) working the short messages & 1 chewing through the long?

- Why is there only 1 processor per queue? Is this a restriction of the package or just how you've written this example? It's unusual for any application, even a package, to open a queue exclusively.

- Why are you so keen to avoid idle queues (by routing different message types)?

Consider this:

You have 2 service classes of message; large & small. We assume you can route them by size using WMB or similar (I don't know DataPower especially). One possible pattern is this

SQA (Small Queue A) serviced by Process 1, Process 2, Process 3
SQB with P4, P5, P6
SQC with P7, P8, P9

Then one Large Queue with enough processes that the large messages are always processed within SLA.

Clearly you could have many more queues with many more processes, probably split over a number of machines to maximize resources.

This is a basic illustration only, other solutions are undoubtably possible & may be better, no warranty express or implied is supplied with this solution and no responsibility is accepted for any loss or damage resulting from it's use howsoever caused.


_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum IndexWebSphere DataPowerworkload management question
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.