ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » Clustering » Cluster Workload Balancing- a specific scenario

Post new topic  Reply to topic Goto page Previous  1, 2, 3  Next
 Cluster Workload Balancing- a specific scenario « View previous topic :: View next topic » 
Author Message
bruce2359
PostPosted: Wed Mar 07, 2012 5:35 pm    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

I'm guessing you meant BIND_NOT_FIXED, rather than BIND_NOT_OPEN.

I was referring to the basic behavior of cluster workload exit provided, or user-supplied. Why? The OP made no reference to WMQ version.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
anveshita
PostPosted: Thu Mar 08, 2012 4:49 am    Post subject: Reply with quote

Master

Joined: 27 Sep 2004
Posts: 254
Location: Jambudweepam

Bruece2359:
MQ Version is 6.1

Vitor:
Quote:
Also, why do you need it to be so even? If there's been a period of inactivity (i.e. no messages) why does it matter if the message is routed to the same queue manager that handled the last message?

Think these three QMs are serviced by three instances of a same application. If there is a batch load of 75 messages each QM gets 25 messages ( not exactly, but little variance). Now I have online sends say 30 messages, what I find QM1 is getting 12+ messages and QM3 getting 18+ messages most of the time QM2 is getting 0 or 1 message. This increases load on apps serving QM1 and QM2 while the app on QM2 is idling.

Quote:
Or if 3 single messages arrive 3 minutes apart they're all routed to the same queue manager

Yes I find the similar behavior. The pattern I explained above do not happen all the time, but sometimes ( but less frequently) QM2 is getting good chunk of messages.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Thu Mar 08, 2012 6:15 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

25/25/25 (approximately) is probably as good as it gets.

You may modify and replace the IBM-supplied cluster workload management exit with one of your own, and have it distribute messages as you desire. In reality, it will still be approximately as you desire.

V7 offers a much more robust set of cluster workload management tools, but makes no guarantees.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
Vitor
PostPosted: Thu Mar 08, 2012 6:23 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

anveshita wrote:
Now I have online sends say 30 messages, what I find QM1 is getting 12+ messages and QM3 getting 18+ messages most of the time QM2 is getting 0 or 1 message. This increases load on apps serving QM1 and QM2 while the app on QM2 is idling.


Aside from any of the provided advice which could be skewing the message distribution, is the fact that QM1 & QM3 are getting an unfair amount of messages a problem? It doesn't sound like there's a load issue, as under load skewing doesn't occur. So what exactly is the issue here apart from an absense of natural justice?

You should consider upgrading the WMQ version, and it's worth double checking that there's no user written workload exit in play. I very much doubt there is given how it reacts under load but you never can tell.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
anveshita
PostPosted: Thu Mar 08, 2012 6:25 am    Post subject: Reply with quote

Master

Joined: 27 Sep 2004
Posts: 254
Location: Jambudweepam

Bruce2359:
Yes. But this is with Batch. THe issue is with the online messages which are not getting load balanced and needs to fixed so all servers are loaded evenly( at least near equal)
Back to top
View user's profile Send private message
Vitor
PostPosted: Thu Mar 08, 2012 6:41 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

anveshita wrote:
THe issue is with the online messages which are not getting load balanced and needs to fixed so all servers are loaded evenly( at least near equal)


I'll ask if you've checked the online applications are using the correct binding options as indicated above, even though you probably have.

I'll also ask again why it's so important for low volume intermittent online messages to be evenly balanced. If it was 2000 online users resulting in 300 tps I'd see the problem. But with those volumes, the problem might be clearer to see.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
bruce2359
PostPosted: Thu Mar 08, 2012 7:04 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

anveshita wrote:
Bruce2359:
Yes. But this is with Batch. THe issue is with the online messages which are not getting load balanced and needs to fixed so all servers are loaded evenly( at least near equal)

Batch and online apps usually behave differently. And, most likely, you have more online apps (transactions) than batch apps.

A batch app usually MQCONNects and MQOPENs once, then MQPUT lots of messages; then ends the app.

An online app usually MQCONNects, MQOPENs, and MQPUTs a single message; then ends the app.

Likely, you have more online app instances than batch.

Again, much depends on the app(s), and how they open the queue - with MQOO_BIND_ON_OPEN vs. MQOO_BIND_NOT_FIXED. Do all the apps do the same bind option?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Thu Mar 08, 2012 7:19 pm    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7717

This non-perfect load balancing has been discussed in depth multiple times on this site already.

MQ load balances in a cluster at the channel level, not the individual queue level. Administrative cluster messages and other app's messages going over the channels drive the channels' sequence numbers. The cluster workload algorithm considers how fast these numbers are rising. If all other things are equal, and the sending QM has 4 destination QMs to load balance across for your queue, but 2 of those destination QMs are getting more traffic because perhaps another app's queues only exist on those 2 QMs, then your load balancing will more heavily, or maybe even completely, favor the other 2 QMs.


Additionally, the numbers you are talking about are very very small. This matters because another thing the cluster workload algorithm considers is the status of the channels. With only a few messages being put occasionally, the channels may be inactive the next time your tiny little group of messages comes along asking to be load balanced. By the time the second channel to QM2 has started and gotten to a RUNNING status, and thus eligible for taking load, the first channel to QM1 may have already moved some if not all of your messages to QM1.


Move large groups of messages with no other traffic in the cluster with all the channels running and the numbers will be very very close to perfectly balanced.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Thu Mar 08, 2012 7:23 pm    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7717

bruce2359 wrote:
I'm guessing you meant BIND_NOT_FIXED, rather than BIND_NOT_OPEN.

I was referring to the basic behavior of cluster workload exit provided, or user-supplied. Why? The OP made no reference to WMQ version.


Yes, I meant BIND_NOT_FIXED. I corrected my post.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
anveshita
PostPosted: Tue Mar 20, 2012 11:04 am    Post subject: Reply with quote

Master

Joined: 27 Sep 2004
Posts: 254
Location: Jambudweepam

Vitor wrote:

Aside from any of the provided advice which could be skewing the message distribution, is the fact that QM1 & QM3 are getting an unfair amount of messages a problem? It doesn't sound like there's a load issue, as under load skewing doesn't occur. So what exactly is the issue here apart from an absense of natural justice?


While the messages from batch are continuous and loaded evenly, the messages from online are going to few servers which is resulting in more load to certain servers.

The application is not setting any OPEN/PUT options. The queues are defined with DEFBIND(OPEN).

Quote:

You should consider upgrading the WMQ version, and it's worth double checking that there's no user written workload exit in play. I very much doubt there is given how it reacts under load but you never can tell.


There are no user defined exits.
Back to top
View user's profile Send private message
mqjeff
PostPosted: Tue Mar 20, 2012 11:56 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

anveshita wrote:
The application is not setting any OPEN/PUT options. The queues are defined with DEFBIND(OPEN).


And how does that affect the load balancing?
Back to top
View user's profile Send private message
Vitor
PostPosted: Tue Mar 20, 2012 12:12 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

anveshita wrote:
The application is not setting any OPEN/PUT options. The queues are defined with DEFBIND(OPEN).


And you posted asking why the load balancing wasn't working...?
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
PeterPotkay
PostPosted: Tue Mar 20, 2012 12:24 pm    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7717

anveshita wrote:
The application is not setting any OPEN/PUT options.

Then your app is not working. At all. You can't open a queue with no options specified and do anything with that queue.
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
bruce2359
PostPosted: Tue Mar 20, 2012 1:26 pm    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

anveshita wrote:
The application is not setting any OPEN/PUT options. The queues are defined with DEFBIND(OPEN).

From this, I gather that you are not a programmer.

Did you look at the application code? Or did you ask a programmer to look at the code?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
anveshita
PostPosted: Tue Mar 20, 2012 2:37 pm    Post subject: Reply with quote

Master

Joined: 27 Sep 2004
Posts: 254
Location: Jambudweepam

Wow
The application is not a regular program that does the put. The messages are put by a Datapower device. When I indicated it does not set any put/open options I meant to say as related to the binding
No I am not a datapower programmer. my understanding is that if binding is not specified the application(datapower in this case) then the options set on the queue will be the default.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page Previous  1, 2, 3  Next Page 2 of 3

MQSeries.net Forum Index » Clustering » Cluster Workload Balancing- a specific scenario
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.