| Author |
Message
|
| Challenger |
 Posted: Tue Mar 03, 2009 7:11 am Post subject: Challenge Question - 03 / 2009 |
 |
|
Centurion
Joined: 31 Mar 2008
Posts: 115
|
My organization has a variety of platforms (Win, UNIX, z/OS, iSeries); various topologies (hub and spoke, point-to-point, cluster); and application models (request-reply, datagram, triggered and not-triggered, batch and real-time).
Management discovered TRIGTYPE(EVERY), and is asking me and my team to offer multiple possibilities for implementing this option in our WMQ environment, a few reasons for the ideas along with any other relevant considerations.
This type of discussion is usually very productive. It demonstrates to management that we know what we're talking about, that we have considered a wide variety of options, and we understand the risks, cost and benefit of our recommendations.
I've read lots of posts explaining why EVERY will not work, and I believe I understand the risks. I've seen few posts where EVERY seems to offer a solution.
Where will TRIGTYPE(EVERY) work? What are the benefits of EVERY? Why does IBM offer this option? How can we effectively manage in this environment? What tools can we use?
Help!
Challenger |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Tue Mar 03, 2009 9:18 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
I would come back and ask you in which environment can you control the resources consumed by the triggered process outside of the triggered process and outside of MQ? That is the only environment I would recommend using trigger every.
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Wed Mar 04, 2009 4:42 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
Trigger Type EVERY can be used if there is a moderate through-put on a queue (say 1 message / second) and each message must be processed as soon as it arrives. It allows for the depth to become more than 1 if the application processing time is longer than the inter message arrival time.
There must be sufficient resources available to cope with the overhead of starting and running multiple copies of the triggered application.
As in all triggered applications, it should loop to get messages and process them until there are no more available, and then cleanly exit.
If there is a high through-put and immediate processing is required, it is better to have a long running application that sits on a MQGET with an unlimited wait. Trigger Type FIRST can be used to initially start this up.
Multiple copies of the long running applications should be run if a single copy cannot keep up with the work load, in which case Trigger Type FIRST is not very useful.
_________________
Glenn |
|
| Back to top |
|
|
| KxK3 |
| Posted: Thu Mar 05, 2009 11:14 am Post subject: |
|
|
Newbie
Joined: 01 Apr 2008
Posts: 6
|
As gbaddeley says using a trigger type of EVERY would cause the queue manager to generate a trigger message every time a message is put to the application queue. This is inefficient because the queue manager would need to do a lot more processing, but I guess if you have an older application that can not process more than one message at a time it would be ok. (CICS?) IBM recommends to use trigger type EVERY if you want a program to process only one message, then finish.
However trigger type of EVERY will be triggered even if there are active applications retrieving from the queue. Where on the other hand if you are using DEPTH or FIRST as a trigger type and there are applications that have the queue open to get messages the trigger will not start. (Because why start an application that is already running?)
This is where I could see the need for trigger type of EVERY. Say you have application that generates requests for insurance quotes, stock, or what have you (pick one example) ...and specify the same reply-to queue. This way each time a reply arrives, the reply could trigger an instance of the server to process the reply (even though an instance of the application may already be running). I guess that would increase the processing time of the requests from the applications level if it was high volume. Still it seems like a lot of overhead.
Not sure what you mean by what 'tools' can we use? Anyhow i think gbaddely got it |
|
| Back to top |
|
|
| Challenger |
| Posted: Thu Mar 05, 2009 3:46 pm Post subject: |
|
|
Centurion
Joined: 31 Mar 2008
Posts: 115
|
Regarding controlling resources:
Like other shops I've worked at, the tail wags the dog. Programmers slam new/updated apps in as they see fit. We are reactive. We try to anticipate peak load, disk space requirements, and the like.
Regarding sufficient resources available to cope with the overhead of starting and running multiple copies of the triggered application:
Right now, and generally speaking, message volume is fairly constant, with some peaks and valleys; but we've managed to meet SLAs - so far.
What if a single instance of the consuming application can't keep up? From other posts here, I know I can manaully start a second instance. EVERY seems to offer automated instantiation as needed.
Regarding what 'tools' can we use?
Let's say that thousands of messages suddenly and unexpectedly arrive in the queue. How do we (what tools can we use to) control (limit) the sudden and seemingly infinite increase in resource demand? This is the puzzling part. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Thu Mar 05, 2009 6:12 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
| Challenger wrote: |
...Regarding what 'tools' can we use?
Let's say that thousands of messages suddenly and unexpectedly arrive in the queue. How do we (what tools can we use to) control (limit) the sudden and seemingly infinite increase in resource demand? This is the puzzling part. |
Here's an idea: The app that is started by trigger EVERY should initially MQINQ on the IPPROCS value. If it is greater than a certain threshold, the app should MQSET the triggering to off (NOTRIGGER) and end without getting any messages. When one of the (possibly many) other running apps finds there are no more messages to get, and it has processed at least 1 message, it should MQINQ on the IPPROCS value. If it has fallen below a certain threshold (maybe even to 1), it should MQSET to turn triggering back on (TRIGGER).
_________________
Glenn |
|
| Back to top |
|
|
| kevinf2349 |
| Posted: Fri Mar 06, 2009 5:52 am Post subject: |
|
|
Grand Master
Joined: 28 Feb 2003
Posts: 1311
Location: USA
|
....and if you are talking about a z/OS system there is WLM that can be used or TCLASS under CICS or Tranaction Server or whatever else they decide to call CICS these days!  |
|
| Back to top |
|
|
| ucbus1 |
| Posted: Fri Mar 06, 2009 7:13 am Post subject: Manging thousands of messages |
|
|
Knight
Joined: 30 Jan 2002
Posts: 560
|
| Quote: |
| Let's say that thousands of messages suddenly and unexpectedly arrive in the queue. How do we (what tools can we use to) control (limit) the sudden and seemingly infinite increase in resource demand? This is the puzzling part. |
May not be relavant. But I understand the situation where we had thousands of messages in coming into our BPM application. The messages used to get loaded in bunches from a batch legacy job ( no one wanted touch it). The way I managed it is to have a java app polling constantly at a cetain rate ( messages/sec). Whenever we see a surge in the queue depth, the properties used to be changed on FLY, so that the app would poll at higher rate of messages/sec.
I understand here we are talking about multiple copies of the app, I find similar construct could be used I guess |
|
| Back to top |
|
|
| Challenger |
| Posted: Sun Mar 08, 2009 5:50 pm Post subject: |
|
|
Centurion
Joined: 31 Mar 2008
Posts: 115
|
kevin2349: from your post and others I've read here, it seems that managing demand for resources due to unanticipated workloads is something that z/OS does well and out-of-the-box.
How would you handle unanticipated workloads in of UNIX? In Windows?
gbaddeley: having the application(s) monitor IPPROCS looks like a very cool solution. Programmers here are fond of saying that performance (SLAs) is mostly a techie problem; and management seems to agree - unless we can prove the code is needlessly wasting resources. We have a variety of OEM software, too; and modifying this code is not encouraged.
ucbus1: I will find a Java person to try to explain this to me.
KxK3: ...trigger type of EVERY would cause the queue manager to generate a trigger message... is inefficient because the queue manager would need to do a lot more processing...
I understand this as a cost of TRIGTYPE(EVERY). The benefit might be meeting SLAs that require concurrent applications. Can you suggest how I could quantify the cost (overhead or whatever you call it)? |
|
| Back to top |
|
|
| Challenger |
| Posted: Fri Apr 03, 2009 4:10 pm Post subject: |
|
|
Centurion
Joined: 31 Mar 2008
Posts: 115
|
In summary, EVERY is a triggering option to be implemented with careful planning, caution, or extreme caution.
Thanks to those that contributed to this discussion. Thanks, as well, to those that viewed the posts either out of curiousity or to learn from others. |
|
| Back to top |
|
|
|
|
