| Author |
Message
|
| queuetip |
 Posted: Fri Jul 28, 2006 11:34 am Post subject: Trigger Application Design Challenge (with CICS and Tuxedo) |
 |
|
Acolyte
Joined: 03 Feb 2005
Posts: 67
|
Generally, when you plan your triggering strategy for a given application you must decide upon a combination of:
1) queue properties
2) queue manager properties
3) trigger monitor settings
4) application GMO settings and design
5) message properties and design
6) operating system configuration and environment
7) ...and third party TP monitor software (CICS, Tuxedo, etc) configurations, limitations and the size of the application transaction running.
PLATFORM ENVIRONMENTS: I am trying to plan for triggering for our system that can run across 2 platforms (MVS/CICS and HPUX/Tuxedo). Using MQSeries V5.3 now.
MESSAGE DESIGN: I am using the message sequencing feature to logically tie together multiple real-time generated messages so they are processed in order.
APPLICATION: The application getting and processing the messages waits for a maximum of 15 seconds after the last message on the queue is processed...just in case anymore messages arrive on the queue. A daemon is not an option.
TRIGGERING THE APPLICATION: Ideally, I would like to trigger the transaction once the entire logical sequence of messages arrive AND only if there is not a transaction already processing the queue. Contractually, I can not have messages sitting on the queue, waiting for a triggering condition to be met so they could be processed. Also, I am not allowed to force the queue manager's triggerinterval property to a value. I am looking for a triggering solution that would not require any MQSeries global processing configuration changes (like queue manager settings, etc).
So, triggering FIRST with a triggerinterval is not an option. I think triggering EVERY would be great if only the queue manager would check to make sure sure the queue isn't open - like it does for FIRST and DEPTH. Unfortunately, it does not do this. 
However, I was wondering if CICS and Tuxedo could save the day by letting me configure the transaction so that a maximum of 1 occurence of it could be running at any given time. In that case, the TP monitor (CICS or Tuxedo) would ignore the transaction - as opposed to queuing it up.
Does anyone know if this can be done? Any other ideas?
Thanks!! 
Mike |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Fri Jul 28, 2006 2:05 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
If you configure Trigger On First, and two "sets" of transactions come in, then your program will only get triggered ONCE.
So how could that not meet your requirement of "Ideally, I would like to trigger the transaction once the entire logical sequence of messages arrive AND only if there is not a transaction already processing the queue. "?
The first message comes in, your program gets triggered. Your GETs have specified that you want to wait for the complete sequence. Your first GET doesn't return until all the logical messages are available. You process the first sequence of messages, and then check to see if there are any more sequences. If the queue is empty, then you end. If the queue is not empty, you keep processing until the queue IS empty, and then you end.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| queuetip |
| Posted: Fri Jul 28, 2006 2:30 pm Post subject: |
|
|
Acolyte
Joined: 03 Feb 2005
Posts: 67
|
| Quote: |
| So how could that not meet your requirement of "Ideally, I would like to trigger the transaction once the entire logical sequence of messages arrive AND only if there is not a transaction already processing the queue. "? |
Here's one scenario...
The GET timeout occurs before all messages in the group arrive.
Example:
* Message 1 of 10 (logically related sequence) comes in and program is triggered.
* Message 2,3 arrives.
* Program reads queue but must wait for all 10 to arrive.
* Message 4,5,6,7 arrives.
* Wait interval expires without reading a message because it's not all there.
* Message 8 arrives.
* Program starts closing down.
* Message 9.
* Program ends.
* Queue manager recognizes queue is no longer open and doesn't trigger because it is not "FIRST" (there are 9 other messages on file).
* Message 10 arrives - and all future messages will pile up like crazy!
The fact it's waiting doesn't override the GMO timeout, correct? |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Sat Jul 29, 2006 6:34 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| queuetip wrote: |
| Quote: |
| So how could that not meet your requirement of "Ideally, I would like to trigger the transaction once the entire logical sequence of messages arrive AND only if there is not a transaction already processing the queue. "? |
Here's one scenario...
The GET timeout occurs before all messages in the group arrive. |
See Condition 12.
http://publib.boulder.ibm.com/infocenter/wmqv6/v6r0/index.jsp?topic=/com.ibm.mq.csqzal.doc/trgrule.htm
| queuetip wrote: |
| The fact it's waiting doesn't override the GMO timeout, correct? |
Correct. So make sure you understand how long your typical message traffic will require you to wait.
You can do this with triggering.
You *may* have to change the Trigger Interval on the queue to a lower number, something perhaps in the 10 minute range, depending on your SLA.
But condition 12 should cover most of your cases.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| queuetip |
| Posted: Mon Jul 31, 2006 7:43 am Post subject: |
|
|
Acolyte
Joined: 03 Feb 2005
Posts: 67
|
First of all - I want to thank you for working this through with me. The trigger stuff is great - but because it is so complex, it takes a lot of thought to make sure there aren't situations that fall through the cracks that would result in a failure to trigger the application. I want to run a few more by you...
Regarding:
I did look at that. If the application abends for an unrelated to MQSeries processing...the program will never issue a CLOSE and thus, I think we need more than condition 12 to avoid a backup of messages. Do you agree?
Again, my main goal is to trigger the application for every message, as long as a program does not have the application queue open. I want to avoid backing up messages and I don't want to change any of the queue manager properties.
Thanks,
Mike
 |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Jul 31, 2006 7:50 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| queuetip wrote: |
| If the application abends for an unrelated to MQSeries processing...the program will never issue a CLOSE and thus, I think we need more than condition 12 to avoid a backup of messages. Do you agree? |
Yes. That's why I said that condition 12 should cover MOST of your cases, and why I mentioned the Trigger Interval on the queue EDIT: manager.
| queuetip wrote: |
| Again, my main goal is to trigger the application for every message, as long as a program does not have the application queue open. I want to avoid backing up messages and I don't want to change any of the queue manager properties. |
I don't understand that last requirement at all. Can you explain better?
EDIT. Apparently I'm being a bit stupid here.  The Trigger Interval is a qmgr property, and not a queue property.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Jul 31, 2006 7:56 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
Hey Vitor - I saw that...  You weren't wrong. I wasn't agreeing with his statement about program abends, just that to be 100% sure, he needs more than that.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Jul 31, 2006 8:05 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| jefflowrey wrote: |
| Hey Vitor - I saw that... You weren't wrong. I wasn't agreeing with his statement about program abends, just that to be 100% sure, he needs more than that. |
Busted.....
It was my opinion that my post, combined with yours, would have added to rather than diminished the confusion. So I yielded the floor to the Grand Poobah (All Hail! All Hail!!)
In summary, I expressed the opinion that note b of Condition 12 indicated there would be a trigger message in the case of a non-MQ abend but that there would (as indicated) be a possible poison message situation and that needed to be handled within the app.
But in a much more long winded way 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| queuetip |
| Posted: Mon Jul 31, 2006 8:43 am Post subject: |
|
|
Acolyte
Joined: 03 Feb 2005
Posts: 67
|
| queuetip wrote: |
| If the application abends for an unrelated to MQSeries processing...the program will never issue a CLOSE and thus, I think we need more than condition 12 to avoid a backup of messages. Do you agree? |
Yes. That's why I said that condition 12 should cover MOST of your cases, and why I mentioned the Trigger Interval on the queue EDIT: manager.
| queuetip wrote: |
| Again, my main goal is to trigger the application for every message, as long as a program does not have the application queue open. I want to avoid backing up messages and I don't want to change any of the queue manager properties. |
| jefflowrey wrote: |
| I don't understand that last requirement at all. Can you explain better? |
Sure! Unfortunately, more than just our application uses queues. Thus, I mentioned that I do not want my design to including any specific values that can globally affect other MQSeries processing...one of our clients may have a desired value for it anyway. Even if I used a queue manager triggerinterval I can find scenraios that would have it fall through the cracks - but that's out of scope anyway.
About the app requirements. When a message arrives on a queue, I want to trigger the application (8MB load module) to process it pretty much ASAP. After it processes the last message on the queue, the app basically will "loop" (actually will wait for a configurable timeout period) just in case any more messages come in. So if a message arrives and the app is running it will be processed. I would like to make sure the application doesn't get kicked off more than it has to because they will take up a ton of space. Also, I don't want a scenario where there is no message backup because it is an enterprise solution.
If EVERY could check to see if the app is already processing the queue like FIRST and DEPTH do, I'd be set. This is why I originally was wondering if the TP monitor (e.g. Tuxedo, CICS) could work in tandem to prevent more than one of my particular txn to be running at once - since MQSeries may not be capable. Makes sense? Any ideas?
Thanks,
Mike |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Jul 31, 2006 5:23 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
As you did specify CICS and triggering let me fill you in on what we do:
Sorry if it doesn't all make sense as I am not a zos person.
- TClass the application/transaction. This way only a limited number of processes can run in parallel.
- write the application as if trigger first. I.e. do the get in a loop on the queue and process until empty. You may want to add a wait time on the get to allow for all messages to arrive (be committed) through the channel before exiting with a 2033 on the trigger.
- Make sure you have handled the poison message scenario
- Set your trigger to every, priority 0 etc...
Enjoy
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| queuetip |
| Posted: Tue Aug 01, 2006 8:16 am Post subject: |
|
|
Acolyte
Joined: 03 Feb 2005
Posts: 67
|
| Quote: |
| TClass the application/transaction. This way only a limited number of processes can run in parallel. |
This makes sense for CICS. Do you know it would be a configurable TP monitor option to "queue up" or suppress the transactions above the TClass?
I also mentioned Tuxedo. Can this be done with the Tuxedo TP monitor also - if so, how?
Knowing this could solve my dilema. Thanks!
| Quote: |
| write the application as if trigger first. I.e. do the get in a loop on the queue and process until empty. You may want to add a wait time on the get to allow for all messages to arrive (be committed) through the channel before exiting with a 2033 on the trigger. |
Do you mean use EVERY (instead of FIRST) since the TP monitor will meter the parallel requests? The problem with FIRST is when the application abends and does not close...you can get a message backup in certain scenarios. From what I read, I don't see an enterprise solution for ever using FIRST. Maybe I missed something?
Actually, my understanding is the loop is poor design because it racks up MIPs. Conversly, the GET issued with a WaitInterval does not poll...it uses "callback" operating system features. But I think this accomplishes the same thing.
| Quote: |
| Make sure you have handled the poison message scenario. |
I think I am OK here. I don't use the backout count / queue approach...I use another local queue for restarting. I copy the message (truncating if I have to) to the restart queue and take a commit to establish the snapshot that I wish to create if I abend. Then I destructively pull it off the restart queue and process it - if everything works I commit and it's gone.
Don't know where the MQSeries improvements suggestion box is...
I wish MQSeries would have a "FollowUpTrigger" property on the queue that would indicate to the queue manager to follow up a trigger once an application is done using that queue (whether it issues a close or abend). This property would apply for any trigger type - EVERY, FIRST or DEPTH. It would simplify understanding the triggering conditions since it is more explicit to the user and gives them more power.
Another queue property that would be handy would for triggering be a "MaximumOpenHandles" for the queue manager to check before issuing another trigger of the application. Very handy for load balancing.
Thanks,
Mike |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Aug 01, 2006 2:14 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
No I meant "Write the application to work as if you would have specified trigger First" i.e. whenever triggered the process servicing the queue needs to read the queue and process the message until empty.
The fact that you run with a Tclassed transaction and trigger EVERY allows you to maximize throughput while throttling/limiting resource usage.
As for the rest I told you I am not a zOS person. Talk to your local zOS representative. Don't know anything about Tuxedo on a mainframe...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Aug 01, 2006 6:43 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| queuetip wrote: |
Regarding:
I did look at that. If the application abends for an unrelated to MQSeries processing...the program will never issue a CLOSE and thus, I think we need more than condition 12 to avoid a backup of messages. Do you agree?
|
I don't agree. This is the quote from the manual:
| Quote: |
If the program serving the application queue backs out its get request (or if the program abends) before it closes the queue, the same happens.
|
The SAME being a trigger message is generated. The program abending after it has opened the queue is nothing to worry about, because a trigger message will be genereated when the q is closed, explicitly (MQCLOSE), implicitly (MQDISC) or incidentaly (program abends).
BUT, here is where you will be vulnerable. A message lands, your app is triggered ***and abends before it ever MQOPENs the queue***. Maybe it couldn't open a DB that it needs access to prior to the MQ calls. In this case, triggering worked 100%. The app started. But it never got to the point of opening the queue, so it can't close it in any fashion thus insuring a retrigger (for Trigger First).
Your only way out of this tricky situation is that another message lands on the queue AND it happens after Trigger Interval millseconds. If no other messages land, that 1st message is now stranded, even though trigger interval has passed. If a million more messages land, but Trigger Interval has not passed (999,999,999 is a long time!), they are all stranded.
You *need* to set Trigger Interval to lessen the chances of problems. We have it set to 60000. But its not 100% as I mentioned above. To cover those even rarer situations, you need a smart MQ monitor that alerts only after the q is > 0 for x seconds with no messages being consumed for x seconds. We use QPASA for this.
Trigger Every is not the solution for you, and in fact has a greater chance of stranding messages, because Trigger Every is not Trigger Any
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Aug 01, 2006 6:45 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| queuetip wrote: |
Don't know where the MQSeries improvements suggestion box is...
|
We need Trigger Group. A trigger message is only generated when a comple MQ Group of messages has arrived. Now that is a useful trigger!!!
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
|
|
