| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| MQ v6.0.2 and MQ Triggering |
« View previous topic :: View next topic » |
| Author |
Message
|
| AviD |
 Posted: Thu Apr 03, 2008 1:17 pm Post subject: MQ v6.0.2 and MQ Triggering |
 |
|
Acolyte
Joined: 06 Apr 2004
Posts: 62
|
I'm trying to locate an article written by an IBM developer where the developer states the caveats to the usage of Triggering in MQ 5.3.
It basically stated you really shouldn't use triggering unless you absolutely MUST, and especially is the case with trigger EVERY...and you should NEVER use DEPTH triggering.
Of course this begs the question, why do we have these options to choose from then?
I'm not sure if that was or still is the common viewpoint when it comes to MQ Triggering.
Nevertheless, from my own experience with 5.3 and somewhat with 6.0+, I've seen variable results with MQ Triggering based on current system conditions. My experiences with triggering are isolated to Windows environments, in that we have not utilized triggering in our AIX, Mainframe, or iSeries environments.
In a nutshell, we've seen variation in triggers being "lost" due to system issues (i.e. CPU being maxed/overloaded, etc), and although there is a trigger interval that can be leveraged to "re-trigger", it must be a consideration when implementing the respective MQ application that is going to process messages off the queue.
Anyway, I am wondering if the general census on the utilization of MQ Triggering has changed between 5.3 and 6.0+. Good, bad, ugly?
Does anyone here actively use it on iSeries, AS/400 and have any good or bad feedback on it's usage and reliability?
For the MQ application savvy, we have a group on the iSeries that wants to kick off a process with trigger FIRST, but also wants to let that application "sleep" during an "off-processing" timeframe such that the message will remain on the queue.
Obviously this introduces an issue when implementing a TriggerInterval setting as multiple instances of their application may kick off and just "hang" in a "sleep" state waiting to process messages.
This could all be managed, and coded to handle the messages appropriately, but it also increases the complexity of the solution (perhaps even mildly).
Generally speaking, we (MQ Admins) recommend developers utilize a "listener" approach in always polling the queue for messages to be actively processed. We persist messages where needed to address any application or system failures. This has proven to be an effective recommendation and implementation...and quite commonplace across all technologies (i.e. open source solutions, Java MDB oriented solutions, etc).
So my real questions are:
1) Does triggering ever have a place (over a "listener" or daemon style solution)?
2) What caveats are there (if any) to triggering in it's current form in MQ 6.0+? Any specific to iSeries?
Thoughts? Suggestions? |
|
| Back to top |
|
 |
| Vitor |
| Posted: Thu Apr 03, 2008 2:00 pm Post subject: Re: MQ v6.0.2 and MQ Triggering |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| AviD wrote: |
| It basically stated you really shouldn't use triggering unless you absolutely MUST, and especially is the case with trigger EVERY...and you should NEVER use DEPTH triggering. |
I don't recall this article. FIRST triggering is widely and rightly used in a number of circumstances. EVERY & DEPTH must be used with caution and are less often appropriate, but EVERY is quite common on z/OS where CICS offers limiting options not available elsewhere.
| AviD wrote: |
| Of course this begs the question, why do we have these options to choose from then? |
For the same reason WMQ allows you to set MsgId within an application. In 99.9% of cases it's either stupid or suicidal, but the option's built into the software for that 0.01% where you have to. Likewise using EVERY is usually stupid or sucidial, but is sometimes the valid solution to a problem.
IMHO it's good that the software provides these options; allows informed choices.
| AviD wrote: |
| I'm not sure if that was or still is the common viewpoint when it comes to MQ Triggering. |
I can't speak for others, who will undoubtably speak for themselves
| AviD wrote: |
| In a nutshell, we've seen variation in triggers being "lost" due to system issues (i.e. CPU being maxed/overloaded, etc), and although there is a trigger interval that can be leveraged to "re-trigger", it must be a consideration when implementing the respective MQ application that is going to process messages off the queue. |
I'll avoid my usual rant about Windoze.
Trigger messages should not be "lost" any more than any other message should be. My experience of triggering is that I use & have used trigger FIRST extensively without issues on all the platforms you mention (including Windows), except unfortunately iSeries or AS/400!!
| AviD wrote: |
| Anyway, I am wondering if the general census on the utilization of MQ Triggering has changed between 5.3 and 6.0+. Good, bad, ugly? |
I think it's fine on both.
| AviD wrote: |
| For the MQ application savvy, we have a group on the iSeries that wants to kick off a process with trigger FIRST, but also wants to let that application "sleep" during an "off-processing" timeframe such that the message will remain on the queue. |
Sounds fine to me.
| AviD wrote: |
| Obviously this introduces an issue when implementing a TriggerInterval setting as multiple instances of their application may kick off and just "hang" in a "sleep" state waiting to process messages. |
You shouldn't need to be fiddling with that. But I don't immediately see the problem with hanging transactions unless you have managed to generate a stupidly large number of them or have message affinity issues.
Which is not to say there isn't a problem, just that I'm not seeing it.
| AviD wrote: |
| Generally speaking, we (MQ Admins) recommend developers utilize a "listener" approach in always polling the queue for messages to be actively processed. We persist messages where needed to address any application or system failures. This has proven to be an effective recommendation and implementation...and quite commonplace across all technologies (i.e. open source solutions, Java MDB oriented solutions, etc). |
There's nothing wrong with a long-running transaction that constantly polls the queue, as long as it's properly administered. After all, that's what the trigger monitor itself is.
| AviD wrote: |
| 1) Does triggering ever have a place (over a "listener" or daemon style solution)? |
Yes
| AviD wrote: |
| 2) What caveats are there (if any) to triggering in it's current form in MQ 6.0+? Any specific to iSeries? |
Don't use anything outside FIRST without thinking about what you're doing, be sure your triggered application has a sufficient wait time on it's get and that it reads the queue until it's empty. See above re: my lack of iSeries expereince
| AviD wrote: |
| Thoughts? Suggestions? |
Figure out why you keep losing trigger messages. It shouldn't happen.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| AviD |
| Posted: Fri Apr 04, 2008 7:01 am Post subject: |
|
|
Acolyte
Joined: 06 Apr 2004
Posts: 62
|
Vitor, thanks so much for the detailed response!
| Quote: |
| Figure out why you keep losing trigger messages. It shouldn't happen. |
Ah the plague of Windows...many an IBM PMR were opened in the past to "find" what happened...nothing conclusive came about. The only common factor was the CPU utilization capping out. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Apr 04, 2008 7:24 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| AviD wrote: |
| Ah the plague of Windows... |
Look up my usual rant on Windoze. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| tleichen |
| Posted: Fri Apr 04, 2008 10:07 am Post subject: Re: MQ v6.0.2 and MQ Triggering |
|
|
Yatiri
Joined: 11 Apr 2005
Posts: 663
Location: Center of the USA
|
| AviD wrote: |
| ...Generally speaking, we (MQ Admins) recommend developers utilize a "listener" approach in always polling the queue for messages to be actively processed. We persist messages where needed to address any application or system failures. This has proven to be an effective recommendation and implementation...and quite commonplace across all technologies (i.e. open source solutions, Java MDB oriented solutions, etc)... |
Who are you referring to when you say "we MQ Admins"? I agree with Vitor that there is no problem with triggering and, in fact, there are several advantages that have not been aired here.
I AM an MQ Admin and I do not shy away from triggering, if it seems like a good fit. Do not condemn a feature simply because you are not versed in it. I suggest you read the appropriate parts of the Intercommunication Manual and the Application Programming Guide on triggering to gain a greater understanding of it. 
_________________
IBM Certified MQSeries Specialist
IBM Certified MQSeries Developer |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Fri Apr 04, 2008 10:17 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
AviD,
don't forget that regular SNDR channels rely on triggering. If IBM thinks its good enough for the core product to rely on.....
All triggering problems I've encountered were due to a lack of understanding on how triggering works. Once all the triggering related options are set up correctly, its very reliable.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| AviD |
| Posted: Fri Apr 04, 2008 11:27 am Post subject: |
|
|
Acolyte
Joined: 06 Apr 2004
Posts: 62
|
All good and valid points.
tleichen, I meant the MQAdmins here at my company generally recommend using a listener approach given the issues we've seen with triggering in the past and were relative to trigger EVERY, not FIRST, and also given the application development medium (software and OS).
From an application perspective we don't use trigger FIRST here at all currently, so given this feedback and further reading of the documentation, we are going to move forward with it for this iSeries project based on the MQ guidelines and documented consideration points.
Thanks for the feedback guys!  |
|
| Back to top |
|
|
| Bni |
| Posted: Wed Apr 09, 2008 2:08 pm Post subject: |
|
|
Apprentice
Joined: 09 Aug 2006
Posts: 29
|
| Since you experts are disucssing triggers and concerns about trigger "every", I have the exact same question. We have a project going on which requires the destination application(SAP) be shutdown without shutting down MQ, that means transactions will continue to be put in the queue. So what I need to do is to turn off the trigger, after the SAP is back up, I will reenable the trigger. Now currently our trigger is set as trigger "first" and trigger depth is set to "1"(we have never had problems in our production environment by the way), but I have to change it to trigger "every" after I enable the trigger based on my experience in the testing environment( I am newbie; I found out that if I did not change to trigger every, only the first message will get processed and nothing will happen late on); My question is that is this the correct way for me to reenable the trigger to process those transactions in the queue? I am scared to hear that messages may get "lost" by using trigger "every". Please help me out here. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Apr 09, 2008 2:52 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Triggers work. You might ponder then, why there are pages in the App Programming Guide and Reference about triggers, and when they go bad.
As noted in this post, TRIGTYPE(FIRST) works to launch an instance of the message-consuming application when queue depth goes from 0 to 1. If you turn off triggering, then turn it back on while there are messages in the queue, the queue manager should fire the trigger - even though depth did not go from 0 to 1 in the usual sense of a message arriving on an empty queue.
There is a queue manager attribute called TRIGGERINTERVAL that causes to queue manager to handle odd instances of triggers failing to fire.
TRIGTYPE(DEPTH) may make sense where it takes 5 messages to comprise a business transaction. The odd, yet documented, behavior is that triggering is turned off after the trigger fires. It is up to the application to turn it back on.
TRIGTYPE(EVERY) has its advocates and detractors. Potentially, it could launch many instances of the message-consuming application in short order. (short order is a technical term for darn quickly.) This will stress most distributed systems; but, as noted here and elsewhere, mainframes usually have lots of resources to spare. EVERY can be used to meet SLAs not possible on smaller systems.
As always, the business requirement will determine which trigger type you will use.
From the APR, source of all knowledge:
TriggerInterval (MQLONG)
This is a time interval (in milliseconds) used to restrict the number of trigger messages. This is relevant only when the TriggerType is MQTT_FIRST. In this case trigger messages are usually generated only when a suitable message arrives on the queue, and the queue was previously empty. Under certain circumstances, however, an additional trigger message can be generated with MQTT_FIRST triggering even if the queue was not empty. These additional trigger messages are not generated more often than every TriggerInterval milliseconds.
For more information on triggering, see the WebSphere MQ Application Programming Guide.
The value is not less than 0 and not greater than 999 999 999. The default value is 999 999 999.
To determine the value of this attribute, use the MQIA_TRIGGER_INTERVAL selector with the MQINQ call.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Apr 09, 2008 2:52 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| Bni wrote: |
| Since you experts are disucssing triggers and concerns about trigger "every", I have the exact same question. We have a project going on which requires the destination application(SAP) be shutdown without shutting down MQ, that means transactions will continue to be put in the queue. So what I need to do is to turn off the trigger, after the SAP is back up, I will reenable the trigger. |
Right this is what you should do.
| Bni wrote: |
| Now currently our trigger is set as trigger "first" and trigger depth is set to "1"(we have never had problems in our production environment by the way), but I have to change it to trigger "every" after I enable the trigger based on my experience in the testing environment( I am newbie; I found out that if I did not change to trigger every, only the first message will get processed and nothing will happen late on); My question is that is this the correct way for me to reenable the trigger to process those transactions in the queue? I am scared to hear that messages may get "lost" by using trigger "every". Please help me out here. |
Get a supersized trout and hit your developers with it. A well behaved trigger application ALWAYS reads the queue until empty. So it does not matter whether you trigger every or trigger 1st, all waiting messages will get processed.
 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Apr 09, 2008 3:10 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Bni wrote: |
| I found out that if I did not change to trigger every, only the first message will get processed and nothing will happen late on |
What you've found is an application error; best practice is for any triggered application to read the queue until empty. The wait interval should be long enough so that, if a message arrives within TRIGGERINTERVAL, the application will still pick it up. If there is any depth to the queue and this isn't happening then it's a problem with the application.
Now a solution to your problem is to use EVERY, and it's unlikely that messages will get lost unless the platform crashes under the load & you experience other problems (e.g. database errors due to resource shortage). You'll certainly get 1 trigger for each message as soon as the trigger's reenabled, what the applications do with this information is their business.
It's not a solution I'd recomend, for all the reasons in this post, unless you've got bags of resources or some means of throtting the applications, and a burning business requirement to clear the queue out asap after triggering is reenabled.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Apr 09, 2008 3:17 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Again:
The usage of trigger every is mostly seen on mainframe and CICS environments or their equivalents where the usage of the triggered resource is throttled by means outside of MQ.
As the resource usage is being capped, no damage is being done.
In a windows or Unix environment you could bring a box to an immediate halt or even bring it down by exhausting the available resources on a trigger every...
Enjoy
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Apr 09, 2008 3:21 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
A big mainframe 'high five'!
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Apr 09, 2008 10:05 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| Vitor wrote: |
Now a solution to your problem is to use EVERY, and it's unlikely that messages will get lost unless the platform crashes under the load & you experience other problems (e.g. database errors due to resource shortage). You'll certainly get 1 trigger for each message as soon as the trigger's reenabled, what the applications do with this information is their business.
|
If you turn on triggering of a queue that is Trigger Every, and there are 1 or more messages on the queue, and all other trigger conditions are satasfied, you will only got 1 trigger message, regardless of how many messages are on the queue. This is the most common reason why Trigger Every can leave a rolling backlog of messages. They're not lost. The app is coded incorrectly and doesn't read until 2033. Say there are 10 messages on the q and you toggle the trigger. The app gets triggered, incorrectly reads only one message, and leaves messages 2 thru 10. Messages # 11 arrives and 1 trigger fires. The app reads message # 2 and ends, leaving 3 thru 11, etc, etc.
http://publib.boulder.ibm.com/infocenter/wmqv6/v6r0/topic/com.ibm.mq.csqzal.doc/fg13860_.htm
(See Note 1 at the bottom)
Triggered apps, First, Depth and Every, must be coded to read until 2033.
Triggered apps, First, Depth and Every, must be coded to gracefully handle a 2033, even if its their first MQGET.
If you ignore these 2 rules you will have problems. They are not triggering problems, they are application design problems.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Apr 10, 2008 12:39 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| PeterPotkay wrote: |
http://publib.boulder.ibm.com/infocenter/wmqv6/v6r0/topic/com.ibm.mq.csqzal.doc/fg13860_.htm
(See Note 1 at the bottom)
|
Again I learn. 
| PeterPotkay wrote: |
If you ignore these 2 rules you will have problems. They are not triggering problems, they are application design problems. |
Probably the most important point.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
|
  |
Goto page 1, 2 Next |
Page 1 of 2 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
