| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
| Are trigger messages part of a Unit of Work |
« View previous topic :: View next topic » |
| Author |
Message
|
| PeterPotkay |
 Posted: Mon Apr 16, 2012 5:07 pm Post subject: Are trigger messages part of a Unit of Work |
 |
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
The generation of a trigger message by the Queue Manager, and the consumption of the trigger message by the Trigger Monitor are two separate and distinct events, sometimes separated by a significant amount of time.
We have a thread going on the list serve talking about if a trigger message generated by an application message that was put under syncpoint is itself under syncpoint. My training years ago said yes. And that the trigger message would be committed, and thus finally available for the trigger monitor to read, only once the app messages that satisfied the trigger conditions was finally committed. Or rolled back (yes, the trigger message gets committed even if the app message gets rolled back).
Some agreed with me that the trigger message is put in a unit of work if the app message was in a unit of work, others disagree. So what say you?
The manuals seem to agree with me:
http://publib.boulder.ibm.com/infocenter/wmqv7/v7r0/topic/com.ibm.mq.csqzal.doc/fg13860_.htm
| Quote: |
a. For non-shared local queues, the queue manager counts both committed and uncommitted messages when it assesses whether the conditions for a trigger event exist. Consequently an application might be started when there are no messages for it to retrieve because the messages on the queue have not been committed. In this situation, consider using the wait option with a suitable WaitInterval, so that the application waits for its messages to arrive.
b. For local shared queues the queue manager counts committed messages only. |
I think point a. is trying to say the message or messages that finally satisfied the trigger conditions may have been rolled back, at which point the trigger message is committed, and the triggered app finds an empty queue. Better advice here would be for the app to be coded to deal with a 2033 gracefully. Doing a Get With Wait is generally good advice for triggered apps, but not for this reason.
http://publib.boulder.ibm.com/infocenter/wmqv7/v7r0/topic/com.ibm.mq.csqzal.doc/fg13930_.htm
| Quote: |
Trigger messages created because of trigger events that are part of a unit of work are put on the initiation queue as part of the same unit of work. Trigger monitors cannot retrieve these trigger messages until the unit of work completes. This applies whether the unit of work is committed or backed out.
|
Further down that same page:
| Quote: |
The queue manager counts both committed and uncommitted messages when it assesses whether the conditions for a trigger event exist.
With triggering of type FIRST or DEPTH, trigger messages are made available even if the unit of work is backed out so that a trigger message is always available when the required conditions are met. For example, consider a put request within a unit of work for a queue that is triggered with trigger type FIRST. This causes the queue manager to create a trigger message. If another put request occurs, from another unit of work, this does not cause another trigger event because the number of messages on the application queue has now changed from one to two, which does not satisfy the conditions for a trigger event. Now if the first unit of work is backed out, but the second is committed, a trigger message is still created.
However, this means that trigger messages are sometimes created when the conditions for a trigger event are not satisfied. Applications that use triggering must always be prepared to handle this situation. It is recommended that you use the wait option with the MQGET call, setting the WaitInterval to a suitable value. |
While that last sentence is good advice, I think they missed what should have been written here: Trigger Apps should be coded to gracefully handle a No Message Found (2033) Reason Code. The trigger message may have been committed as a result of the app message (that satisfied the trigger conditions) being rolled back. Why wait? There may never be another message. But you have to deal with that 2033 gracefully.
| Quote: |
| Now if the first unit of work is backed out, but the second is committed, a trigger message is still created. |
This needs to be more explicit. Exactly what caused the trigger message to be committed? I think the first message rolling back, since its what caused the trigger message to be created. The second message being committed is just a coincidence.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
 |
| Mr Butcher |
| Posted: Tue Apr 17, 2012 1:36 am Post subject: |
|
|
Padawan
Joined: 23 May 2005
Posts: 1716
|
IIRC the creation of a trigger message is started by the queuemanager when the message, that caused the trigger, is still in the commit phase (or before?!?). It is done asynchronous.
Thats why a trigger message can be availabe to the trigger monitor which then starts the application which itself issues the MQGET, but the application message may still not be committed. Thats why a small wait may be more effective (faster and less ressource donsuming) rather then another trigger message creation (e.g. due to the close of an non empty queue) and a new application start.
So a possible backout of the application message is not the only cause for an application hitting an (unexpected) 2033, theres a "normal" chance for the trigger message overtaking the application message.
however, i agree with you, a triggered (or better - every) application should handle a 2033 returncode properly even if no message was ever read.
Dont know how that deals with UOW ..... but for performance reasons (just think of trigger every) i think its okay that the UOW of the trigger message and the UOW of the application message do not rely on each other. Its not a big deal to take care of this in the application, but its a big mess if you dont do so 
_________________
Regards, Butcher |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Apr 17, 2012 3:20 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Mr Butcher, do you find any references in the manuals that state that the trigger message is produced outside of a UOW, if the app message that satisfied the trigger conditions is in a UOW?
All I can find is a statement that says:
| Quote: |
Trigger messages created because of trigger events that are part of a unit of work are put on the initiation queue as part of the same unit of work. Trigger monitors cannot retrieve these trigger messages until the unit of work completes. This applies whether the unit of work is committed or backed out.
|
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Mr Butcher |
| Posted: Tue Apr 17, 2012 3:53 am Post subject: |
|
|
Padawan
Joined: 23 May 2005
Posts: 1716
|
I trust your manual digging. If you did not find it so far then its not in. I will not search again... (too much to dig).
For me its not important if its the same unit of work or not. it does not make any difference to the application coding, as you still may be triggered finding an empty queue, no matter if a backout (of the app and the trigger msg) has been before or not because of the possible takeover of trigger and application message.
but maybe i missed the real point of the discussion.
_________________
Regards, Butcher |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Apr 17, 2012 8:15 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| Mr Butcher wrote: |
| but maybe i missed the real point of the discussion. |
Originally the post on the list server started taling about a CICS app that was getting triggered thousands of time, only to keep getting 2033s. As we discussed potential causes, the discussion began to focus on whether an uncommitted message in the app queue could cause the app to be triggered.
Some of the people said no. I think the manual says no based on what I what I have posted above.
Some of the people said they've seen it happen, and that yes, the uncommitted message can and does cause a trigger message to be created AND COMMITTED to the Init queue. Which I imagine could cause the loop:
Uncommitted app message lands on app queue
Start loop
trigger message generated and committed immediatly(I'm not sure this is possible according to manuals)
app gets triggered'
app gets 2033
app closes queue
trigger condition for On First queue satisfied
app message still not comitted
end loop
The above scenaio is one of the reason I think uncomitted or non backed out app messages do not commit a trigger message to the init queue.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Apr 17, 2012 7:49 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Based on what you read the manual says yes in following condition:
- The message is posted in a unit of work
- The trigger message is posted in a unit of work
- The message is committed or rolled back
- The trigger message is committed
The reasonning behind this is quite sound.
The trigger message could have been caused by the queue depth going from 0 to 1.
Meanwhile another message hits the queue, as the trigger message is not yet committed and the queue depth went from 1 to 2 no additional trigger occurs (@ first)
Let's say the first message gets rolled back. If you rollback the trigger message the second message will not get triggered until trigger conditions are satisfied again (usually after another message hit the queue and trigint has been exceeded, or the queue is unserviced etc...)
Not quite satisfactory, so MQ commits the trigger message anyway.
Worst case scenario, you just committed a trigger message with no corresponding application message (it got rolled back).
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| gouda |
| Posted: Wed Apr 18, 2012 2:15 am Post subject: |
|
|
Apprentice
Joined: 20 May 2001
Posts: 32
Location: Germany, Nuremburg
|
Recently we had an investigation of SMF type 116 queue records.
A mainframe batch application puts messages in some queues in one UOW and one of them is triggered by CICS (type every).
SupportPac MP1B does not show the initqueue in the UOW, it does not show the initqueue anyway for put in the SMF records.
It just shows the task of CICS that consumes them.
_________________
Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots.
So far, the Universe is winning.
https://en.wikiquote.org/wiki/Rick_Cook |
|
| Back to top |
|
|
| Mr Butcher |
| Posted: Wed Apr 18, 2012 3:00 am Post subject: |
|
|
Padawan
Joined: 23 May 2005
Posts: 1716
|
PeterPotkay said:
| Quote: |
| As we discussed potential causes, the discussion began to focus on whether an uncommitted message in the app queue could cause the app to be triggered. |
IMHO yes, e.g. because of this reason described in the manuals:
| Quote: |
| With triggering of type DEPTH, a delay can occur even if all relevant messages are eventually committed. Suppose that the TriggerDepth queue attribute has the value 2. When two messages arrive on the queue, the second causes a trigger message to be created. However, if the second message is the first to be committed, it is at that time that the trigger message becomes available. The trigger monitor starts the server program, but the program can retrieve only the second message until the first one is committed. So the program might need to wait for the first message to be made available. |
so far i did not find any reference to the scenario i had in mind, that trigger message creation is asynchronous and that the trigger message may just overtake the commit of the application message. maybe i had something wrong in mind .... dont know.
_________________
Regards, Butcher |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Apr 18, 2012 3:18 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Somewhat More straightforward is Triggering involving
- one producing app Putting in one uow; and
- one consuming app getting in one Uow
More complicated is Multiple producers, multiple uows, some puts outside uows, multiple consumers, lost (forgotten) triggers.
Many pages of documentation on this curious subject. I recall there were Haiku's written about triggers. I'll search a bit when I'm next at my pc.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Apr 18, 2012 4:16 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| Mr Butcher wrote: |
PeterPotkay said:
| Quote: |
| As we discussed potential causes, the discussion began to focus on whether an uncommitted message in the app queue could cause the app to be triggered. |
IMHO yes, e.g. because of this reason described in the manuals:
| Quote: |
| With triggering of type DEPTH, a delay can occur even if all relevant messages are eventually committed. Suppose that the TriggerDepth queue attribute has the value 2. When two messages arrive on the queue, the second causes a trigger message to be created. However, if the second message is the first to be committed, it is at that time that the trigger message becomes available. The trigger monitor starts the server program, but the program can retrieve only the second message until the first one is committed. So the program might need to wait for the first message to be made available. |
so far i did not find any reference to the scenario i had in mind, that trigger message creation is asynchronous and that the trigger message may just overtake the commit of the application message. maybe i had something wrong in mind .... dont know. |
How long should it wait? It may be minutes / hours / days. The first message may never commit and instead roll back.
They make an exception for Shared Queues on z/OS, where they only count committed messages towards satisfying trigger conditions. I wonder what the logic was to do this for Shared Queues and not all other scenarios? Why would you want to count uncommitted messages - they're not 'real' yet. It maybe a long time before they are commited. They may never commit and instead rollback. Why do they count them? But on Shared queues they don't. 
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Mr Butcher |
| Posted: Wed Apr 18, 2012 4:45 am Post subject: |
|
|
Padawan
Joined: 23 May 2005
Posts: 1716
|
PeterPotkay wrote:
| Quote: |
| How long should it wait |
I always suggest to our application programmers to go for something between .1 and .5 seconds. Of course it depends on the kind of application and different factors. I always treated a short wait less expensive then a new trigger and application start.
However, that was not part of the question. The question was "whether an uncommitted message in the app queue could cause the app to be triggered." and the answer is yes (IMHO).
Nobody forces you to wait, there are quite a few mechanisms build into mq to make sure messages are processed, e.g. re-triggering under various conditions, and last but not least it is your monitoring to alert if messages are not processed.
For the shared queues ... maybe ressource usage was an issue to count comitted messages. Because - depending on the type of trigger - you may have multiple trigger messages created, depending on the number of members in the QSG that fullfill the trigger conditions. And all those may hit an empty queue (because of uncomitted message). I think thats more "expensive".
Or just think of a faster (or less busy) LPAR hitting the empty queue, while the slower (or more busy) LPAR gets the message. Not really a good load balancing.
But - again - just a guess. Maybe someone from IBM can explain why.
btw - how do you get that "Mr Butcher wrote:" instead of the "Quote:" ?!?
_________________
Regards, Butcher |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Apr 18, 2012 4:53 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
On z/OS mq, the initq is also shared, with multiple qmgrs - each with a trigger monitor.running.
The presumption is that multiple qmgrs with multiple potential consumers are running.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Apr 18, 2012 5:11 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| PeterPotkay wrote: |
| Why would you want to count uncommitted messages - they're not 'real' yet. It maybe a long time before they are commited. They may never commit and instead rollback. Why do they count them? But on Shared queues they don't. |
You do have to count them because they are part of the qdepth count.
So on trigger first -- how do you ensure a trigger message gets created if multiple apps put to the queue and the one causing the trigger to fire gets rolled back, but now queue depth is > 1 and the trigger never fires?
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Wed Apr 18, 2012 5:22 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
| PeterPotkay wrote: |
| Why do they count them? But on Shared queues they don't. |
Two questions here.
As to the first question: I suspect that the qmgr doesn't literally (need to) count the number of uncommitted messages; rather, it is the presence (or absence) of uncoms that is important to the triggering mechanism.
Let's suppose that there are hundreds or thousands of queues. It would make life easier for the triggering software to ignore queues that contain no messages at all. Conversely, it would be valuable for the triggering mechanism to pay attention to queues with queues with uncommitted messages in anticipation of them being committed soon.
If I were to write a triggering mechanism, I'd probably set a flag in the queue (uncoms) to distinguish this queue as one needing attention soon.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Wed Apr 18, 2012 8:37 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
| Mr Butcher wrote: |
| btw - how do you get that "Mr Butcher wrote:" instead of the "Quote:" ?!? |
Click the Quote button in my post and you'll see a quote tag that looks like
[quote="PeterPotkay"] instead of [quote]
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
|
|
|
  |
Goto page 1, 2 Next |
Page 1 of 2 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
