| Author |
Message
|
| Miguel Sanders |
 Posted: Mon Dec 15, 2008 11:55 pm Post subject: Question on Triggering |
 |
|
Newbie
Joined: 15 Dec 2008
Posts: 2
|
Hello all
I have a question concerning the triggering functionality. I have been able to set up everything correctly, however I would like your advice on the following:
Let's say that messages on a local queue (triggering enabled, for every message) represent orders which need to be sent to an external supplier. If, however, a certain order cannot be delivered, the following orders should not be delivered as long as the erroneous order is not successfully delivered (FIFO).
I am having the impression that the triggering monitor has no knowledge on the successful/unsuccessful return of the triggered application. Is this correct? If so, how can the above FIFO processing be guaranteed?
Thanks for your feedback
Miguel |
|
| Back to top |
|
 |
| exerk |
| Posted: Tue Dec 16, 2008 1:11 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
The question to ask may be: How does each individual instance of your application know another instance is processing a message? What happens if you receive 5 messages and 1, 2, 3 and 5 are correctly processed but 4 is held up? If you trigger on first, and the application is sequentially removing messages, no problem that I can see - if the application fails no more messages get processed until recovery actions are implemented.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| zpat |
| Posted: Tue Dec 16, 2008 1:15 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| You could open the queue with input exclusive to avoid parallel processing. |
|
| Back to top |
|
|
| exerk |
| Posted: Tue Dec 16, 2008 1:20 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| zpat wrote: |
| You could open the queue with input exclusive to avoid parallel processing. |
Would that cause the other instances to fail? Or if the application logic allows a wait state, introduce contention, thereby soaking resource on the server?
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| zpat |
| Posted: Tue Dec 16, 2008 1:39 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
It would cause the other MQOPEN with MQOO_INPUT_EXCLUSIVE to fail with MQRC_OBJECT_IN_USE.
What the application does depends on how it handles error codes. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Dec 16, 2008 1:41 am Post subject: Re: Question on Triggering |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Miguel Sanders wrote: |
| If, however, a certain order cannot be delivered, the following orders should not be delivered as long as the erroneous order is not successfully delivered (FIFO). |
What you're describing is known as message affinity and is a bad thing. You've found one of the problems with it. You're better off designing it out.
| Miguel Sanders wrote: |
| I am having the impression that the triggering monitor has no knowledge on the successful/unsuccessful return of the triggered application. Is this correct? If so, how can the above FIFO processing be guaranteed? |
Note that the trigger monitor is not involved in your applicaiton. It will start your application when a message arrives but after that it's down to your application to deal with the message.
I'm assuming here you're using a FIRST trigger. Attempting FIFO with any other doesn't make sense.
Opening the queue as exclusive would stop other applications reading the queue, but still leaves your application with the problem. What is your requirement when a message can't be processed? Should the unprocessed message be left on the input queue and re-attempted at intervals? Should the application shut down and not be retriggered until the problem has been manually resolved? In the first example all you need is a sleep timer of some kind in your application. In the second you need to alter the attribute of the queue to disable triggering.
But the best solution is to remove the message affinity.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| exerk |
| Posted: Tue Dec 16, 2008 1:46 am Post subject: Re: Question on Triggering |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| Vitor wrote: |
| ...I'm assuming here you're using a FIRST trigger. Attempting FIFO with any other doesn't make sense... |
| Miguel Sanders wrote: |
| ...(triggering enabled, for every message)... |
Hence zpat's answer I think...
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Dec 16, 2008 1:57 am Post subject: Re: Question on Triggering |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| exerk wrote: |
| Vitor wrote: |
| ...I'm assuming here you're using a FIRST trigger. Attempting FIFO with any other doesn't make sense... |
| Miguel Sanders wrote: |
| ...(triggering enabled, for every message)... |
Hence zpat's answer I think... |
Doh! 
Opening exclusive with trigger every is still odd advice.
It's worth repeating the advice here about triggering on every message being a really bad idea unless you have a throttling mechanism (like CICS) in place.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Miguel Sanders |
| Posted: Tue Dec 16, 2008 2:05 am Post subject: |
|
|
Newbie
Joined: 15 Dec 2008
Posts: 2
|
Hello
I am aware of the fact that the message affinity is a bad thing.
However, since this is caused by the design of the application, there is not really much I can do about it unfortunately.
So I think I have to find a way to use interprocess communication so that the instances know each other's state. |
|
| Back to top |
|
|
| exerk |
| Posted: Tue Dec 16, 2008 2:14 am Post subject: |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| Miguel Sanders wrote: |
| ...So I think I have to find a way to use interprocess communication so that the instances know each other's state. |
And how will they know which is the 'master' instance, i.e. the first one triggered? If one fails, how are you going to ensure the others 'below' it roll back? It might be easier, and quicker, to trout the designer and start again.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Dec 16, 2008 2:26 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Miguel Sanders wrote: |
| So I think I have to find a way to use interprocess communication so that the instances know each other's state. |
So if you've got 2 instances, they handshake to determine which is processing the "first" order. The second one then (presumably) waits until that is successfully processed. What happens if you've got 5 instances? 10? 100? You'll potentially have n instances sending (n-1) squared messages trying to work out who's got the first order.
The other way to do it is to have a lookup table on a database holding the last processed order id. If an instance reads an order with a higher instance it waits. But if you're doing that, why not break the affinity and store the entire order in a database until the earlier orders have been processed?
The only other viable way to do this, with message affinity in place, is to use FIRST not EVERY and single thread the orders. Because of their affinity they can only ever be processed in a single thread so trying to multi-thread is pointless and likely to break either the business process or the machine.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Dec 16, 2008 3:30 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
And you still need to DESIGN for the poison message!
Nothing in your design is telling us how you want to handle the poison message! Remember your app is triggered. So as soon as you close the queue another trigger is being generated...
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| zpat |
| Posted: Tue Dec 16, 2008 3:44 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
I never mentioned trigger every. Trigger first would be normal.
For higher volume applications, a "listener" style of interface is much better. Keep it running all the time.
There is still a risk of inadvertantly starting more than one instance of the listener. One way to avoid that is to use exclusive open. Another way is to obtain serialisation via another method like a database row lock. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Dec 16, 2008 3:54 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| fjb_saper wrote: |
So as soon as you close the queue another trigger is being generated...
|
Hence my comment that the app would have to disable triggering if it was going to shut down when a poison message was detected.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
