| Author |
Message
|
| vradhik |
 Posted: Tue Feb 17, 2009 3:58 am Post subject: MQ Triggering |
 |
|
Novice
Joined: 08 Nov 2008
Posts: 10
|
Hi
I have 2 questions :
1) What ways do we adopt for triggering on Mainframes by MQ. Is it a JCL triggering or any other forms of triggering available?
2) If we have to trigger this job on arrival of every message and we plan to limit the number of parallel runs of the same job to a particular number, is there anything in particular that we need to consider? Can we do parallel MQGETs on the same queue by different calls of the same job? Also, what will happen if the max number of parallel runs has reached (as per our configuration) and a new message arrives? I assume it will have to be handled by the application code?
We plan to have a status table which will be updated as soon as a job is triggered due to arrival of a message and while it is processing, if another message arrives, the status table will be checked for the number of instances currently running and accordingly, another instance of job will be started and will perform MQGET on the same queue.
Thanks |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Tue Feb 17, 2009 4:00 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
|
| Back to top |
|
|
| zpat |
| Posted: Tue Feb 17, 2009 4:01 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
You can use the CICS trigger monitor if your application is a CICS transaction. I would recommend this over batch triggering.
In the event of using batch triggering - I cannot see a need for than one batch job to run at the same time processing the same queue.
Use trigger FIRST and have the application process the queue until it is empty.
Get a mainframe expert involved in this design - do not treat z/OS as simply a large PC or UNIX platform. It is entirely different.
Do not design your own operating system like controls - WLM, z/OS and JES2 have more than enough facilities to achieve anything needed. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Feb 17, 2009 4:05 am Post subject: Re: MQ Triggering |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| vradhik wrote: |
| I have 2 questions : |
Rather more than 2 it seems! 
| vradhik wrote: |
| 1) What ways do we adopt for triggering on Mainframes by MQ. Is it a JCL triggering or any other forms of triggering available? |
Trigger by batch or CICS
| vradhik wrote: |
| 2) If we have to trigger this job on arrival of every message and we plan to limit the number of parallel runs of the same job to a particular number, is there anything in particular that we need to consider? |
Not using trigger every? That wll produce one started application per message.
| vradhik wrote: |
| Can we do parallel MQGETs on the same queue by different calls of the same job? |
Yes
| vradhik wrote: |
| Also, what will happen if the max number of parallel runs has reached (as per our configuration) and a new message arrives? I assume it will have to be handled by the application code? |
Unless you're controlling it via CICS (and it sounds like you're using batch). AFAIK if you trigger an application via batch, you'll get one application per message. Nothing the application can do will prevent this.
| vradhik wrote: |
| We plan to have a status table which will be updated as soon as a job is triggered due to arrival of a message and while it is processing, if another message arrives, the status table will be checked for the number of instances currently running and accordingly, another instance of job will be started and will perform MQGET on the same queue. |
This is not trigger every. This is a home brew solution that doesn't appear to use MQ triggering. It also seems (at best) blindingly ineffecient and (at worst) doesn't work.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| vradhik |
| Posted: Wed Feb 18, 2009 9:11 am Post subject: |
|
|
Novice
Joined: 08 Nov 2008
Posts: 10
|
Thanks for the replies.
| Quote: |
| This is not trigger every. This is a home brew solution that doesn't appear to use MQ triggering. It also seems (at best) blindingly inefficient and (at worst) doesn't work. |
The reason we have this control is to avoid possible resource contention. However, we do need the trigger every so that we are able to provide a very short turn around time. Do you suggest some other approach? |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Feb 18, 2009 10:48 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| vradhik wrote: |
| The reason we have this control is to avoid possible resource contention. |
Contention with what?
| vradhik wrote: |
| However, we do need the trigger every so that we are able to provide a very short turn around time. |
These points occur:
- if you have trigger(every) you'll get an application started for each message that arrives. If this status table of yours determines "too many" are running what happens? The excess instance(s) are cancelled? What then processes the messages for which no application is started?
- what is the expected processing time for the messages? Specifically how much does the processing time exceed the speed with which messages will arrive on the queue?
- if your design is intended to process messages as soon as they arrive, this design fails as soon as "too many" instances are running and messages start to pile up on the queue.
| vradhik wrote: |
| Do you suggest some other approach? |
Depending on the answers above of course, you could simplify your design by starting as many as instances as the "resource" will allow and leave them waiting for messages. Unused instances will be paged out by the system and paged back as required.
If you're wedded to a more "technical" solution, use monitoring and/or depth triggers to start more messages if the queue starts to back up.
Best of all, follow the suggestion of zpat and get your z/OS architecture people involved. I'd be surprised if this "resource contention" was an insoluable problem. Especially if the application can be tweaked.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| vradhik |
| Posted: Thu Feb 19, 2009 3:17 am Post subject: |
|
|
Novice
Joined: 08 Nov 2008
Posts: 10
|
| Quote: |
| Contention with what? |
Because these jobs access and insert into db2 tables, spawning n jobs might lead to a contention is what we thought (we may be wrong though  , need to verify our assumption)
| Quote: |
| if you have trigger(every) you'll get an application started for each message that arrives. If this status table of yours determines "too many" are running what happens? The excess instance(s) are cancelled? |
Yes.
| Quote: |
| What then processes the messages for which no application is started? |
Am I right here to assume that they will get processed when another new message arrives (and the number of instances running is less than n) and triggers the job and does a mqget? The pitfall here could be that if no message arrives for a long time after the n instances of the application has reached and there are pending messages, they will have to wait 
| Quote: |
| what is the expected processing time for the messages? |
approx a minute
| Quote: |
| Specifically how much does the processing time exceed the speed with which messages will arrive on the queue? |
a very valid point, need to check on this.
Thanks for all those points for us to consider. After answering your questions, i am more inclined towards - 'starting as many as instances as the "resource" will allow'. However, we will also try to get a z/OS architect involved.
This discussion has really given some good points for us to think about. Thanks a ton. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Feb 19, 2009 3:22 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| vradhik wrote: |
Because these jobs access and insert into db2 tables, spawning n jobs might lead to a contention is what we thought (we may be wrong though , need to verify our assumption)
|
If that is a problem then you need to speak to your DBAs about locking strategies. It's also better for the application to retry the insert than abort the transaction.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Feb 19, 2009 3:29 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| vradhik wrote: |
| Quote: |
| if you have trigger(every) you'll get an application started for each message that arrives. If this status table of yours determines "too many" are running what happens? The excess instance(s) are cancelled? |
Yes. |
Causing the messages to roll back onto the queue with an increased backout count, and wasting all the resources used to trigger and start the application along with any processing or database time used.
| vradhik wrote: |
| Am I right here to assume that they will get processed when another new message arrives (and the number of instances running is less than n) and triggers the job and does a mqget? |
Only if they're coded incorrectly! An application using a trigger(every) is typically coded to read a single message (the message causing the trigger). There's no point it reading any other messages because, from a design standpoint, all the other messages have been read off by all the other triggered applications.
If they do read one message and then look for the next, then you might as well use trigger(first) or just start a long running job.
| vradhik wrote: |
| The pitfall here could be that if no message arrives for a long time after the n instances of the application has reached and there are pending messages, they will have to wait |
Exactly.
| vradhik wrote: |
| After answering your questions, i am more inclined towards - 'starting as many as instances as the "resource" will allow'. |
If your contention is this fear of database locks, you might be well served to do some stress testing. You might not need as many instances as you think.
| vradhik wrote: |
| This discussion has really given some good points for us to think about. Thanks a ton. |
Happy Trails! 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Feb 19, 2009 4:26 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| Vitor wrote: |
| Only if they're coded incorrectly! An application using a trigger(every) is typically coded to read a single message (the message causing the trigger). There's no point it reading any other messages because, from a design standpoint, all the other messages have been read off by all the other triggered applications. |
However BEST PRACTICE says that any application coded for trigger every should be coded as if you were using trigger first: i.e. process messages until queue empty. This becomes specially critical with trigger every and resource throttling (max of x instances running in parallel).
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Mr Butcher |
| Posted: Thu Feb 19, 2009 6:59 am Post subject: |
|
|
Padawan
Joined: 23 May 2005
Posts: 1716
|
yes, this is best practice, and there is a special case that shows why it should be used...
in case the trigger monitor was not started BEFORE the message arrive on the queue, MQ will create a trigger message when the trigger monitor is started. but only one, even if there are many messages on the queue.
So... in that case .... if the every-triggered application only consumes one message then you will never get an empty queue. new messages will create new triggers, but the application will process old messages, not the new arrived ones.
in addition, for trigger every, there is no such mecanism as for trigger first that mq creates additional trigger messages because you left a queue non empty after the last close ....
_________________
Regards, Butcher |
|
| Back to top |
|
|
|
|
