| Author |
Message
|
| cvag |
 Posted: Thu Jan 10, 2013 10:31 pm Post subject: Suggestion on Flow Design |
 |
|
Centurion
Joined: 17 Mar 2008
Posts: 127
|
Need suggestion on flow design:
I have to design two Main message flows using MQ transport protocols ---
| Code: |
| 1) Trigger Flow 2) Status Flow. |
1) Everytime,Trigger flow will activate or trigger the Status flow. At first attempt means after deployment we manually put the message into the Trigger Flow MQInput Queue, then and thereafter the flow has to continuously run without any manual intervention.
2) QueueName of MQOutput node belonging to Trigger Flow is the Input to the Status Flow MQInput QueueName.
3) QueueName of MQOutput node belonging to Status Flow is the Input to the Trigger Flow MQInput QueueName.
4) There are cases where the Status Flow might get delayed in processing and putting the message into the MQOutput Queue or it may get failed due to some exception. At that point Trigger Flow will not activate, as there is no message from Status Flow.
a) Here we have a requirement, Trigger Flow has to check for the message in the MQInput Queue, if the message does'nt exist it should wait for 10sec, if still doesn't have Trigger Flow should automatically has to create some message atleast Header part to activate the Status Flow.
I developed using Timer nodes, but the requirement has changed that it has to be done without Timer nodes. I checked for several options for events on the Queue but its becoming complex design.
Need help on this. 
Thanks in Advance. |
|
| Back to top |
|
 |
| smdavies99 |
| Posted: Fri Jan 11, 2013 12:01 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
Why has the requirement changed so that you can't use timer nodes?
These are part of the functionaluty of Broker and have been for a long time?
Then read my post of a few days ago. http://www.mqseries.net/phpBB2/viewtopic.php?t=63133
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| mqsiuser |
| Posted: Fri Jan 11, 2013 12:40 am Post subject: Re: Suggestion on Flow Design |
|
|
Yatiri
Joined: 15 Apr 2008
Posts: 637
Location: Germany
|
| cvag wrote: |
| I developed using Timer nodes, but the requirement has changed that it has to be done without Timer nodes. |
You could write cron jobs that use amqsput to put messages on queues (based on time-intervalls) and trigger flows that way.
_________________
Just use REFERENCEs |
|
| Back to top |
|
|
| kash3338 |
| Posted: Fri Jan 11, 2013 1:43 am Post subject: Re: Suggestion on Flow Design |
|
|
Shaman
Joined: 08 Feb 2009
Posts: 709
Location: Chennai, India
|
From the design you have explained, the Status flow just triggers the Trigger Flow and the message is not used in it by any means. Hence your option of using a MQIput Node for the Trigger Flow should be dropped and make use of the Timer Node in 'Automatic' mode (10 sec interval).
That is the purpose of Timer Nodes.
| cvag wrote: |
I developed using Timer nodes, but the requirement has changed that it has to be done without Timer nodes. |
Why should it be done with out Timer Nodes?
| cvag wrote: |
I checked for several options for events on the Queue but its becoming complex design. |
The use of Timer Nodes are just to reduce such complex designs. |
|
| Back to top |
|
|
| neWMBGuy |
| Posted: Fri Jan 11, 2013 7:49 am Post subject: |
|
|
Novice
Joined: 01 Aug 2012
Posts: 19
|
use shell script (thru cron) to Trigger the flow with MQ message.
if you need help with script i can share script to use amqsput... |
|
| Back to top |
|
|
| visasimbu |
| Posted: Fri Jan 11, 2013 9:13 am Post subject: |
|
|
Disciple
Joined: 06 Nov 2009
Posts: 171
|
Shell script will be one of the option, if timer is not available.
Write the shell script with following check.
check the last put time of the Trigger flow input queue. If it is greater than 10 sec put dummy message in the Trigger flow input queue to activate the trigger flow and followed by the status flow.
Shell script can be triggered in every 10 sec using UNIX scheduler Cron or use the Control M Job.
Last edited by visasimbu on Fri Jan 11, 2013 9:28 am; edited 1 time in total |
|
| Back to top |
|
|
| NealM |
| Posted: Fri Jan 11, 2013 9:24 am Post subject: |
|
|
Master
Joined: 22 Feb 2011
Posts: 230
Location: NC or Utah (depends)
|
Are you on MB8? Here's a thought:
1. Leave your status flow as-is.
2. On the trigger flow, change that MQInput queue to an MQGet with a 10 second timeout inside an endless loop (that contains at a minimum the MQGet and MQOutput nodes, and maybe a Compute node if that status message is not just a constant).
OK, so how do you start/restart the trigger flow on a broker restart without using a timer node? Add a new MQInput node to the flow for that purpose, for a new queue, maybe named AlwaysThere.Q, and put a starter trigger message (permanently) on that queue right after you create the queue. On the new MQInput node Properties, check the Browse Only checkbox and leave the browse timeout at -1. Where you would wire the MQInput node's entry into the endless loop would depend on whether you want to first let the MQGet check for a status message already on it's queue before sending a message to the status flow.
Oh, and your MQOutput node had better have its Transaction mode property set = No.
Mind you, I haven't tried this, but I have used the MQGet node before as a great way to pause a flow instance for a set period of time.
If you aren't allowed to do that, nor use Timer nodes, then I don't see any other option than a cron triggered shell script, but that would fire every 10 seconds regardless of what the status flow was doing. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Fri Jan 11, 2013 9:37 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| NealM wrote: |
| On the trigger flow, change that MQInput queue to an MQGet with a 10 second timeout inside an endless loop (that contains at a minimum the MQGet and MQOutput nodes, and maybe a Compute node if that status message is not just a constant). |
Yuck. Cron is much better solution than this. Think of all the wasted CPU and I/O on this silly scheme.
Some banks pay for system resources per CPU tick. Your scheme would ring up quite a bill.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| NealM |
| Posted: Fri Jan 11, 2013 9:49 am Post subject: |
|
|
Master
Joined: 22 Feb 2011
Posts: 230
Location: NC or Utah (depends)
|
| Quote: |
| Your scheme would ring up quite a bill |
Not really. It would die a painful death. (Neal cleans up his loopy post from last week here after a good weekend's rest). See below for something better.
Last edited by NealM on Mon Jan 14, 2013 8:29 am; edited 1 time in total |
|
| Back to top |
|
|
| NealM |
| Posted: Mon Jan 14, 2013 8:08 am Post subject: |
|
|
Master
Joined: 22 Feb 2011
Posts: 230
Location: NC or Utah (depends)
|
Well, did you ever wake up in the morning with the thought, not "Voila!" but rather, "Oh crap!" ? Well, I did today, the oh crap is for my endless loop proposal; I forgot that, from experience, the Broker doesn't like endless loops. It isn't too many iterations before it blows its stack. Literally.
So, since I'm idling at 37,000 feet at present, I thought about it some more, and there is a rather simple solution, same approach as my loopy loop, without the loop. It still requires one extra queue, the TRIGGER_CTL.Q, whence cvag's post-deploy initial msg goes. And an MQGet with a 10 second wait interval. Here is a picture of it in its simplest form:
No cron needed, no Timer nodes needed. But it meets ALL cvag's requirements.
(Update: The trigger flow has a serious wiring error. See my new picture several posts further down for a corrected and improved version)
Last edited by NealM on Tue Jan 15, 2013 7:52 pm; edited 1 time in total |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Mon Jan 14, 2013 8:13 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
|
| Back to top |
|
|
| smdavies99 |
| Posted: Mon Jan 14, 2013 8:14 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
Nice pictures however,
(there is always a but, or some idiot who asks the question you hoped that no one would ask...)
What happens when your HACMP system fails over in the middle of the upper flow operation? No a controlled failover but the 'oh shit, I just pulled out the wrong mains lead' sort of failure? will it restart itself all of its own accord?
These are the sort of questions you have to ask yourself when doing stuff like this. How robust is thie solution?
I have lost count of the number of wierd and wonderful solutions used to delay messages for a period that I have seen over the years. Some worked in an HACMP environment and a good few didn't. It takes a good developer to think about this sort of stuff whereas a normal developer will just make it work on their dev env and that is it... Job done.
Whereas the TimerNode solution will carry on when the broker on the backup/standby server starts up.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| NealM |
| Posted: Mon Jan 14, 2013 8:33 pm Post subject: |
|
|
Master
Joined: 22 Feb 2011
Posts: 230
Location: NC or Utah (depends)
|
Well, as I said, I met all the poster's requirements, which, if you remember, stated no Timer nodes, and never mentioned failover. (And regarding which, if your failover setup involves running a script, which many do, then no problemo, that, a persistent ctl.q, and maybe throw in some global cache.....)
The point is, with the Broker there is almost always more than one solution for a situation. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Jan 15, 2013 5:39 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
I'm missing something. Each of the 2 flows' Output Nodes feeds the other's Input node. How is this not an endless loop?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| NealM |
| Posted: Tue Jan 15, 2013 6:52 pm Post subject: |
|
|
Master
Joined: 22 Feb 2011
Posts: 230
Location: NC or Utah (depends)
|
| Quote: |
| I'm missing something. Each of the 2 flows' Output Nodes feeds the other's Input node. How is this not an endless loop? |
1. From the standpoint of a message flow thread, there is no loop at all. Each flow does its thing and ends, thus freeing up all flow resources being tied up for backtracking a failure, etc. And that part is why a true endless loop blows its stack.
2. From the standpoint of the flows repeating ad infinitum, yes that is the requirement. And if you look again, you see that the trigger flow doesn't even need the status flow to keep running, with (in that case) a new thread replacing the old ended thread every 10 seconds.
But, like I said, I pasted the flow in its simplest form. I would expect that a real implementation would include some way (of many choices that come to mind) of stopping the flow, and of restarting it again when needed.
There is a more serious flaw though that has gone unnoticed. The way I wired it in the "pretty picture" (  ), If the Status flow dies and doesn't place a new message on its output queue, the trigger flow would never send another message to it. I should have instead wired the "No message" terminal on the MQGet to behave like the "Out" terminal. With maybe an MQGet-Browse on the status input queue to make sure it actually needed a new message.....
Well, I did say I was in-flight at the time. I'll blame it on the thinner air. |
|
| Back to top |
|
|
|
|
