| Author |
Message
|
| mc01285 |
 Posted: Fri Aug 13, 2010 11:33 am Post subject: Flow Design Assistance |
 |
|
Novice
Joined: 11 May 2009
Posts: 11
|
I have messages coming in on a MQ queue that need to update a database.
I only want to process the messages if the database is available. If the database is not available I want to somehow delay the processing for a period of time and then retry.
Anybody have design pattern?
Thanks for the help!
Mark |
|
| Back to top |
|
 |
| Gaya3 |
| Posted: Fri Aug 13, 2010 11:40 am Post subject: |
|
|
Jedi
Joined: 12 Sep 2006
Posts: 2493
Location: Boston, US
|
couple of questions to be cleared
1. Whats the Volume?
2. Do you want a delta load or full load?
Design 1.
1. Get the message from MQ
2. Try Connecting to Database
2.1. If not available, try for another 3[keep this threshold as minimum] times again, if it is still not available, keep your message in another queue and throw an error desc. trigger it after 5 mins[Keep this threshold depends upon your business criteria, but keep it as long as you could, as keep on trying DB is not a good idea] do the same process, if it is failing, throw an error
with proper desc
2.2. if is available, insert the data to database.
Well all depends upon the volume again 
_________________
Regards
Gayathri
-----------------------------------------------
Do Something Before you Die |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Fri Aug 13, 2010 11:40 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
I wouldn't do what you are trying to accomplish, but if I would...
Use transactionality, and in the unhappy path, put some code (maybe using a JCN) to get disable the queue and/or stop the message flow.
You could have another flow that periodically checks the status of the database and re-enable the first flow/queue when database comes available.
Ultimately you are trying to use Broker to manage operational status of infrastructure componetry and there are much better ways/tools to do this. WebSphere Business Monitor is one. Tivoli Omeggamon is another. BMC Patrol is a third.
Good luck.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| inMo |
| Posted: Fri Aug 13, 2010 11:58 am Post subject: |
|
|
Master
Joined: 27 Jun 2009
Posts: 216
Location: NY
|
Just to join the party ... If you go so far as to have the periodic flow check database availability & start 2nd flow if available, why not also do the flip, have it shutdown 2nd flow if not available.
Another technique ... perhaps put the flow to sleep for a few seconds, then cause your error, rollback. Ensure your queue settings allow for reprocessing (I think it is backout threshold needs to be higher then 1). The flow will automatically pickup the message and retry the logic (DB interaction). |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Fri Aug 13, 2010 12:22 pm Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
I saw a guy try to change the oil in his car with a hammer once.
My dad used to tell me: choose the right tool for the job.
Toolman Tim Taylor from Home Improvement always got bested by his co-hort, Al.
I would encourage you to take a holistic view of the world, and implement a strategic solution that manages database state and application state the right way.
My favorite always was HP OpenView which is now HP Operations Manager software. If you have money, you could buy Tivoli.
If you have no money, you could try JMX tooling that is a standard part of all JREs.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| AkankshA |
| Posted: Sun Aug 15, 2010 8:09 pm Post subject: |
|
|
Grand Master
Joined: 12 Jan 2006
Posts: 1494
Location: Singapore
|
Also note that all these solutions provided will take away the sequencing of messages...
hence specify all the requirements you have and must need .. to get the viable option
_________________
Cheers |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Mon Aug 16, 2010 6:02 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
@ AkankshA
Good morning.
AkankshA said:
| Quote: |
all these solutions provided will take away the sequencing of messages
|
What do you mean by this?
The solution I suggested does not loose messages and the sequence of messages is maintained.
In further discussion, I recommended a choice of proper tools to accomplish the task specified (managing application and database state).
Trying to manage application state and database state in a broker flow is mis-use of the broker flow.
What do you mean in your statement?
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| SAFraser |
| Posted: Mon Aug 16, 2010 1:40 pm Post subject: |
|
|
Shaman
Joined: 22 Oct 2003
Posts: 742
Location: Austin, Texas, USA
|
| I don't see how a "proper" monitoring tool helps this fellow with handling the exception-type situation when a DB is down. It may help everyone to know that the DB needs attention, but it doesn't help with the immediate issue of what to do with his incoming messages. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Aug 16, 2010 3:17 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| SAFraser wrote: |
| I don't see how a "proper" monitoring tool helps this fellow with handling the exception-type situation when a DB is down. It may help everyone to know that the DB needs attention, but it doesn't help with the immediate issue of what to do with his incoming messages. |
A proper monitoring tool could issue a command to stop the flow when the database is known to be down.
Of course, a proper scheduling tool could do that just as easily. |
|
| Back to top |
|
|
| SAFraser |
| Posted: Tue Aug 17, 2010 4:15 am Post subject: |
|
|
Shaman
Joined: 22 Oct 2003
Posts: 742
Location: Austin, Texas, USA
|
I use a proper monitoring tool. It won't stop a flow "right out of the box". I can script it for such, to be sure. But it's a monitoring tool. It's purpose is to monitor.
A scheduling tool would be a better choice.
But perhaps the OP is not the guy who gets to decide what they buy. I don't get to decide what we get to buy. I can say my opinion, but I don't sign the checks. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Aug 17, 2010 4:53 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| SAFraser wrote: |
I use a proper monitoring tool. It won't stop a flow "right out of the box". I can script it for such, to be sure. But it's a monitoring tool. It's purpose is to monitor.
A scheduling tool would be a better choice.
But perhaps the OP is not the guy who gets to decide what they buy. I don't get to decide what we get to buy. I can say my opinion, but I don't sign the checks. |
Without going to far afield, though, I expect slightly more out of my monitoring tools than JUST monitoring. As you said, it could LET SOMEONE KNOW. So, Monitoring plus Alerting. But Alerting is really Taking An Action in response to a condition.
Whether that action is as simple as 'send a page' or more elaborate like 'run a script' is a difference of extent and not of kind. It's still taking an action.
But I don't think this particular situation calls for action at this level. Unless the database is known to be turned off on a given interval or known to be disabled for a long period of time.
For transient issues, either normal MQ rollback and reprocessing is appropriate, or something perhaps involving the Timeout nodes or ESQL SLEEP if it's available. |
|
| Back to top |
|
|
| mc01285 |
| Posted: Tue Aug 17, 2010 5:20 am Post subject: |
|
|
Novice
Joined: 11 May 2009
Posts: 11
|
Thanks for the great posts/discusssion!
I'm leaning toward going with "get inihibit"ing the queue with a JCN and then creating TimeNotification flow that would check the database and turn the queue back on using the same JCN class. |
|
| Back to top |
|
|
| Amitha |
| Posted: Tue Aug 17, 2010 6:53 am Post subject: |
|
|
Voyager
Joined: 20 Nov 2009
Posts: 80
Location: Newyork
|
|
| Back to top |
|
|
|
|
