| Author |
Message
|
| Sam Uppu |
 Posted: Mon Feb 08, 2010 7:37 pm Post subject: Design issue |
 |
|
Yatiri
Joined: 11 Nov 2008
Posts: 610
|
Hi Guys,
We are in the design phase. The requirement is as below.
The messages will be arriving on the input queue from 2:00Am to 4:00AM(lets assume) and these should get processed during that timeframe.
The messages might also come before 2:00Am. But these should not get processed until 2:00AM.
Application sends a start message(at 2:00AM) which indicates the start of the messages processing.
So once we get the start message we should process all the messages.
I was thinking of a design like this:
Use browse option on the MQinput node and check if it is a start message. If yes, set a temperory variable X to True. If not just browse the message, do not read it.
When the next messages arrives check if X is true. If true the message is retrieved using MQGET node and processed further.
For those messages which arrived earlier to start message, will only be browsed and not read. I will also set the reset browser option to process the messages which arrived before the start message.
Let me know your thoughts and suggestion on the above design.
Thanks |
|
| Back to top |
|
 |
| WMBDEV1 |
| Posted: Mon Feb 08, 2010 11:17 pm Post subject: |
|
|
Sentinel
Joined: 05 Mar 2009
Posts: 888
Location: UK
|
Id probably have a separate trigger queue to read the start message via an MQInput node rather than your polling / browsing approach. You can then use your MQGet node as before (making sure not to loop at the flow level of course  )
Other options equally valid |
|
| Back to top |
|
|
| WMBDEV1 |
| Posted: Tue Feb 09, 2010 3:03 am Post subject: |
|
|
Sentinel
Joined: 05 Mar 2009
Posts: 888
Location: UK
|
Also, thinking about this a little more, depending on how many messages you are expecting on the queue and how tolerant of outages you can be; it may be worth processing the messages off the queue into a temporary store (eg a DB) and then when the trigger message is receievd you could read all the data from the DB and process as needed.
This solution is probably more applicable if you are worried about a lot of messages ending up on queues though and is really intended to be more food for though |
|
| Back to top |
|
|
| Sam Uppu |
| Posted: Tue Feb 09, 2010 5:30 am Post subject: |
|
|
Yatiri
Joined: 11 Nov 2008
Posts: 610
|
| WMBDEV1 wrote: |
Id probably have a separate trigger queue to read the start message via an MQInput node rather than your polling / browsing approach. You can then use your MQGet node as before (making sure not to loop at the flow level of course )
Other options equally valid |
If we use MQ input node for reading a start message, how do we get all the messages at a time using MQGET node(as we use MQGET in the same flow/subflow)? Our start message comes only once. And also we ge a stop message after which we should stop processing. |
|
| Back to top |
|
|
| WMBDEV1 |
| Posted: Tue Feb 09, 2010 6:10 am Post subject: |
|
|
Sentinel
Joined: 05 Mar 2009
Posts: 888
Location: UK
|
| Sam Uppu wrote: |
If we use MQ input node for reading a start message, how do we get all the messages at a time using MQGET node |
Have a compute node before the MQGet that keeps on propagating until you reach the stop message / empty queue?
And actually, maybe your "stop" message should be your trigger as this is when you know all the messages are there? |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Feb 09, 2010 6:47 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
I've had to solve this problem once before. Here is my solution.
I created a separate QMGR to hold all the messages but not in a normal queue.
Eh? you might be thinking?
The separate QMGR receives all the messages via a remote Q def from the broker QMGR. IN this QMGR, there was another remote Q Def that basically turned the message around and sent it back to the Broker QMGR.
The trick here is that each flow of data had its own channel from this qmgr to the Broker QMGR. This would be normally in a stopped state.
A batch/cron job would run at the appropriate time whoose only job was to start the channel thus releasing the queue of messages to the flow in the broker. A while later, another job was run to stop the channel.
IMHO, this is simple and reliable to implement and does not involve any tricks in the broker. The only hard coded times are in the batch/cron job definition.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Feb 09, 2010 6:55 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Wow. That's the long way around it.
I'd entirely use an MQInput node pointed to a Collector node, and use the stop message to finish the collection.
I can't see a good reason for distinguishing between "messages are sitting on a public external queue waiting to be processed" and "messages are sitting on the Collector node's internal queue waiting to be processed". |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Feb 09, 2010 9:04 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
| mqjeff wrote: |
Wow. That's the long way around it.
I'd entirely use an MQInput node pointed to a Collector node, and use the stop message to finish the collection.
I can't see a good reason for distinguishing between "messages are sitting on a public external queue waiting to be processed" and "messages are sitting on the Collector node's internal queue waiting to be processed". |
It was in the days before the collector node...
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Tue Feb 09, 2010 10:41 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7722
|
Assuming the sending app is on another QM....
For WMBDEV's solution:
What if the start message arrives before the first application message?
For mqjeff's solutions:
What if the stop message arrives before the last application message?
Maybe message grouping is a better option? Until the entire group arrives, none of the messages will be picked up?
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Sam Uppu |
| Posted: Tue Feb 09, 2010 12:07 pm Post subject: |
|
|
Yatiri
Joined: 11 Nov 2008
Posts: 610
|
The messages may arrive like the below.
10 messages before start message.
Start Message.
100 messages after start message.
Stop Message.
The order of processing is also improtant. ie., the messages should be processed from 1 - 100.
Can we change any property on MQInput node at runtime?
I was thinking like if the condition for a start mesage gets satisfied, I need to reset the browse option on MQInput node to the first message.
So that the messages are processed in order of sequence. |
|
| Back to top |
|
|
| Sam Uppu |
| Posted: Fri Feb 12, 2010 12:16 pm Post subject: |
|
|
Yatiri
Joined: 11 Nov 2008
Posts: 610
|
Hi Guys,
I am planning fora design something like this
I am having some like this
Timeoutnotification ----->MQGETNode------->compute------->MQOut
MQINPUT (start/stopmsgs)------->COMPUTE------>Timeoutcontrol
MQInput node gets start/stop messages. Based on the start or stop, I will generate acontrol message to start getting the messages from MQGETNODE.
I have a query here.
If I get a start message and I issue a control message to get messages using MQGET node, will only one message is read?
Will the MQGET node read all the mesages until I generate a control command with Action CANCEL?
If a control message is sent to timoutnotification node to start a flow at specified time, will the flow be running until we send a different control command to stop? |
|
| Back to top |
|
|
|
|
