| Author |
Message
|
| shediiwork |
 Posted: Mon Jul 25, 2005 5:29 am Post subject: Best Practice Question |
 |
|
Acolyte
Joined: 07 Jun 2005
Posts: 52
|
Guys,
I am looking to understand and implement the best way to store messages on a queue. Then trigger a process to gather all the message bodies from the messages on that queue into one message body. The trigger would need to be actived by a script, scheduler, timer or cron job. Being new to MQ development I'm not sure how do this so any help would much appreciated.
TIA
Shed. |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Mon Jul 25, 2005 5:46 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| shediiwork wrote: |
| Being new to MQ development |
I'm confused.
Are you using Broker, or plain MQ?
I guess, either way, the actual steps to develop are relatively straight foward.
You are trying to implement message aggregation - depending on the nature of the aggregation and the size and quantity of the messages... using a queue as a datastore may not be your best option.
You should consider using a database instead.
You should also strongly consider your edge cases : - when does a new aggregation set (new group) start?
- When does one end?
- Do you know if a group is incomplete?
- How long can a group stay incomplete?
- etc. etc.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| shediiwork |
| Posted: Mon Jul 25, 2005 6:50 am Post subject: |
|
|
Acolyte
Joined: 07 Jun 2005
Posts: 52
|
| I meant to say Broker. I'm fine with the use of a database to store messages. Would it be best to control the edge case with the triggering process. For example, run a process that ends a aggregation set and starts the next set. So, the message that the process executes is always last in group. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Jul 25, 2005 6:58 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| shediiwork wrote: |
| Would it be best to control the edge case with the triggering process. |
I don't know.
It depends on the business process in question...
Are you supposed to aggregate all messages until a certain time? Are you supposed to aggregate all messages that belong to particular entities or match certain criteria? Are you supposed to aggregate all messages in a given input group?
For a smallish group size (less than ten messages or so) you might be able to use the aggregation nodes to handle this. Otherwise, I'd build a flow to insert records into a database, and a flow to aggregate records from the database. The insert flow could act to trigger the aggregate flow based on certain conditions. You can also schedule any number of tools or a custom process to write fixed or dynamic messages to a queue to trigger the aggregation flow.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| shediiwork |
| Posted: Mon Jul 25, 2005 8:00 am Post subject: |
|
|
Acolyte
Joined: 07 Jun 2005
Posts: 52
|
| Quote: |
| Are you supposed to aggregate all messages until a certain time? |
The business process for Production/QA/DEV calls for a daily/hourly/as needed so the insert flow cannot control the trigger. Base on that model its difficult to determine the size. Two flows was my first thought my problem is how to implement the second flow in a control matter to extract messages from the database.
| Quote: |
| You can also schedule any number of tools or a custom process to write fixed or dynamic messages to a queue to trigger the aggregation flow. |
What tools are there? |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Jul 25, 2005 8:13 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| shediiwork wrote: |
| Quote: |
| You can also schedule any number of tools or a custom process to write fixed or dynamic messages to a queue to trigger the aggregation flow. |
What tools are there? |
Anything that can write a message to a queue and can be invoked by your scheduler.
Amqsput/putc would do the job for small messages. The q program from a support pack (I forget which) would do the job. CapitalWare's MQBatchToolkit is designed for exactly such things. And there's probably dozens more.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| shediiwork |
| Posted: Mon Jul 25, 2005 9:49 am Post subject: |
|
|
Acolyte
Joined: 07 Jun 2005
Posts: 52
|
| Thanks Jeff, thought I would just pose some question before heading too depth into unfamiliar territory. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Jul 25, 2005 10:11 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| shediiwork wrote: |
| The business process for Production/QA/DEV calls for a daily/hourly/as needed so the insert flow cannot control the trigger. |
The insert flow could certainly check the current time against a rules table or compare against the last batch time to see if it's been an hour, and check the input message to see if it's a "batch up now" control message rather than an input message.
But if you've got an active, monitored and managed scheduling system, you may want to use it to write messages and change the batch system to send back a status reply.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| shediiwork |
| Posted: Mon Jul 25, 2005 10:27 am Post subject: |
|
|
Acolyte
Joined: 07 Jun 2005
Posts: 52
|
| Quote: |
The insert flow could certainly check the current time against a
rules table or compare against the last batch time to see if it's been an hour, and check the input message to see if it's a "batch up now" control message rather than an input message.
|
Right, I could use the batch time (current and last) as my select constraints. I was thinking along the same line for the input message.
I was thinking of use ReplyToQueue on the "Batch up Now" to store status reply since have no monitoring in place, yet. |
|
| Back to top |
|
|
|
|
