| Author |
Message
|
| alethea |
 Posted: Mon Jul 30, 2007 2:35 am Post subject: Dynamically get message from various input queues |
 |
|
Apprentice
Joined: 30 Jul 2007
Posts: 45
|
Hello.
First of all thanks in advance for viewing or answering this.
Here is what I am looking for:
I up to creating a single process that retrieves all the messages from various input queues and processing them periodically. I have to deal with maximum 100 different input queues in this single flow.
Can anyone please tell me how to dynamically get message from various input queues using MQGet node or by any other way.
Thanks |
|
| Back to top |
|
 |
| Vitor |
| Posted: Mon Jul 30, 2007 2:45 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
Can you clarify your requirement? Why are there so many different queues if all the messages are processed through a single flow?
Either build a single flow with 100 input nodes, or define 100 alias queues and point the flow at a single input point! 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| alethea |
| Posted: Mon Jul 30, 2007 3:03 am Post subject: |
|
|
Apprentice
Joined: 30 Jul 2007
Posts: 45
|
Actually this process should retrieve the messages from all the different Backout queues(which I called as 100 different input queues) and reprocess those messages to be put back to their respective Input queues for processing again.
The names of the Input Queues and their backout queues are similar except a prefix "BCQ_" is to be added to the name of the Input queue to make it a Backout queue.
And this whole process i.e dealing with 100 different backout queues one at a time should be in a single flow.
Feel free to ask any other question if you have.
Thanks. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Jul 30, 2007 3:11 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| alethea wrote: |
Actually this process should retrieve the messages from all the different Backout queues(which I called as 100 different input queues) and reprocess those messages to be put back to their respective Input queues for processing again.
The names of the Input Queues and their backout queues are similar except a prefix "BCQ_" is to be added to the name of the Input queue to make it a Backout queue.
And this whole process i.e dealing with 100 different backout queues one at a time should be in a single flow.
Feel free to ask any other question if you have.
Thanks. |
Well you can have 100 MQinput nodes and dynamically set the output queue on the MQOutput node depending on the inputq...
Or you could get rid of the backout queues altogether and run a DLQ handler to retry the messages on the DLQ?
Anyways I do hope that somebody had a look at the messages on the Backout queues and you are not just randomely reprocessing.... (What makes you think that what failed the first time will eventually succeed without anything changing (environmental like DB availability or programmatical like code) )?
I believe you need to revisit the design... 
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Jul 30, 2007 3:14 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| alethea wrote: |
Actually this process should retrieve the messages from all the different Backout queues(which I called as 100 different input queues) and reprocess those messages to be put back to their respective Input queues for processing again.
|
How do you know they will work this time? How do you stop them failing again, going into the backout queue again, and ending up with an infinate loop? If you want this kind of retry technology, use the backout count on the original queue; the backout queue should be the final resting place of failed messages where trained humans pick through the bones to determine the problem.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| alethea |
| Posted: Mon Jul 30, 2007 3:23 am Post subject: |
|
|
Apprentice
Joined: 30 Jul 2007
Posts: 45
|
| The messages on the first back out queues should be automatically resubmitted by this process on a periodic basis and potentially multiple times. If the message has been successfully processed then it is transferred to the output destination otherwise the second back out queue will receive any messages from this process that could not simply reprocess automatically. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Jul 30, 2007 3:31 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
Why 2 backout queues? I repeat, if you just want messages resubmitted a few times to deal with transient problems, use the backout count on the original input queue designed for this purpose. Only put things on a backout queue when you've convinced yourself (by however many attempts you deem necessary) that it's not going to work.
But if you want to stick with this strange & bizarre design, code 100 input nodes. Don't use the DLQ for this and if you decide you are going to use a DLQ for whatever reason don't use the one defined to the queue manager. That should be left for undeliverable messages not messages with an application issue.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| alethea |
| Posted: Mon Jul 30, 2007 3:38 am Post subject: |
|
|
Apprentice
Joined: 30 Jul 2007
Posts: 45
|
I have this requirement to retry messages a defined number of times after a specified time interval. The delay is needed because of a business process or the availability of a back-end system or database.
And therefore i need to dynamically get message from various backout queues in a single process. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Jul 30, 2007 3:54 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| alethea wrote: |
| I have this requirement to retry messages a defined number of times after a specified time interval. The delay is needed because of a business process or the availability of a back-end system or database. |
This is not an unusual requirement.
| alethea wrote: |
And therefore i need to dynamically get message from various backout queues in a single process. |
This is a strange and bizarre way to achieve it. But you know your system best, you asked for advice & we've given it. Not the best solution (I'd go nuts after the 1st 20 nodes) but it ought to work even if it clunks a bit.
Good luck to you.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| alethea |
| Posted: Mon Jul 30, 2007 4:14 am Post subject: |
|
|
Apprentice
Joined: 30 Jul 2007
Posts: 45
|
Thank you Vitor and fjb_saper.
I understand that its an usual requirement. But 100 nodes in one flow will be a very bad design.Though I will try to go with this initially.
Please let me know if anyone of you come across a good way of achieving this.
Thanks once again. |
|
| Back to top |
|
|
| Vitor |
| Posted: Mon Jul 30, 2007 4:28 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| alethea wrote: |
| But 100 nodes in one flow will be a very bad design. |
Indeed. As I said, I'd go crazy trying to implement it. But it's the logical outcome of how you've chosen to handle this.
| alethea wrote: |
Please let me know if anyone of you come across a good way of achieving this.
|
That would be the method we suggested. 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Jul 30, 2007 11:33 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
If you really need to process messages from many many queues, but do the same thing to all of them, regardless of the queue.
Then deploy many copies of the same flow, but adjust the configurable properties to change the input queue and the output queue.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Mon Jul 30, 2007 1:56 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
I would go as far as to let the messages go to the system DLQ. Have a DLQ handler there move them to the app DLQ.
Run at the interval a DLQ handler on the app DLQ that retries once.
Check what's left in the app DLQ retry again...
After that it's all manual ...
And be careful... due to timing and size you can still have a vicious circle...
Enjoy 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Mon Jul 30, 2007 1:59 pm Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
I'd rather use alias queues for BOQs on flows, all pointing to the same queue, than ever use the system DLQ for app error messages.
That's just me, though.
And obviously one can't do this if one needs to route messages back to the original queue, unless one can know what the original queue was from the message itself.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
