| Author |
Message
|
| prasadpav |
 Posted: Tue Oct 04, 2005 4:50 am Post subject: Resubmitting backed-out messages |
 |
|
Centurion
Joined: 03 Oct 2004
Posts: 142
|
Hi,
I'm designing an application in which all message flow input queues has a backout queue & backout threshold specified. In case of database being down or other resource unavailable situations, the message is retried 3 times before the message is put to backout queue. Later sometime, if I want to resubmit the backed-out message from backout queue to the original queue, I dont have the input queue name from which the message is originally backed-out. Couple of options I can think of to get the input queue in the message:
1) Have some sort of identifier in MQRFH2/<usr> folder to identify the kind of message and from there get the input queue. But this will fail if more than one message flow use the same identifier
2) Have one backout queue for each input queue. But I guess this would be a crazy solution.
Any better ideas to implement this??
Thanks
Prasad |
|
| Back to top |
|
 |
| sieijish |
| Posted: Wed Oct 05, 2005 8:04 am Post subject: |
|
|
Acolyte
Joined: 29 Nov 2004
Posts: 67
Location: London
|
If you can't uniquely identify the input queue from the message data, then the only option is to put that information before you route your message to the backout/Error queue.
Put a filter node in the mail flow to check for backout count and if it exceeds 3, add the inputq name to the msgs & route it to error/backout queue. |
|
| Back to top |
|
|
| wschutz |
| Posted: Wed Oct 05, 2005 8:22 am Post subject: |
|
|
Jedi Knight
Joined: 02 Jun 2005
Posts: 3316
Location: IBM (retired)
|
| Quote: |
| Have one backout queue for each input queue. But I guess this would be a crazy solution. |
Why would that be a crazy solution? It would double the number of queues, but that might no be so bad. And then there's no question about which was the original input queue.
_________________
-wayne |
|
| Back to top |
|
|
| hopsala |
| Posted: Wed Oct 05, 2005 8:39 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| wschutz wrote: |
| Quote: |
| Have one backout queue for each input queue. But I guess this would be a crazy solution. |
Why would that be a crazy solution? It would double the number of queues, but that might no be so bad. And then there's no question about which was the original input queue. |
I agree with wayne here, having a backout queue for each input queue is quite normal, even customary - I know a few sites that work this way.
In fact, it's the best way to achieve this, it means that if backout queue fills, then only one application is affected, and that you don't have to mess about with the message itself; To me it seems the most logical, robust and easy-to-maintain solution. |
|
| Back to top |
|
|
| prasadpav |
| Posted: Thu Oct 06, 2005 1:10 am Post subject: |
|
|
Centurion
Joined: 03 Oct 2004
Posts: 142
|
| Quote: |
| Why would that be a crazy solution? It would double the number of queues, but that might no be so bad. And then there's no question about which was the original input queue. |
I thought this would be a crazy solution because it increases the number of queues to be maintained. These backout queues are not used very often, except when there is an error. So, till then it is just a maintainence overhead. Also if someone want to monitor the backout queues for depth and alert particular staff, then they need to monitor on all the backout queues. Wouldn't that be simple in case of just one backout queue? May I also know the numbers, the number of backout queues that are created in your system? Just want to know how high you can go with number of backout queues without maintenance overhead and ease of monitoring them.
I rather liked the solution of checking the backout count on the input message, modify the RFH2 Header to include the necessary information and then put that on the backout queue using MQOUTPUT node.
Prasad |
|
| Back to top |
|
|
| hopsala |
| Posted: Thu Oct 06, 2005 5:43 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| prasadpav wrote: |
| I rather liked the solution of checking the backout count on the input message, modify the RFH2 Header to include the necessary information and then put that on the backout queue using MQOUTPUT node. |
I have to disagree with you on this, communication data and buisness data are two different things, and never the two shall meet; if you use your message (being a buisness message, MQMQD included!) as a port for comm data, some day it will come back and bite you in the keister.
| kiranraj wrote: |
| because it increases the number of queues to be maintained |
Contrary to common opinion, it is quite easy to maintain any number of queues if you have a decent monitoring product (as it is implicitly stated that you have); there's really no overhead to more queues, they are stable, peaceful entities, which like to sit by the fireplace day and night and warm their virtual bones. Channels (and thus QMs) are the true perpetrators of the peace here, which indeed have a tendancy to require maintenance.
There's a little known design principle I like to call "keep everything in front" which I learned the hard way; it means that everything should be on the first screen, or one click away - the rarer the need to use something, the further it should be in the background.
In this particular case, this means that communication data (or "relational data" - who relates to who) should be out there for all to see with the greatest possible ease; putting fields such as "where did this come from" deep in your RFH is not what I call easy access.
Anyway, just my 2 cents... |
|
| Back to top |
|
|
| prasadpav |
| Posted: Tue Oct 18, 2005 3:17 am Post subject: |
|
|
Centurion
Joined: 03 Oct 2004
Posts: 142
|
Hi,
Thanks for your suggestions. I decided to move forward with the idea of having different backout queues for each Input queue. I've one question though around replaying (or better called moving the message from backout queue to the original input queue). I know that there are some support packs available which can move messages from one queue to another queue. I'm interested to share your experiences of replaying the backout messages using these support packs.
I'm planning to have a message flow which reads from each backout queue and then build the output queue name based on backout queue name and then put the message to input queue. But obvious problem with this one being, the backout message is replayed immediately by message flow after the message is backed out. The backed-out message could potentially be a poison message and loops forever....
Any comments or suggestions are highly appreciated.
Thanks,
Prasad |
|
| Back to top |
|
|
| sanu_mit |
| Posted: Tue Oct 18, 2005 3:58 am Post subject: |
|
|
Apprentice
Joined: 03 Jul 2003
Posts: 25
Location: Kolkata
|
1. Regarding your problem of identifying the input queue of a message in a given message flow, you can get it from the SourceQueue field of MQMD. You can access this field as InputRoot.MQMD.SourceQueue. You store this value in the usr folder of MQRFH2 and then use it sebsequently to replay the message.
2. Regarding your issue to detect poisonous message, you have to do it through an error handler. The error handler has to be devised in such a way that it will analyse the ExceptionList of the message and then detect the message as poisonous or non-poisonous. |
|
| Back to top |
|
|
| hopsala |
| Posted: Tue Oct 18, 2005 4:56 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| prasadpav wrote: |
| I know that there are some support packs available which can move messages from one queue to another queue. I'm interested to share your experiences of replaying the backout messages using these support packs. |
I personally favor MA01, others use VisualEdit, IH03, or some other tools. Search the forum for these and you'll find useful comments.
| prasadpav wrote: |
| I'm planning to have a message flow which reads from each backout queue and then build the output queue name based on backout queue name and then put the message to input queue. But obvious problem with this one being, the backout message is replayed immediately by message flow after the message is backed out. The backed-out message could potentially be a poison message and loops forever.... |
There's an interesting design option at hand, that may be beneficial - you can use the dead letter queue header as your basic structure for problem determination: whenever your appl comes across a problematic message, it builds a MQDLH with the RC that you defined denotes "too large" or "first field messed up" etc; this way you can decide whether to replay the message, or deleted it or whatever. Better yet, you should probably put some index in the MQDLH which points to the exact place in the application error log where the error message + exception list appears.
In the MQDLH you also have the original MQMD and the source queue (the queue the message came from) - but I still say it will be much easier for you to manage one backout queue per appl queue. The nice thing here is, that you can use the normal dead letter queue handler (runmqdlq) with your backout queue, so no need to code.
All this aside, you must compile a list (a general draft) of possible reasons that messages should arrive to the backout queue, and decide what possible actions you may wish to apply (replay, discard, report to admin, manual intervention etc); I believe you're thinking about this too abstractly, and that you'll probably find that no messages in backout queue may be automatically handled - all manually. If this is indeed the case, then forget about writing code or using runmqdlq, just use some tool like those I mentioned earlier... |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Oct 18, 2005 5:05 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
I prefer to leave the DLQ for system level stuff, and not misuse it for application level stuff.
However, there's nothing that says you can't make another qlocal, and "pretend" it's a DLQ, and do all the same stuff, including putting the DLH and etc.
I'm just not sure that the DLH is the best format for general purpose errors.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| Mr Butcher |
| Posted: Tue Oct 18, 2005 6:24 am Post subject: |
|
|
Padawan
Joined: 23 May 2005
Posts: 1716
|
i would prefer 1 backout queue per application queue, then it is clear where the message came from and where to put for retry.
otherwise i would go with the dlqh solution from jefflowrey. you can then use the dlq handler to put them back to the queue for retry (dont go into a loop  )
_________________
Regards, Butcher |
|
| Back to top |
|
|
| hopsala |
| Posted: Tue Oct 18, 2005 9:19 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| jefflowrey wrote: |
I prefer to leave the DLQ for system level stuff, and not misuse it for application level stuff.
However, there's nothing that says you can't make another qlocal, and "pretend" it's a DLQ, and do all the same stuff, including putting the DLH and etc. |
That's what I meant - I did not intend that he should use the DLQ itself, but the application backout queues as a type of DLQ with DLH headers; I hope I wasn't unclear on this in my post...
| jefflowrey wrote: |
| I'm just not sure that the DLH is the best format for general purpose errors. |
Why not? It has everything you need, and you already have a ready-made program to handle it (runmqdlq), aside from even more sophisticated dead letter queue handlers obtainable on the net; is there any critical field missing in the DLH? |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Tue Oct 18, 2005 9:27 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| hopsala wrote: |
| is there any critical field missing in the DLH? |
Room for 2000 characters of Java stack trace...
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| hopsala |
| Posted: Wed Oct 19, 2005 2:03 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
| Don't you think it's sufficient to have an index to the application log where the stack trace appears, as I suggested? |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Wed Oct 19, 2005 3:40 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| hopsala wrote: |
| Don't you think it's sufficient to have an index to the application log where the stack trace appears, as I suggested? |
It's somewhat problematic to maintain an application error log in WBIMB. Well, it's not, but people don't think about it in the same way. You've got a good idea, or at least the start of one. Have you looked at the work IBM has done with Common Event Infrastructure?
My point is more general, though. The MQDLH is designed to report on errors within the MQ system, and is not designed specifically for reporting application level errors. Consequently, it may be more useful from a lot of criteria to develop a schema or etc. for error messages coming out of in-house applications.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
